# Distributed Multi-objective Optimization in Cyber-Physical Energy Systems

Sanja Stark<EMAIL_ADDRESS>OFFISEscherweg 2OldenburgGermany26121 ,
Emilie Frost<EMAIL_ADDRESS>OFFISEscherweg 2OldenburgGermany26121 and
Marvin Nebel-Wenner<EMAIL_ADDRESS>OFFISEscherweg
2OldenburgGermany26121

(2023; 01 December 2023)

###### Abstract.

Managing complex Cyber-Physical Energy Systems (CPES) requires solving various
optimization problems with multiple objectives and constraints. As distributed
control architectures are becoming more popular in CPES for certain tasks due
to their flexibility, robustness, and privacy protection, multi-objective
optimization must also be distributed. For this purpose, we present MO-COHDA,
a fully distributed, agent-based algorithm, for solving multi-objective
optimization problems of CPES. MO-COHDA allows an easy and flexible adaptation
to different use cases and integration of custom functionality. To evaluate
the effectiveness of MO-COHDA, we compare it to a central NSGA-2 algorithm
using multi-objective benchmark functions from the ZDT problem suite. The
results show that MO-COHDA can approximate the reference front of the
benchmark problems well and is suitable for solving multi-objective
optimization problems. In addition, an example use case of scheduling a group
of generation units while optimizing three different objectives was evaluated
to show how MO-COHDA can be easily applied to real-world optimization problems
in CPES.

Distributed multi-objective optimization, Cyber-Physical Energy Systems

††copyright: acmlicensed††journalyear: 2023††doi: XXXXXXX.XXXXXXX††journal:
TOG††journalvolume: 37††journalnumber: 4††article: 111††publicationmonth:
8††ccs: Computing methodologies Multi-agent systems††ccs: Computing
methodologies Self-organization††ccs: Hardware Smart grid††ccs: Computing
methodologies Optimization algorithms

Availability of Data and Material:

The scenario used for the evaluation is available at:
https://github.com/Digitalized-Energy-Systems/MOO-
CPES/releases/tag/Distributed_Multi-objective_Optimization_in_Cyber-
Physical_Energy_Systems. The resulting data can be found at:
https://zenodo.org/records/10245786.

##  1\. Introduction

A significant number of Distributed Energy Resources (DER) are introduced into
the power system as a result of the energy transition. Many of the DER are
based on renewable resources, such as wind and solar energy. In the future,
these will account for the majority of the electricity supply while the
generation from conventional power plants will decline. As a result, power
generation will no longer be purely demand-driven but also depend on the
weather, season, and time of day. At the same time, the growing number of
electric vehicles, heat pumps, and storage systems can change consumption
patterns. In particular, distribution grids require greater automation to
effectively incorporate DER and manage the overall increasing system
complexity, which calls for advanced Information and Communication Technology
(ICT). ICT subsequently becomes an essential and integral component of the
energy system, giving rise to a CPES.

The efficient and safe planning and operation of CPES can be described by
various optimization problems with multiple objectives and constraints.
Exemplary problems are managing energy systems on microgrid (Karimi and Jadid,
2020; Khezri and Mahmoudi, 2020) or household level (Mayer et al., 2020),
optimal placement and sizing of DER (Ramli et al., 2018; Nowdeh et al., 2019)
or system restoration (Almoghathawi et al., 2019; Stark et al., 2021). The
objectives include minimizing environmental effects or $CO_{2}$ emissions,
minimizing cost, minimizing power loss, and maximizing stability or
reliability.

Due to potential conflicts between these objectives, one optimal solution
usually does not exist but rather a set of trade-off or Pareto-optimal
solutions, where the performance of one objective cannot be increased without
reducing the other. Finding a Pareto front can be accomplished by population-
based metaheuristics such as evolutionary or swarm-based methods. These
algorithms search different regions of the solution space simultaneously and,
with few modifications, can find a set of multiple non-dominated solutions in
a single run (Konak et al., 2006). Many different approaches from both of
these algorithm classes have already been used in the context of energy system
optimization, for example, particle swarm (Naderi et al., 2021), slime mold
(Khunkitti et al., 2021), glow-worm (Salkuti, 2019) or grey wolf (Wu et al.,
2020) optimization from the swarm-intelligence group and NSGA-2 (Wang et al.,
2020; Ren et al., 2019; Ruiming, 2019; Sharma et al., 2020) and MOGA (Ullah et
al., 2021) from the evolutionary group.

All these approaches use a central solver in which all necessary information
is assumed to be available in one place. However, in the case of CPES,
distributed control architectures have become more popular for certain tasks.
In this case, multiple control elements have the joint responsibility for
deciding on control actions to fulfill defined objectives (Han et al., 2017).
A distributed control allows plug-and-play of DER, battery storages, or loads
while at the same time being robust due to no single-point-of-failure (Espina
et al., 2020). Therefore, distributed architectures offer more flexibility and
robustness compared with centralized ones (Ren et al., 2021). Moreover,
distributed approaches allow for preserving the privacy of DER and loads by
storing technical details locally and only exchanging necessary information
with other entities. This is particularly critical as it is common in CPES for
some information to be only available distributed rather than collected at a
central point.

To perform optimization processes with distributed architectures, a Multi-
Agent Systems (MAS) is a suitable choice (Ren et al., 2021). MAS-based
optimization in CPES has not only been studied in the context of microgrid
management (Karimi and Jadid, 2020; Khezri and Mahmoudi, 2020) but also, for
example for the electricity market (Luo et al., 2018) or power system
restoration (Yang et al., 2021). We therefore extended the work from Bremer et
al. in (Bremer and Lehnhoff, 2019) and developed a fully distributed, agent-
based algorithm to solve multi-objective optimization problems of CPES. Due to
the diversity of optimization problems in CPES, we have ensured the
extensibility and flexibility of the algorithm to allow easy adaptation to any
problem at hand. Our goal is to provide a basic framework that allows to fully
exploit the advantages of an agent-based multi-objective optimization – e.g.
having different agents with different solving strategies, which can also be
dynamically adapted during the optimization and for the specific problem,
e.g., taking into account the computational power each agent has.

This paper has the following contributions:

  1. (1)

Introducing MO-COHDA: a fully distributed agent-based multi-objective
optimization algorithm, that is capable of solving distributed optimization
problems with a potentially large number of decision variables where each
entity only has limited control of one decision variable.

  2. (2)

Evaluating the effectiveness of MO-COHDA by comparing it to Non-dominated
Sorting Genetic Algorithm (NSGA-2) using multi-objective benchmark functions
from the Zitzler–Deb–Thiele (ZDT) problem suite.

  3. (3)

Showing with an example use case for scheduling a group of generation units
with regards to three different objectives how MO-COHDA can be easily applied
to real-life optimization problems in CPES.

The remainder of the paper is organized as follows. Section 2 gives an
overview of related work in the context of distributed multi-objective
optimization. Section 3 describes the proposed multi-objective algorithm.
Section 4 presents the evaluation of the algorithm with benchmark problems and
Section 5 with a CPES problem. Finally, Section 6 gives a short summary of the
results and describes possible future work.

##  2\. Related Work

Several approaches in the literature deal with multi-objective optimization in
CPES. CPES are inherently decentralized, as the power is created primarily by
small generating units close to the customers instead of central large power
plants. For the coordinated optimization of hybrid AC/DC distribution
networks, three categories are distinguished in (Wang et al., 2021), which can
be transferred to optimization problems in CPES: First, in centralized
optimization, data is collected to perform decisions centrally. Second, in
distributed optimization, multiple instances coordinate to reach a collective
decision, and each instance controls its object. Finally, decentralized
optimization is a state between centralized and distributed control. Since the
distinction between centralized and distributed optimization is sufficient for
our use cases, we divided the related work into centralized and distributed
multi-objective optimization.

###  2.1. Centralized Multi-objective Optimization

Adiou et al. present a multi-objective optimization for the operation of
decentralized multi-energy systems (Adihou et al., 2019). The objectives
considered are the optimal design, maximizing the renewable energy rate, and
minimizing costs. The authors use a genetic algorithm for their approach.
Additionally, Murray et al. deal with decentralized multi-energy systems,
where the focus is Power-to-Mobility (Murray et al., 2020). The approach
minimizes the costs and the life-cycle emissions of buildings and vehicles. A
mixed integer linear program is used.

Wang et al. consider optimizing a decentralized integrated energy system in
northern rural areas (Wang et al., 2022). Two objectives are regarded: the
economy (including different kinds of costs) and environmental protection
(minimizing pollutants). The optimization is done with NSGA-2.

Using central optimization requires all the necessary information to be
centrally available. This is not always feasible in CPES due to the large
number of optimization variables as well as data privacy reasons. Therefore, a
distributed algorithm for the optimization is suitable, as it does not rely on
global information (Ren et al., 2021).

###  2.2. Distributed Multi-objective Optimization

A distributed multi-objective optimization is done by Ren et al. (Ren et al.,
2021). The authors develop a distributed multi-objective algorithm for ice
storage systems, combining particle swarm optimization with a differential
evolution algorithm. The optimization goals are minimizing energy consumption,
operation cost, and energy loss. In this approach, different nodes, called
computing process nodes, take over the optimization task while exchanging
information with other nodes. The nodes in this example calculate their local
knowledge, make decisions, and then exchange information with others. Thus,
the optimization does not rely on global information. Another example is the
work from Wang et al. in (Wang et al., 2021). Using the alternating direction
multiplier method, they perform a coordinated multi-objective optimization
regarding operational costs, voltage deviation, and network losses in Hybrid
AC/DC Flexible Distribution Networks. The authors present their multi-
objective optimization algorithm and discuss the advantages but do not compare
the effectiveness with benchmark functions or the suitability for other
applications by properties such as flexibility and expandability.

Regarding optimizations in the power system, Combinatorial Optimization
Heuristic for Distributed Agents (COHDA) (Hinrichs and Sonnenschein, 2017) is
well suited because it is fully distributed. COHDA has multiple advantages
relevant to the usage in a power system, as it ensures convergence and
termination. The heuristic is implemented in a MAS, as it is well suited for
this due to its distributive nature. An often considered use case is the
scheduling of DER (Hinrichs and Sonnenschein, 2017; Nieße and Tröschel, 2016;
Hinrichs et al., 2013), but COHDA has already been used for various use cases
and with multiple extensions and adaptations in each case (Stark et al., 2021;
Hinrichs et al., 2015; Holly and Nieße, 2021; Bremer and Lehnhoff, 2017). The
heuristic considers privacy constraints by keeping local information, such as
information about different DER units, private to the distributed agents. Only
the best decision is communicated to other agents during the optimization
process. COHDA was already considered by Schrage et al. using multiple
criteria (Schrage et al., 2023b). The authors designed an evolutionary
algorithm to generate optimized schedules concerning multiple criteria,
integrating the local objective. Additionally, Bremer et al. consider COHDA
for multi-objective optimization while implementing an S-metric selection
algorithm into it (Bremer and Lehnhoff, 2019). This allows fully distributed
agent-based multi-objective scheduling for energy resources. The approach
presented in this paper is based on the one proposed by Bremer et al. and
extends it. Furthermore, we applied the optimization of multiple global
targets, while Bremer et al. considered only local targets. Our extension is
explicitly explained in Section 3.

##  3\. MO-COHDA

In this work, we extended COHDA to apply it to multi-objective problems. We
will refer to this version as MO-COHDA. In the following, we will briefly
describe the mechanisms of COHDA and then present the adaptions we introduced
in MO-COHDA.

###  3.1. Preliminary Work: COHDA and basic MO-COHDA

The heuristic used in this paper is called COHDA. It was introduced by
Hinrichs and Sonnenschein in (Hinrichs and Sonnenschein, 2017). COHDA is a
fully distributed optimization heuristic that uses self-organization
mechanisms for the optimization of a common target.

The key concept of the heuristic is an asynchronous and iterative best-
response behavior of distributed agents. Every single agent has a working
memory that is exchanged with other agents. The working memory contains the
global target function, the most up-to-date information about the decisions of
all agents in the system, and a solution candidate for the optimization
problem. Once an agent receives a message containing the working memory of
another agent, its algorithmic approach includes three steps:

  1. (1)

Perceive: The agent updates information about the current decisions of other
agents. The agent replaces its own solution candidate in case the candidate
its neighbor suggests contains more elements or yields a better rating.

  2. (2)

Decide: The agent then searches for its best solution considering its local
constraints, the information about the current system state of other agents,
and the global target. If the resulting system state yields a better rating
regarding the global target than the current solution candidate, a new
solution candidate is created, replacing the old one.

  3. (3)

Act: If any component of the working memory has been modified, the agent sends
its working memory to its neighbors.

Bremer and Lehnhoff presented the first version of a multi-objective COHDA
approach for Pareto fronts in (Bremer and Lehnhoff, 2019). In their work, they
adapted COHDA in several ways based on the concept of the S-Metric Selection
Evolutionary Multi-Objective Algorithm (SMS-EMOA) (Beume et al., 2007):

  * •

Solution Representation: Instead of one solution, each agent stores a Pareto
front, including several different solutions (so-called individuals) within
its working memory.

  * •

Solution Quality Assessment: In order to compare the quality of two solution
candidates (i.e. two Pareto fronts approximations) they calculate the achieved
hypercube volume – or hypervolume –, a concept first introduced by Zitzler et.
al in (Zitzler and Künzli, 2004). The hypervolume measures the volume of the
space between the approximated Pareto front and a chosen reference point that
should be dominated by all Pareto-optimal solutions, thereby describing the
space dominated by the front. A larger hypervolume indicates closer
convergence towards the true Pareto front. For a graphical representation of
the hypervolume in a 2D space, see Appendix A.

  * •

Generation of new Solution Candidates: Following the approach of evolutionary
algorithms, new individuals are created from the old ones using selection,
mutation and crossover operators. For selection, individuals are randomly
selected and then mutated by adding a Gaussian delta to one of the agent’s
decision variables. Furthermore, as a crossover operator, a uniform crossover
is applied. Upon that, the agent performs a fast, non-dominated sort and
removes the worst individual. This procedure is repeated for several
iterations. Decoder functions, which abstract from individual capabilities as
introduced in (Bremer and Lehnhoff, 2018), were used to respect individual
constraints.

###  3.2. Adaptions in MO-COHDA

In this work, we further extend this approach by integrating the following
changes:

  * •

Fixed reference point: To calculate the hypervolume for comparing two solution
candidates, a reference point has to be selected, which should be dominated by
all Pareto-optimal solutions. This is necessary to ensure the same comparison
characteristics for each agent. In SMS-EMOA, this reference point is defined
using the vector of the current worst objective values increased by 1.0 (Beume
et al., 2007). Since the current worst objective values can change, the
reference point is usually recalculated for each iteration. However, differing
reference points may interfere with the convergence process of MO-COHDA. In
the decide step, a new solution candidate is only created when it receives a
better rating regarding the global target. Therefore, COHDA converges when no
agent can find a better configuration and all agents have the same solution
candidate. Since agents act asynchronously and can receive messages in any
order, any agent must reach the same decision regarding which candidate is the
best to guarantee this convergence, no matter in which order the solution
candidates are compared (transitivity). This transitivity is not given with a
dynamic reference point, which can result in deteriorative cycles (Berghammer
et al., 2012). See Appendix A for a simple example of this behavior. To
prevent this, we made it obligatory to choose the reference point initially at
the start of the optimization process. The drawback of a fixed reference point
is that the worst values for each objective need to be known beforehand.

  * •

Flexible integration of local and global constraints: Our approach provides a
flexible interface to integrate the local constraints of the agents. For
example, the resulting local flexibility could be represented as a discrete
set of possibilities, or it can be defined as a continuous range of feasible
values. Generally, any representation is supported as long as it is possible
(a) to get an initial feasible decision without any further inputs and (b) for
the mutation algorithm to integrate the given representation of flexibility.
Global constraints have to be communicated to all agents along with the global
target functions. The concrete implementation of constraints, e.g., not
allowing infeasible solutions altogether or the use of a penalty function, is
also flexible. Some constraints in CPES use cases might require more detailed
knowledge about other agents like their location, for example, if the total
power produced by units in a certain area should not exceed a defined
threshold. In this case, this information should be prepared so that only the
selected agents can interpret it, to continue to guarantee data protection and
privacy.

  * •

Flexible way of generating new solution candidates: Regarding the way new
solution candidates are generated (the decide step within COHDA), we offer a
flexible basic structure that allows different ways to create new individuals.
We distinguish between two parts within the process: pick and mutate. The pick
function is used to select any number of individuals from the current system
state any number of times, which will be taken as input for the mutation (for
example, pick one or pick all individuals). The mutate function will now
generate new individuals. It will receive the local constraints of the agent
as well as the formerly chosen individuals. Again, the concrete way the
mutation is implemented is flexible. However, only the decision variables of
the local agent can be mutated, and the local constraints must always be
respected. The result of the pick function is a set of new individuals.
Finally, a non-dominated sort is executed, and the worst individuals are
removed until the initial number of individuals is reached (reduce). This
procedure is repeated for a predefined number of iterations.

  * •

Minimal change for updates of other agents: We introduce the possibility of
setting a margin regarding the minimum improvement in hypervolume necessary to
update other agents. If a new candidate whose hypervolume was not
significantly improved (not above the selected amount) was found, no update to
other agents would be sent, and the candidate would be discarded.

Figure 1. Process of one iteration within decide

[An iterative process is shown with the 5 steps described in the text]The
activity diagram shows an iterative process, starting with x=0. If x is
smaller than the defined number of iterations, the process starts. It follows
the five steps that are described in the text, pick, mutate, reduce, calculate
hypervolume and update. For each step it is shown, which data objects are used
as input and which data objects are created as output. Pick takes the
SystemConfig as an input and creates Picked Individuals as an output. Mutate
takes the picked individuals as input and creates new individuals as output.
Reduce uses the SystemConfig and new individuals as input and creates new
front as output. Calculate hypervolume takes the candidate and the new front
as input. If the new front is better, update takes it as input and creates
updated candidate and sysconfig as output. x is increased by one and the
process starts over again. It is finished when x is equal to the number of
iterations.

The extensions lead to the following process in decide, as depicted in Figure
1 and listed in the following.

  1. (1)

Pick: In each iteration, the individuals to be mutated are chosen from all
solution points.

  2. (2)

Mutate: New individuals are generated by mutating the picked individuals using
the selected mutation function.

  3. (3)

Reduce: The set of new individuals and the original individuals from the
SystemConfig is reduced to the initial number of solution points, creating a
new front.

  4. (4)

Calculate Hypervolume: The hypervolume of the new front is calculated and
compared with the hypervolume of the solution candidate. If it is not higher,
the new front is discarded, and the next iteration begins with Step (1).

  5. (5)

Update current SystemConfig and current best front: If the new front has a
higher hypervolume, it replaces the old candidate, and the agent also updates
his SystemConfig accordingly. The next iteration considers the updated
information, beginning with Step (1).

In our implementation, solutions are always represented as ”schedules”, since
this is a common representation for energy system scenarios where a set of
DER, loads, battery storages, and other flexible elements are coordinated over
a period of time. One schedule is a time series of power values with one power
value per time interval. However, the implementation can also be used for
other representations. For example, if every agent controls just one variable
in an optimization problem, it is possible to have a schedule with a single
interval. This way, our implementation offers the flexibility and adaptability
to multiple use cases. This also includes tuning the other parameters that are
part of MO-COHDA, which are summarized in the following:

  * •

Number of solution points

  * •

Pick function

  * •

Mutate function

  * •

Minimal change

  * •

Number of local iterations

###  3.3. Contributions of MO-COHDA

MO-COHDA extends and improves the work from Bremer and Lehnhoff (Bremer and
Lehnhoff, 2019) in several aspects. While still using the core functionality
of the initial approach, MO-COHDA was explicitly developed to create a
flexible and expandable, fully distributed multi-objective optimization
heuristic that allows for a flexible definition of the agent’s internal
optimization strategies. All parts of the algorithm, including integration of
objectives, constraints, decision variable format, flexibility representation,
and optimization aspects, such as pick and mutate functions, can easily be
adapted to test the best performance for a given problem. By introducing
minimal change as an adjustable parameter, the convergence speed of MO-COHDA
can also be influenced in one more aspect.

All the parameters can be set for each agent individually, allowing the
testing of different optimization strategies in one optimization process.
Overall, this allows more detailed testing of optimal parameter settings of
the fully distributed multi-objective optimization concept for CPES and
addresses one of the points mentioned by Bremer and Lehnhoff as future work,
namely the design of the optimization parameters. It should be noted that any
performance improvement of MO-COHDA over the work from Bremer and Lehnhoff
would only come from different parameter settings – using the exact pick and
mutate functions that Bremer and Lehnhoff used, the performance would most
likely still be the same, as the core functionality of the SMS-EMOA was not
changed.

Finally, the MO-COHDA algorithm is open source and comes with several options
for parameter settings and use-case examples to make it easy to use. This
includes, for example, a set of different mutate functions, ranging from a
random mutation to a more advanced local optimization using NSGA-2. As MO-
COHDA is implemented within the multi-agent framework mango (Schrage et al.,
2023a), it can easily be combined with other functionalities offered in mango,
such as the termination detection or coalition formation.

##  4\. Evaluation - Benchmark Problem

To compare the performance of MO-COHDA with a central NSGA-2 solver, well-
known MO benchmark problems and their respective Pareto fronts – in the
following called reference fronts – are used. While it would also be possible
to represent an optimization problem with a single optimization variable in
MO-COHDA – by having each agent controlling only a part of this variable’s
search space – we chose only to use benchmark functions with multiple
optimization variables for comparison. This way, each agent is only
responsible for one variable, which best reflects the usual use case for
distributed optimization.

In this Section, we introduce the chosen benchmark functions, describe the
experimental setup in Subsection 4.2, and discuss the evaluation results in
Subsection 4.3.

###  4.1. Optimization Problems

We chose three functions from the ZDT problem suite for evaluating MO-COHDA as
they are well-known benchmark functions for MO optimization algorithms and
fulfill our requirement for considering multiple variables (Zitzler et al.,
2000). All problems are based on the same construction process and aim to
minimize two objectives:

(1) $\begin{split}\text{min }f_{1}(x)&\\\ \text{min
}f_{2}(x)&=g(x)h(f_{1}(x),g(x))\end{split}$

The following equations depict the chosen problems, namely ZDT1, ZDT2, and
ZDT3\. While the Pareto-optimal front from ZDT1 is convex, the front from ZDT2
is non-convex, and ZDT3 has five disconnected fronts. The functions $f_{1}(x)$
and $g(x)$ are identical for all three problems:

(2) $\begin{split}f_{1}(x)&=x_{1}\\\
g(x)&=1+\frac{9}{n-1}\sum_{i=2}^{n}x_{1}\\\ \end{split}$

The difference between the three problems lies in different $h(f_{1},g)$.

(ZDT1) $h(f_{1},g)=1-\sqrt{f_{1}/g}$

(ZDT2) $h(f_{1},g)=1-(f_{1}/g)^{2}$

(ZDT3) $h(f_{1},g)=1-\sqrt{f_{1}/g}-(f_{1}/g)\text{ sin}(10\pi f_{1})$

For all three problems $0\leq x_{i}\leq 1\text{and }i=1,...,n$ applies, with
n=30, resulting in 30 variables that need to be optimised. To solve these
problems in MO-COHDA, each variable is represented by one agent that knows the
allowed range between $0$ and $1$.

###  4.2. Experimental Setup

Here we present an example of a parameter setting that has led to good results
in the solution of the benchmark functions. The decision variables were
distributed among the agents to optimize ZDT1-3 in a distributed way. Each
agent is responsible for exactly one variable, leading to a total number of 30
agents. We represent the variables as one schedule with 30 entries, where each
agent can only change one variable. At the beginning of the optimization, the
agents do not know the values for any of the other agents’ variables. Hence,
they must make assumptions for the unknown variables, as the objectives can
only be evaluated with all 30 values. Typically, an agent would assume 0 in
this case. However, since the optimum for all ZDT functions is $0\leq
x_{1}\leq 1\text{ and }x_{i}=0\text{ for }i=2,…,n$, assuming 0 would yield a
very good initial performance, possibly distorting the results. Instead,
agents assume the value 1 for all unknown variables to start from bad initial
solutions. Another option would be to circumvent this issue altogether by
having the agents wait with the actual optimization until they know an initial
choice of each of the other agents.

The optimization process is now the following: When an agent receives its
first COHDA message, it chooses a random variable at its position in the
schedule within its allowed range (anything between 0 and 1). The agent then
updates its neighbors with only its own value changed. After this initial
participation, an agent proceeds with the normal decide process described in
Figure 1. In the pick function, all solution points are chosen to be mutated,
with the mutation function working as follows: for each picked solution point,
the agent decreases the value by subtracting a random value (minimum to 0) and
increases it by adding a random value (maximum to 1), thereby creating two new
solution points. The range for the random value is set to $0.4-0.6$ for all
three benchmark functions.

A Pareto front with 25 solution points is created, and the reference point is
set to $(1.1,6.9)$, considering the worst possible values for $f_{1}$ and
$f_{2}$ in all ZDT functions are $1.0$ and $6.8$, respectively. The number of
iterations is set to 1, as it seems due to the communication between the
agents, which triggers regular calls of pick\- and mutate, the optimization
process is efficient even with just one iteration per decide. The only
parameter not identical for all three functions is minimal change, which is
set to $0.0001$ for ZDT1 and $0.001$ for ZDT2,3 to achieve the best results.

To evaluate the performance of our MO-COHDA implementation, we compare our
results with the results of a central approach. We use the NSGA-2
implementation from the pymoo library and create Pareto fronts with 25
solution points to allow comparison (Blank and Deb, 2020). All experiments for
MO-COHDA and NSGA-2 are repeated 100 times. Furthermore, we consider the
reference front for the benchmark problems.

###  4.3. Results & Discussion

Figure 2 shows the aggregated fronts of 100 runs with 25 solution points each
for both MO-COHDA and NSGA-2 for the ZDT3 problem. For the results of ZDT1 and
ZDT2 see Appendix B. Figure 2 shows that the two fronts are very similar and
that the MO-COHDA implementation approximates the reference front very well.

Figure 2. Aggregated front of 100 runs each for MO-COHDA and Central approach
compared with reference front for ZDT3

[Four images show four graphs for MO-COHDA, NSGA-2, Reference front and All
together. The fronts shown in all the pictures are nearly the same, except for
a few outliers in the NSGA-2 front.]The image is distributed in four graphs,
each showing the front for ZDT3 created by MO-COHDA (top left), NSGA2 (top
right), reference front (bottom left) and all fronts combined (bottom right).
The reference front shows the true front for ZDT3 which consists of five
disconnected fronts that start at the top left corner of the graph around
(0,1) and continue until the bottom right corner of the graph around (0.85,
-0.1). The fronts for MO-COHDA and NSGA-2 look nearly identical to the
reference front, only NSGA-2 has a few outliers in some of the disconnected
fronts. These are especially visible in the picture showing all fronts
together, where the reference front does not cover all of the points found by
NSGA-2.

To have a closer look at the results, the hypervolume and standard deviation
for the simulation results are presented in Table 1, for ZDT1, ZDT2, ZDT3,
considering the results of MO-COHDA, the centralized implementation, and the
reference front. The larger the hypervolume, the better the performance of the
Pareto front.

Table 1. Comparing Hypervolume mean and standard deviation of 100 runs for MO-COHDA and Central approach (rounded to 5 decimal places) Problem | MO-COHDA | Central | Reference
---|---|---|---
ZDT1 | $7.236\pm 0.000$ | $7.230\pm 0.002$ | $7.256$
ZDT2 | $6.904\pm 0.001$ | $6.898\pm 0.002$ | $6.923$
ZDT3 | $7.699\pm 0.001$ | $7.682\pm 0.029$ | $7.712$

It is recognizable that the MO-COHDA implementation shows excellent results.
The values for the hypervolume for all three benchmark functions are very
close to the values of the reference front (maximum deviations of 0.02). The
standard deviation of the solutions is also very low. If we look at the
results of the central solution for comparison, it can be seen that these are
similar to, and in some cases, slightly worse than, those of the MO-COHDA
implementation. It should be noted, however, that the choice of a suitable
mutate function can greatly influence the goodness of the results. While
moving in random steps towards the border values of the allowed range seems to
be a good approach for the ZDT problems, it might not give as good results for
other problems. Overall, the results of the benchmark problems show that the
implementation of MO-COHDA is suitable for use in multi-objective
optimization.

##  5\. Evaluation - CPES Problem

With the second evaluation, we want to show the functionality of MO-COHDA for
problems specific to CPES. We have chosen an exemplary multi-objective problem
from the energy domain and used MO-COHDA to solve it. Subsection 5.1 describes
the chosen problem, and Subsection 5.2 describes the implementation of this
problem in MO-COHDA with two different parameter settings to introduce the
various parameters. Finally, Subsection 5.3 presents the results from both
settings.

###  5.1. Optimization Problem

The goal of the exemplary problem is to optimize the schedules of a set of $n$
Combined Heat and Power (CHP) units and $m$ wind power plants, considering
three different and potentially conflicting objectives. A schedule
$\boldsymbol{s^{c}}$ of CHP unit $i$ is described by
$\boldsymbol{s_{i}^{c}}=(s_{i1}^{c},...,s_{il}^{c})$ with $l$ defining the
length of the schedule and therefore the number of time intervals that this
schedule covers. The schedule $\boldsymbol{s^{w}}$ of wind power plant $j$ is
described by $\boldsymbol{s_{j}^{w}}=(s_{j1}^{w},...,s_{jl}^{w})$
respectively. Each element of a schedule defines the active power value of the
respective CHP or wind power plant for this time interval. The sum of all
schedules of both, CHP units and wind power plants, describes the cluster
schedule $\boldsymbol{u}$ with
$\boldsymbol{u}=\sum_{i=1}^{m}s_{i}^{c}+\sum_{j=1}^{n}s_{j}^{w}$. The first
objective is to minimize the difference between the cluster schedule
$\boldsymbol{u}$ and a target schedule $\boldsymbol{t}=(t_{1},...t_{l})$. The
target schedule could, for example, be defined by restrictions from the grid
or a given target of a virtual power plant. To achieve a single performance
value, the sum of all differences per interval of the schedule is considered
using the Manhattan distance:

(3) $\text{min
}f_{1}=\sum_{k=1}^{l}\left|\left|\boldsymbol{t}-\boldsymbol{u}\right|\right|_{k}$

The second objective is to minimize the share of produced power in the cluster
schedule causing emissions. In this example, only the CHP plants cause
emissions, so only their contribution to the total produced power is
considered. Again, the individual values for each schedule interval are summed
up to one total emissions share:

(4) $\text{min }f_{2}=\sum_{x=1}^{l}\frac{\sum_{i=1}^{m}s_{ix}^{c}}{u_{x}}$

Finally, the third objective aims at minimizing the uncertainty of the
resulting cluster schedule. This means minimizing the share of power produced
by units where the real available power can deviate from the forecasted power,
in this case, the wind power plants. We assume that the uncertainties increase
with time, as schedule intervals further in the future are more challenging to
forecast and can deviate more from the estimated value. Therefore, we include
a weight vector $\boldsymbol{w}=(w_{1},...w_{l})$ that assigns an uncertainty
weight to each schedule interval with
$w_{i}=\left(100/\frac{l(l+1)}{2}\right)\cdot i$, creating a linear increase
in weights with all weights summing up to $100$. The objective is to minimize
the sum of all weighted uncertain power shares over all schedule intervals:

(5) $\text{min
}f_{3}=\sum_{y=1}^{l}\frac{w_{y}\sum_{j=1}^{n}s_{yj}^{w}}{u_{y}}$

For optimal comparison of the objectives, all performance values are
normalized between 0 and 1, using the $f_{min}$ and $f_{max}$ values. While
$f_{min}=0$ for all objectives, $f_{max}$ deviates. $f_{1max}$ is either the
sum of all values in the target schedule or the sum of the maximum power
values produced in each timestep by all units, depending on which value is
larger. $f_{2max}=l$, meaning that all of the produced power causes emissions
in each schedule interval. Finally, $f_{3max}=100$, which occurs when in each
schedule interval, the complete produced power is considered uncertain.

###  5.2. Experimental Setup

We created a scenario with 15 CHP units (7x200kW, 8x400kW) and 15 wind power
plants (2x 200kW, 3x 250kW, 3x300kW, 3x350kW, 3x400kW), which have to plan
their power schedules for the next 6 hours, meaning they have to choose an
active power value for 24 15-minute-intervals. Each CHP has ten different
possible schedules to choose from: one ”off”-schedule consisting only of 0
values and nine schedules which were created using the
$generate\\_schedules()$-method from the pysimmods CHP
model111https://gitlab.com/midas-mosaik/pysimmods. The wind power plants do
not have schedules but a maximum power value they can produce in each
interval. We used the data from
50Hertz222https://www.50hertz.com/de/Transparenz/Kennzahlen/ErzeugungEinspeisung/
EinspeisungausWindenergie, starting 30.01.2022 at 12:30:00 pm and continuing
over the following 6 hours. The power values were then scaled down to match
the sizes of the wind power plants. Usually, wind turbine power control aims
to produce the maximum amount of power while maintaining safe operating
conditions, which can be achieved using various stall- and pitch-control
techniques (Apata and Oyedokun, 2020). However, for the CPES example, we also
allow for a reduction of power output for wind plants: A wind power plant can
create a power schedule by choosing any integer value between the maximum and
0 for each interval. This complements the limited flexibility of the CHP
plants and allows us to test different approaches for the mutate function,
even though it should be noted that this behavior might not be desired or even
feasible in a real scenario.

Regarding the target schedule, we defined the requirements that a) both, CHP
units and wind power plants should be able to reach it without any additional
power from the respective other unit type and b) the maximum generation from
all units should not exceed the target schedule by more than the target
schedule itself.

The first requirement ensures that the Pareto front includes three ”extreme”
solutions with two out of three objectives having the optimal value and the
last objective having the worst possible value. For example, when the target
is fully reached by only wind power plants, objectives $f_{1}$ and $f_{2}$
would be $0$, while $f_{3}$ is $1$. This gives some information about the
otherwise unknown true Pareto front and helps in the analysis of the results.
The second requirement simplifies the calculation of $f_{1max}$, as it would
exactly equal to twice the amount of the target schedule in every timestep.

To achieve this, the following adaptions have been made to the CHP schedules
and the wind generation forecasts:

  * •

For all CHP units, the first schedule is defined to be the schedule with the
most produced power, so all other schedules were reduced so as not to exceed
this schedule in any interval. The sum of all these highest schedules is used
to create the target schedule.

  * •

The wind power plants should be able to reach this target schedule exactly by
producing their maximum possible power in each interval. This means the sum of
all wind power in each interval should not exceed the target schedule. All
wind forecasts have been adapted to fulfill this requirement.

Each CHP and each wind power plant is represented by an agent with all the
necessary knowledge about available schedules or maximal power values. Agents
choose schedules for their units and add them to the cluster schedule, which
is then used to calculate the performance of the three objectives described in
Subsection 5.1. The information necessary for the agents to compute the
performance values and the hypervolume of the resulting Pareto front is passed
at the beginning of the optimization. This includes the type of all units in
the cluster schedule and the reference point, which is set to $(1.1,1.1)$. For
communication, the agents are connected in their overlay by a small-world
topology, using $k=2$ (Strogatz, 2001).

We tested MO-COHDA with this scenario by varying the pick- and mutate
functions. Table 2 shows Settings A and B and their respective parameter
values. Marked in yellow are those parameters that differ between the two
settings. Since the CHP plants have less flexibility with only ten schedules
each, we chose simple pick and mutate functions for the agents to use in both
settings, namely picking a random solution point and replacing the schedule
there with a random other schedule. For the agents representing wind power
plants, we varied picking just one solution point vs. picking all of them and
used a more ”intelligent” mutate function. It uses the currently selected
schedule of the solution point as a base and then randomly increases or
decreases the schedule in each time interval by a random value between 0 and a
maximum allowed change value. This value defines how different the mutated
schedule can be from the previous schedule and was varied between a lower and
a higher value in the two settings.

Table 2. Overview of parameter combinations for Setting A and Setting B Parameter |  Setting A |  Setting B
---|---|---
pick CHP |  pick random point |  pick random point
pick wind |  pick random point |  pick all points
mutate CHP |  random schedule |  random schedule
mutate wind |  rnd neighbour $\pm 25$ |  rnd neighbour $\pm 100$
min change |  0.0005 |  0.0005
# points |  25 |  25
# iterations |  1 |  1

In CPES optimization, agents would usually run distributed on field appliances
(for example, industrial Raspberry Pi / Revolution Pi) and not on the same
machine. To study the behavior of MO-COHDA for distributed agents, all agents
are executed on one machine33311th Gen Intel i5-1135G7 (8), Intel Device 9a49,
8910MiB / 15589MiB, 4267 MT/s, Ubuntu 20.04.6 LTS x86_64, 5.15.0-89-generic as
different processes run in parallel. All experiments are repeated 30 times
each. Since the actual front is not known, we compare the results of the
different settings with each other.

###  5.3. Results & Discussion

Figure 3 and Figure 4 show the results from all 30 runs for setting A and
setting B, respectively, depicting the entire front with three objectives and
combinations of all pairs of two objectives. Each front has a different color,
so it can be seen that the solution points of individual fronts are well
distributed across the Pareto front. There are significant differences in the
Pareto fronts of settings A and B. While the solution points for setting A in
Figure 3 are all close together, only covering a small area of the solution
space, the results for setting B in Figure 4 show a well-distributed front. In
setting A, the performance values for minimizing emissions lie on average
between $(0.08,0.50)$ and for minimizing uncertainty between $(0.51,0.97)$,
only covering half the possible range for the performance values for those two
objectives. This suggests, that in all the identified cluster schedules, the
percentage of power produced by wind plants is higher than of power produced
by CHP units. Moreover, in setting A not for all combinations of objectives,
Pareto-optimal solutions have been identified, as the performance of the
solution points increases for both minimizing emissions and deviations. In
contrast, setting B covers the full range of performance values for all three
objectives, providing a greater variety of solutions to choose from.

Figure 3. Aggregated front of 30 runs for Setting A, executed in several
processes

[Four separate pictures show all the solution points found in all fronts for
setting A. The top left picture shows a 3D plot with all objectives, the other
pictures show 2D plots with two objectives each. The aggregated front forms a
slightly curved line from high values in f1 and low values in f2 and f3 to low
values in f1, low values in f2 and higher values in f3 with less points
deviating from the curve]The results for setting A are shown in four separate
pictures. The top left picture shows a 3D plot with all objectives visible.
The points are placed close together in the plot, nearly forming a 2D line in
the 3D plot, that ranges from high values in f1 and low values in f2 and f3 to
low values in f1, low values in f2 and higher values in f3. The top right
picture shows the same aggregated front in a 2D plot with only objectives f1
and f2. Here, the front is not Pareto-optimal, when f1 increases, f2 also
increases. The front reaches from around (0, 0) to (1.0, 0.5). Bottom left
shows the 2D Plot for f1 and f3, showing a clear Pareto-front. Here the front
draws a slightly curved line between (0, 1) and (1, 0.5). Finally, the bottom
right 2D plot shows the front for f2 and f3, which shows a linear decrease
from (0.1, 0.9) to (0.5, 0.5)

Figure 4. Aggregated front of 30 runs for Setting B, executed in several
processes

[The aggregated front for the multi-objective optimization described in
setting B is shown in four figures: including all three objectives
(minimization of deviations to target, emissions and uncertainties and
subfigures including two objectives each]This figure shows the aggregated
front from of in total 30 runs for Setting B of the multi-objective
optimization in CPES. This setting includes differences in the pick and mutate
functions. The first subfigure (top left) shows a 3D graphic, including the
aggregated front, taking into account all three objectives of the
optimization. In each figure, the single solutions coming from different runs
are depicted in different colors. The solution points forming the front are
arranged in a very distributed manner, meaning that solution points fulfilling
different ranges of the respective objective functions exist. The second
figure (top right) shows the results of the front regarding the first two
objectives (minimization of the deviation from the target and the minimization
of emissions). Here, the different solution points of the fronts are
furthermore arranged very distributed. Solution points cover the range from
0.0 to 1.0 for each of the respective objectives. The third subfigure (bottom
left) shows the front considering the objectives minimization of the
deviations from the target and of the uncertainties. The arrangement of the
points on the front is similar to the figure before, the front consists of
distributed points, covering the complete ranges of the target (0.0 - 1.0)
Finally, the last subfigure (bottom right) includes the front regarding the
objectives for the minimization of emissions and uncertainties. The
arrangement here differs from the previous ones, a clear line is recognizable,
running from 0.9 of one target to 0.9 of the other target. Points that exceed
0.9 in one of the two targets are therefore not included.

Table 3 shows different performance indicators with mean and standard
deviation for settings A and B. HV (aggr) defines the hypervolume of the
aggregated front of all 30 runs as a reference value for the mean values of
individual runs. The number of decide calls is measured for the same reference
agent – a wind power plant agent – in all scenarios. One decide call always
encompasses the process described in Figure 1, including calculating the
objective functions and the hypervolume. The number of messages considers all
messages exchanged between the agents during the MO-COHDA optimization. The
higher hypervolume value of setting B reflects the improved selection and
distribution of solution points compared to setting A. Considering the much
higher number of messages and decide-function calls, the agents optimize for a
longer time, resulting in more mutations and more exploration of the solution
space. When comparing the HV and HV (aggr) values for each experiment, it can
also be seen that different Pareto-optimal solutions are found in the 30 runs,
resulting in a slightly better HV value for the aggregated front.

Table 3. Comparing performance indicators of 30 runs of MO-COHDA for different settings and implementations (mean and standard deviation) | HV |  | HV
---
(aggr)
# decide calls | # msgs
A | $0.421\pm 0.006$ | $0.448$ | $275\pm 70$ | $37209\pm 12090$
B | $0.935\pm 0.004$ | $1.012$ | $788\pm 125$ | $230041\pm 63065$

While these results show that MO-COHDA can find a set of Pareto-optimal
solutions in each parameter setting and implementation, they also indicate
that correct parameter tuning is critical to get the best results. A trade-off
between performance and computational complexity (and therefore runtime) can
be observed between the two settings. Based on the concrete use case, it has
to be decided what to prioritize: While some use cases might have strict time
limits but do not require the optimal solution, other use cases might require
the optimal solution but have more time to find it.

###  5.4. Computational Complexity of MO-COHDA

Optimization problems in CPES can come with certain requirements or
limitations for the respective algorithm, such as a time limit to find a
feasible solution or communication requirements such as a limited amount of
messages. The computational complexity of optimization algorithms is therefore
highly relevant in order to assess their applicability for specific problems
and their scalability. This includes time complexity, space complexity (in
case of limited computational power of the devices the agents are executed
on), and communication complexity.

Basic COHDA has already been thoroughly analyzed regarding its scalability.
The results show that with a larger number of agents, run-time increases
logarithmically, and the communication expenses increase superlinearly as in
$\mathcal{O}(n^{1.4})$, while increasing search spaces and planning horizons
(length of the schedules) have little effect (Hinrichs and Sonnenschein,
2017). Hinrichs and Sonnenschein state, that due to the fast decision process,
they were able to solve instances with thousands of agents within a few
minutes.

In MO-COHDA however, the complexity of the decision process of individual
agents can increase significantly, depending on the choice of parameters. The
complexity of the mutate function, the number of iterations and the number of
solution points to be optimized can be especially critical here. Additionally,
there is not just one target function to be evaluated, but a set of target
functions plus the hypervolume calculation of the whole front. In MO-COHDA we
use the evoalgos444https://www.simonwessing.de/evoalgos/doc/ package to
calculate the hypervolume. This package has implemented the hypervolume
calculation from Fonseca et. al which has $\mathcal{O}(n^{d-2}logn)$ time
complexity and $\mathcal{O}(n)$ space complexity, with $n$ being the number of
solution points and $d$ the number of objectives (Guerreiro et al., 2021).

To learn more about the scalability and limitations of MO-COHDA, further
experiments are required with different parameter settings.

##  6\. Conclusion

Operating a large number of DER and other controllable elements in a CPES
requires multi-objective problem-solving. Due to the popularity of distributed
algorithms for monitoring and control of CPES, it should also be possible to
perform multi-objective optimization in a fully distributed way. In this paper
we introduced MO-COHDA, an agent-based optimization algorithm, which is
capable of identifying a set of Pareto-optimal solutions for a given multi-
objective optimization problem. MO-COHDA allows a flexible choice of
optimization strategies for the different agents to best fit the respective
flexibility representation, including changing these strategies over the
course of the optimization.

Using three benchmark functions from the ZDT problem suite we showed that with
the right parameter tuning MO-COHDA is capable of approximating the reference
front and gives similarly good solutions as a central NSGA2 approach. We then
proceeded to show the implementation of MO-COHDA in an exemplary use case from
the energy domain by scheduling a set of DER such that the distance to a
target schedule, the amount of emissions and the uncertainty of the resulting
schedule are minimized. By studying the results of two different parameter
settings and two different forms of implementation of agents (single- and
multi-processing) we could show how these different versions affect the
performance.

Further research is necessary to get more insight into the parameter tuning of
MO-COHDA for different multi-objective optimization problems in CPES,
especially considering the real Pareto front is usually not known. In this
regard, the different agent implementations and their influence on finding
optimal solutions should be studied in more detail as well. In addition, the
comparison of the presented MO-COHDA variant with other existing distributed
algorithms in terms of performance, complexity, and efficiency for different
CPES use cases represents another area of future work.

CHP

    Combined Heat and Power
COHDA

    Combinatorial Optimization Heuristic for Distributed Agents
CPES

    Cyber-Physical Energy Systems
DER

    Distributed Energy Resources
MAS

    Multi-Agent Systems
NSGA-2

    Non-dominated Sorting Genetic Algorithm
ICT

    Information and Communication Technology
ZDT

    Zitzler–Deb–Thiele

###### Acknowledgements.

To Jörg, for the interesting discussions about (MO-)COHDA.

## References

  * (1)
  * Adihou et al. (2019) Yolaine Adihou, Mohamed Tahar Mabrouk, Pierrick Haurant, and Bruno Lacarrièere. 2019.  A multi-objective optimization model for the operation of decentralized multi-energy systems. In _Journal of Physics: Conference Series_ , Vol. 1343. IOP Publishing, 012104.  https://doi.org/10.1088/1742-6596/1343/1/012104
  * Almoghathawi et al. (2019) Yasser Almoghathawi, Kash Barker, and Laura A Albert. 2019.  Resilience-driven restoration model for interdependent infrastructure networks.  _Reliability Engineering & System Safety_ 185 (2019), 12–23.
  * Apata and Oyedokun (2020) O Apata and DTO Oyedokun. 2020.  An overview of control techniques for wind turbine systems.  _Scientific African_ 10 (2020), e00566.
  * Berghammer et al. (2012) Rudolf Berghammer, Tobias Friedrich, and Frank Neumann. 2012.  Convergence of set-based multi-objective optimization, indicators and deteriorative cycles.  _Theoretical Computer Science_ 456 (2012), 2–17.
  * Beume et al. (2007) Nicola Beume, Boris Naujoks, and Michael Emmerich. 2007.  SMS-EMOA: Multiobjective selection based on dominated hypervolume.  _European Journal of Operational Research_ 181, 3 (2007), 1653–1669.
  * Blank and Deb (2020) Julian Blank and Kalyanmoy Deb. 2020.  Pymoo: Multi-objective optimization in python.  _Ieee access_ 8 (2020), 89497–89509.
  * Bremer and Lehnhoff (2017) Joerg Bremer and Sebastian Lehnhoff. 2017.  An agent-based approach to decentralized global optimization-adapting cohda to coordinate descent. In _International Conference on Agents and Artificial Intelligence_ , Vol. 2. SCITEPRESS, 129–136.
  * Bremer and Lehnhoff (2018) Jörg Bremer and Sebastian Lehnhoff. 2018.  Hybridizing s-metric selection and support vector decoder for constrained multi-objective energy management. In _International Conference on Hybrid Intelligent Systems_. Springer, 249–259.
  * Bremer and Lehnhoff (2019) Jörg Bremer and Sebastian Lehnhoff. 2019.  Towards fully decentralized multi-objective energy scheduling. In _2019 Federated Conference on Computer Science and Information Systems (FedCSIS)_. IEEE, 193–201.
  * Espina et al. (2020) Enrique Espina, Jacqueline Llanos, Claudio Burgos-Mellado, Roberto Cardenas-Dobson, Manuel Martinez-Gomez, and Doris Sáez. 2020.  Distributed control strategies for microgrids: An overview.  _IEEE Access_ 8 (2020), 193412–193448.
  * Guerreiro et al. (2021) Andreia P Guerreiro, Carlos M Fonseca, and Luís Paquete. 2021.  The hypervolume indicator: Computational problems and algorithms.  _ACM Computing Surveys (CSUR)_ 54, 6 (2021), 1–42.
  * Han et al. (2017) Xue Han, Kai Heussen, Oliver Gehrke, Henrik W Bindner, and Benjamin Kroposki. 2017.  Taxonomy for evaluation of distributed control strategies for distributed energy resources.  _IEEE Transactions on Smart Grid_ 9, 5 (2017), 5185–5195.
  * Hinrichs et al. (2013) Christian Hinrichs, Jörg Bremer, and Michael Sonnenschein. 2013.  Distributed hybrid constraint handling in large scale virtual power plants. In _IEEE PES ISGT Europe 2013_. IEEE, 1–5.  https://doi.org/10.1109/ISGTEurope.2013.6695312
  * Hinrichs and Sonnenschein (2017) Christian Hinrichs and Michael Sonnenschein. 2017.  A distributed combinatorial optimisation heuristic for the scheduling of energy resources represented by self-interested agents.  _Int. J. Bio Inspired Comput._ 10, 2 (2017), 69–78.
  * Hinrichs et al. (2015) Christian Hinrichs, Michael Sonnenschein, Adam Gray, and Curran Crawford. 2015.  Self-organizing demand response with comfort-constrained heat pumps. In _EnviroInfo and ICT for Sustainability 2015_. Atlantis Press, 353–360.  https://doi.org/10.2991/ict4s-env-15.2015.40
  * Holly and Nieße (2021) Stefanie Holly and Astrid Nieße. 2021.  Dynamic communication topologies for distributed heuristics in energy system optimization algorithms. In _2021 16th Conference on Computer Science and Intelligence Systems (FedCSIS)_. IEEE, 191–200.
  * Karimi and Jadid (2020) Hamid Karimi and Shahram Jadid. 2020.  Optimal energy management for multi-microgrid considering demand response programs: A stochastic multi-objective framework.  _Energy_ 195 (2020), 116992.
  * Khezri and Mahmoudi (2020) Rahmat Khezri and Amin Mahmoudi. 2020.  Review on the state-of-the-art multi-objective optimisation of hybrid standalone/grid-connected energy systems.  _IET Generation, Transmission & Distribution_ 14, 20 (2020), 4285–4300.
  * Khunkitti et al. (2021) Sirote Khunkitti, Apirat Siritaratiwat, and Suttichai Premrudeepreechacharn. 2021.  Multi-objective optimal power flow problems based on slime mould algorithm.  _Sustainability_ 13, 13 (2021), 7448.
  * Konak et al. (2006) Abdullah Konak, David W Coit, and Alice E Smith. 2006.  Multi-objective optimization using genetic algorithms: A tutorial.  _Reliability engineering & system safety_ 91, 9 (2006), 992–1007.
  * Luo et al. (2018) Fengji Luo, Zhao Yang Dong, Gaoqi Liang, Junichi Murata, and Zhao Xu. 2018.  A distributed electricity trading system in active distribution networks based on multi-agent coalition and blockchain.  _IEEE Transactions on Power Systems_ 34, 5 (2018), 4097–4108.
  * Mayer et al. (2020) Martin János Mayer, Artúr Szilágyi, and Gyula Gróf. 2020.  Environmental and economic multi-objective optimization of a household level hybrid renewable energy system by genetic algorithm.  _Applied Energy_ 269 (2020), 115058.
  * Murray et al. (2020) Portia Murray, Jan Carmeliet, and Kristina Orehounig. 2020.  Multi-Objective Optimisation of Power-to-Mobility in Decentralised Multi-Energy Systems.  _Energy_ 205 (2020), 117792.  https://doi.org/10.1016/j.energy.2020.117792
  * Naderi et al. (2021) Ehsan Naderi, Mahdi Pourakbari-Kasmaei, Fernando V Cerna, and Matti Lehtonen. 2021.  A novel hybrid self-adaptive heuristic algorithm to handle single-and multi-objective optimal power flow problems.  _International Journal of Electrical Power & Energy Systems_ 125 (2021), 106492.
  * Nieße and Tröschel (2016) Astrid Nieße and Martin Tröschel. 2016.  Controlled self-organization in smart grids. In _2016 IEEE International Symposium on Systems Engineering (ISSE)_. IEEE, 1–6.  https://doi.org/10.1109/SysEng.2016.7753189
  * Nowdeh et al. (2019) S Arabi Nowdeh, I Faraji Davoudkhani, MJ Hadidian Moghaddam, E Seifi Najmi, Almoataz Y Abdelaziz, Abdollah Ahmadi, Seyed-Ehsan Razavi, and Foad Haidari Gandoman. 2019.  Fuzzy multi-objective placement of renewable energy sources in distribution system with objective of loss reduction and reliability improvement using a novel hybrid method.  _Applied Soft Computing_ 77 (2019), 761–779.
  * Ramli et al. (2018) Makbul AM Ramli, HREH Bouchekara, and Abdulsalam S Alghamdi. 2018.  Optimal sizing of PV/wind/diesel hybrid microgrid system using multi-objective self-adaptive differential evolution algorithm.  _Renewable energy_ 121 (2018), 400–411.
  * Ren et al. (2019) Fukang Ren, Jiangjiang Wang, Sitong Zhu, and Yi Chen. 2019.  Multi-objective optimization of combined cooling, heating and power system integrated with solar and geothermal energies.  _Energy conversion and management_ 197 (2019), 111866.
  * Ren et al. (2021) Yanhuan Ren, Junqi Yu, Anjun Zhao, Wenqiang Jing, Tong Ran, and Xiong Yang. 2021.  A multi-objective operation strategy optimization for ice storage systems based on decentralized control structure.  _Building Services Engineering Research and Technology_ 42, 1 (2021), 62–81.  https://doi.org/10.1177/0143624420966259
  * Ruiming (2019) Fang Ruiming. 2019.  Multi-objective optimized operation of integrated energy system with hydrogen storage.  _International Journal of Hydrogen Energy_ 44, 56 (2019), 29409–29417.
  * Salkuti (2019) Surender Reddy Salkuti. 2019.  Optimal power flow using multi-objective glowworm swarm optimization algorithm in a wind energy integrated power system.  _International Journal of Green Energy_ 16, 15 (2019), 1547–1561.
  * Schrage et al. (2023a) Rico Schrage, Jens Sager, Jan Philipp Hörding, and Stefanie Holly. 2023a.  mango: A Modular Python-Based Agent Simulation Framework.  arXiv:2311.17688 [cs.MA]
  * Schrage et al. (2023b) Rico Schrage, Paul Hendrik Tiemann, and Astrid Nieße. 2023b.  A Multi-Criteria Metaheuristic Algorithm for Distributed Optimization of Electric Energy Storage.  _ACM SIGENERGY Energy Informatics Review_ 2, 4 (2023), 44–59.
  * Sharma et al. (2020) Sachin Sharma, KR Niazi, Kusum Verma, and Tanuj Rawat. 2020.  Coordination of different DGs, BESS and demand response for multi-objective optimization of distribution network with special reference to Indian power sector.  _International Journal of Electrical Power & Energy Systems_ 121 (2020), 106074.
  * Stark et al. (2021) Sanja Stark, Anna Volkova, Sebastian Lehnhoff, and Hermann de Meer. 2021.  Why your power system restoration does not work and what the ICT system can do about it. In _Proceedings of the twelfth ACM international conference on future energy systems_. Virtual Event, Italy, 269–273.
  * Strogatz (2001) Steven Strogatz. 2001.  Strogatz, S.H.: Exploring Complex Networks. Nature 410, 268.  _Nature_ 410 (04 2001), 268–76.  https://doi.org/10.1038/35065725
  * Ullah et al. (2021) Kalim Ullah, Ghulam Hafeez, Imran Khan, Sadaqat Jan, and Nadeem Javaid. 2021.  A multi-objective energy optimization in smart grid with high penetration of renewable energy sources.  _Applied Energy_ 299 (2021), 117104.
  * Wang et al. (2020) Rui Wang, Jian Xiong, Min-fan He, Liang Gao, and Ling Wang. 2020.  Multi-objective optimal design of hybrid renewable energy system under multiple scenarios.  _Renewable Energy_ 151 (2020), 226–237.
  * Wang et al. (2021) Xiaoxue Wang, Liting Gu, and Dong Liang. 2021.  Decentralized and Multi-Objective Coordinated Optimization of Hybrid AC/DC Flexible Distribution Networks.  _Frontiers in Energy Research_ (2021), 646.  https://doi.org/10.3389/fenrg.2021.762423
  * Wang et al. (2022) Yongli Wang, Lu Guo, Yuze Ma, Xu Han, Juntai Xing, WenQiang Miao, and Huan Wang. 2022.  Study on operation optimization of decentralized integrated energy system in northern rural areas based on multi-objective.  _Energy Reports_ 8 (2022), 3063–3084.  https://doi.org/10.1016/j.egyr.2022.01.246
  * Wu et al. (2020) Chunying Wu, Jianzhou Wang, Xuejun Chen, Pei Du, and Wendong Yang. 2020.  A novel hybrid system based on multi-objective optimization for wind speed forecasting.  _Renewable energy_ 146 (2020), 149–165.
  * Yang et al. (2021) Lun Yang, Yinliang Xu, Hongbin Sun, Moyuen Chow, and Jianguo Zhou. 2021.  A multiagent system based optimal load restoration strategy in distribution systems.  _International Journal of Electrical Power & Energy Systems_ 124 (2021), 106314.
  * Zitzler et al. (2000) Eckart Zitzler, Kalyanmoy Deb, and Lothar Thiele. 2000.  Comparison of multiobjective evolutionary algorithms: Empirical results.  _Evolutionary computation_ 8, 2 (2000), 173–195.
  * Zitzler and Künzli (2004) Eckart Zitzler and Simon Künzli. 2004.  Indicator-based selection in multiobjective search. In _International conference on parallel problem solving from nature_. Springer, 832–842.

##  Appendix A Example for non-transitivity of reference point

Figure 5. Example for non-transitivity with dynamic reference point

[The figure shows three images. In each image 2 out of 3 fronts are shown
together with two solution points each. The hypervolume between each front and
the reference point is shown. The HV values per front are different, depending
on the reference point. Depending on the sequence in which the fronts are
compared, a different front would be considered as the best]Three graphs are
shown. In each graph two out of a total of three fronts are shown. The fronts
are named A, B and C and have two solution points each. The first graph shows
fronts A and B and the reference point calculated for A and B. The HV for A is
18 and for B is 8. The second graph shows fronts A and C, with the respective
reference point. Here, the HV for A is 16 and for C 17. The third graph shows
fronts B and C, with the HV of B being 28 and of C being 25.

Figure 5 shows a simple example of non-transitivity of hypervolume with
changing reference points. Three different Pareto fronts for minimization
problems A, B and C are compared. The reference point for comparing two fronts
is chosen using according to the SMS-EMOA approach (worst objective values
increased by 1.0). When looking at the resulting hypervolumes we can see the
deteriorative cycle with $A>B$, $B>C$ and $C>A$.

##  Appendix B Results Benchmark

Figure 6. Aggregated front of 100 runs each for MO-COHDA and Central approach
compared with reference front for ZDT1

[Four images show four graphs for MO-COHDA, NSGA-2, Reference front and All
together. The fronts shown in all the pictures are nearly the same, except for
a few outliers in the NSGA-2 front.]The image is distributed in four graphs,
each showing the front for ZDT1 created by MO-COHDA (top left), NSGA2 (top
right), reference front (bottom left) and all fronts combined (bottom right).
The reference front shows the true front for ZDT1 which is a convex front
reaching from (0,1) to (1,0). The fronts for MO-COHDA and NSGA-2 look nearly
identical to the reference front, only NSGA-2 has a few outliers. These are
especially visible in the picture showing all fronts together, where the
reference front does not cover all of the points found by NSGA-2.

Figure 7. Aggregated front of 100 runs each for MO-COHDA and Central approach
compared with reference front for ZDT2

[Four images show four graphs for MO-COHDA, NSGA-2, Reference front and All
together. The fronts shown in all the pictures are nearly the same, except for
a few outliers in the NSGA-2 front.]The image is distributed in four graphs,
each showing the front for ZDT2 created by MO-COHDA (top left), NSGA2 (top
right), reference front (bottom left) and all fronts combined (bottom right).
The reference front shows the true front for ZDT2 which is a non-convex front
that reaches from (0, 1) to (1,0). The fronts for MO-COHDA and NSGA-2 look
nearly identical to the reference front, only NSGA-2 has a few outliers. These
are especially visible in the picture showing all fronts together, where the
reference front does not cover all of the points found by NSGA-2.

  *[CPES]: Cyber-Physical Energy Systems
  *[DER]: Distributed Energy Resources
  *[ICT]: Information and Communication Technology
  *[MAS]: Multi-Agent Systems
  *[NSGA-2]: Non-dominated Sorting Genetic Algorithm
  *[ZDT]: Zitzler–Deb–Thiele
  *[COHDA]: Combinatorial Optimization Heuristic for Distributed Agents
  *[CHP]: Combined Heat and Power