# The Landscape of Emerging AI Agent Architectures for Reasoning, Planning,
and Tool Calling: A Survey

Tula Masterman*
Neudesic, an IBM Company
<EMAIL_ADDRESS>
Sandi Besen*
IBM
<EMAIL_ADDRESS>
Mason Sawtell*
Neudesic, an IBM Company
<EMAIL_ADDRESS>

* Denotes Equal Contribution Alex Chao
Microsoft
<EMAIL_ADDRESS>

###### Abstract

This survey paper examines the recent advancements in AI agent
implementations, with a focus on their ability to achieve complex goals that
require enhanced reasoning, planning, and tool execution capabilities. The
primary objectives of this work are to a) communicate the current capabilities
and limitations of existing AI agent implementations, b) share insights gained
from our observations of these systems in action, and c) suggest important
considerations for future developments in AI agent design. We achieve this by
providing overviews of single-agent and multi-agent architectures, identifying
key patterns and divergences in design choices, and evaluating their overall
impact on accomplishing a provided goal. Our contribution outlines key themes
when selecting an agentic architecture, the impact of leadership on agent
systems, agent communication styles, and key phases for planning, execution,
and reflection that enable robust AI agent systems.

††footnotetext: The opinions expressed in this paper are solely those of the
authors and do not necessarily reflect the views or policies of their
respective employers.

_K_ eywords AI Agent $\cdot$ Agent Architecture $\cdot$ AI Reasoning $\cdot$
Planning $\cdot$ Tool Calling $\cdot$ Single Agent $\cdot$ Multi Agent $\cdot$
Agent Survey $\cdot$ LLM Agent $\cdot$ Autonomous Agent

##  1 Introduction

Since the launch of ChatGPT, many of the first wave of generative AI
applications have been a variation of a chat over a corpus of documents using
the Retrieval Augmented Generation (RAG) pattern. While there is a lot of
activity in making RAG systems more robust, various groups are starting to
build what the next generation of AI applications will look like, centralizing
on a common theme: agents.

Beginning with investigations into recent foundation models like GPT-4 and
popularized through open-source projects like AutoGPT and BabyAGI, the
research community has experimented with building autonomous agent-based
systems [19, 1].

As opposed to zero-shot prompting of a large language model where a user types
into an open-ended text field and gets a result without additional input,
agents allow for more complex interaction and orchestration. In particular,
agentic systems have a notion of planning, loops, reflection and other control
structures that heavily leverage the model’s inherent reasoning capabilities
to accomplish a task end-to-end. Paired with the ability to use tools,
plugins, and function calling, agents are empowered to do more general-purpose
work.

Among the community, there is a current debate on whether single or multi-
agent systems are best suited for solving complex tasks. While single agent
architectures excel when problems are well-defined and feedback from other
agent-personas or the user is not needed, multi-agent architectures tend to
thrive more when collaboration and multiple distinct execution paths are
required.

Figure 1: A visualization of single and multi-agent architectures with their
underlying features and abilities

###  1.1 Taxonomy

Agents. AI agents are language model-powered entities able to plan and take
actions to execute goals over multiple iterations. AI agent architectures are
either comprised of a single agent or multiple agents working together to
solve a problem.

Typically, each agent is given a persona and access to a variety of tools that
will help them accomplish their job either independently or as part of a team.
Some agents also contain a memory component, where they can save and load
information outside of their messages and prompts. In this paper, we follow
the definition of agent that consists of “brain, perception, and action” [31].
These components satisfy the minimum requirements for agents to understand,
reason, and act on the environment around them.

Agent Persona. An agent persona describes the role or personality that the
agent should take on, including any other instructions specific to that agent.
Personas also contain descriptions of any tools the agent has access to. They
make the agent aware of their role, the purpose of their tools, and how to
leverage them effectively. Researchers have found that “shaped personality
verifiably influences Large Language Model (LLM) behavior in common downstream
(i.e. subsequent) tasks, such as writing social media posts” [21]. Solutions
that use multiple agent personas to solve problems also show significant
improvements compared to Chain-of-Thought (CoT) prompting where the model is
asked to break down its plans step by step [28, 29].

Tools. In the context of AI agents, tools represent any functions that the
model can call. They allow the agent to interact with external data sources by
pulling or pushing information to that source. An example of an agent persona
and associated tools is a professional contract writer. The writer is given a
persona explaining their role and the types of tasks it must accomplish. It is
also given tools related to adding notes to a document, reading an existing
document, or sending an email with a final draft.

Single Agent Architectures. These architectures are powered by one language
model and will perform all the reasoning, planning, and tool execution on
their own. The agent is given a system prompt and any tools required to
complete their task. In single agent patterns there is no feedback mechanism
from other AI agents; however, there may be options for humans to provide
feedback that guides the agent.

Multi-Agent Architectures. These architectures involve two or more agents,
where each agent can utilize the same language model or a set of different
language models. The agents may have access to the same tools or different
tools. Each agent typically has their own persona.

Multi-agent architectures can have a wide variety of organizations at any
level of complexity. In this paper, we divide them into two primary
categories: vertical and horizontal. It is important to keep in mind that
these categories represent two ends of a spectrum, where most existing
architectures fall somewhere between these two extremes.

Vertical Architectures. In this structure, one agent acts as a leader and has
other agents report directly to them. Depending on the architecture, reporting
agents may communicate exclusively with the lead agent. Alternatively, a
leader may be defined with a shared conversation between all agents. The
defining features of vertical architectures include having a lead agent and a
clear division of labor between the collaborating agents.

Horizontal Architectures. In this structure, all the agents are treated as
equals and are part of one group discussion about the task. Communication
between agents occurs in a shared thread where each agent can see all messages
from the others. Agents also can volunteer to complete certain tasks or call
tools, meaning they do not need to be assigned by a leading agent. Horizontal
architectures are generally used for tasks where collaboration, feedback and
group discussion are key to the overall success of the task [2].

##  2 Key Considerations for Effective Agents

###  2.1 Overview

Agents are designed to extend language model capabilities to solve real-world
challenges. Successful implementations require robust problem-solving
capabilities enabling agents to perform well on novel tasks. To solve real-
world problems effectively, agents require the ability to reason and plan as
well as call tools that interact with an external environment. In this section
we explore why reasoning, planning, and tool calling are critical to agent
success.

###  2.2 The Importance of Reasoning and Planning

Reasoning is a fundamental building block of human cognition, enabling people
to make decisions, solve problems, and understand the world around us. AI
agents need a strong ability to reason if they are to effectively interact
with complex environments, make autonomous decisions, and assist humans in a
wide range of tasks. This tight synergy between “acting” and “reasoning”
allows new tasks to be learned quickly and enables robust decision making or
reasoning, even under previously unseen circumstances or information
uncertainties [32]. Additionally, agents need reasoning to adjust their plans
based on new feedback or information learned.

If agents lacking reasoning skills are tasked with acting on straightforward
tasks, they may misinterpret the query, generate a response based on a literal
understanding, or fail to consider multi-step implications.

Planning, which requires strong reasoning abilities, commonly falls into one
of five major approaches: task decomposition, multi-plan selection, external
module-aided planning, reflection and refinement and memory-augmented planning
[12]. These approaches allow the model to either break the task down into sub
tasks, select one plan from many generated options, leverage a preexisting
external plan, revise previous plans based on new information, or leverage
external information to improve the plan.

Most agent patterns have a dedicated planning step which invokes one or more
of these techniques to create a plan before any actions are executed. For
example, Plan Like a Graph (PLaG) is an approach that represents plans as
directed graphs, with multiple steps being executed in parallel [15, 33]. This
can provide a significant performance increase over other methods on tasks
that contain many independent subtasks that benefit from asynchronous
execution.

###  2.3 The Importance of Effective Tool Calling

One key benefit of the agent abstraction over prompting base language models
is the agents’ ability to solve complex problems by calling multiple tools.
These tools enable the agent to interact with external data sources, send or
retrieve information from existing APIs, and more. Problems that require
extensive tool calling often go hand in hand with those that require complex
reasoning.

Both single-agent and multi-agent architectures can be used to solve
challenging tasks by employing reasoning and tool calling steps. Many methods
use multiple iterations of reasoning, memory, and reflection to effectively
and accurately complete problems [16, 23, 32]. They often do this by breaking
a larger problem into smaller subproblems, and then solving each one with the
appropriate tools in sequence.

Other works focused on advancing agent patterns highlight that while breaking
a larger problem into smaller subproblems can be effective at solving complex
tasks, single agent patterns often struggle to complete the long sequence
required [22, 6].

Multi-agent patterns can address the issues of parallel tasks and robustness
since individual agents can work on individual subproblems. Many multi-agent
patterns start by taking a complex problem and breaking it down into several
smaller tasks. Then, each agent works independently on solving each task using
their own independent set of tools.

##  3 Single Agent Architectures

###  3.1 Overview

In this section, we highlight some notable single agent methods such as ReAct,
RAISE, Reflexion, AutoGPT + P, and LATS. Each of these methods contain a
dedicated stage for reasoning about the problem before any action is taken to
advance the goal. We selected these methods based on their contributions to
the reasoning and tool calling capabilities of agents.

###  3.2 Key Themes

We find that successful goal execution by agents is contingent upon proper
planning and self-correction [32, 16, 23, 1]. Without the ability to self-
evaluate and create effective plans, single agents may get stuck in an endless
execution loop and never accomplish a given task or return a result that does
not meet user expectations [32]. We find that single agent architectures are
especially useful when the task requires straightforward function calling and
does not need feedback from another agent [22].

###  3.3 Examples

ReAct. In the ReAct (Reason + Act) method, an agent first writes a thought
about the given task. It then performs an action based on that thought, and
the output is observed. This cycle can repeat until the task is complete [32].
When applied to a diverse set of language and decision-making tasks, the ReAct
method demonstrates improved effectiveness compared to zero-shot prompting on
the same tasks. It also provides improved human interoperability and
trustworthiness because the entire thought process of the model is recorded.
When evaluated on the HotpotQA dataset, the ReAct method only hallucinated 6%
of the time, compared to 14% using the chain of thought (CoT) method [29, 32].

However, the ReAct method is not without its limitations. While intertwining
reasoning, observation, and action improves trustworthiness, the model can
repetitively generate the same thoughts and actions and fail to create new
thoughts to provoke finishing the task and exiting the ReAct loop.
Incorporating human feedback during the execution of the task would likely
increase its effectiveness and applicability in real-world scenarios.

Figure 2: An example of the ReAct method compared to other methods [32]

RAISE. The RAISE method is built upon the ReAct method, with the addition of a
memory mechanism that mirrors human short-term and long-term memory [16]. It
does this by using a scratchpad for short-term storage and a dataset of
similar previous examples for long-term storage.

By adding these components, RAISE improves upon the agent’s ability to retain
context in longer conversations. The paper also highlights how fine-tuning the
model results in the best performance for their task, even when using a
smaller model. They also showed that RAISE outperforms ReAct in both
efficiency and output quality.

While RAISE significantly improves upon existing methods in some respects, the
researchers also highlighted several issues. First, RAISE struggles to
understand complex logic, limiting its usefulness in many scenarios.
Additionally, RAISE agents often hallucinated with respect to their roles or
knowledge. For example, a sales agent without a clearly defined role might
retain the ability to code in Python, which may enable them to start writing
Python code instead of focusing on their sales tasks. These agents might also
give the user misleading or incorrect information. This problem was addressed
by fine-tuning the model, but the researchers still highlighted hallucination
as a limitation in the RAISE implementation.

Figure 3: A diagram showing the RAISE method [16]

Reflexion. Reflexion is a single-agent pattern that uses self-reflection
through linguistic feedback [23]. By utilizing metrics such as success state,
current trajectory, and persistent memory, this method uses an LLM evaluator
to provide specific and relevant feedback to the agent. This results in an
improved success rate as well as reduced hallucination compared to Chain-of-
Thought and ReAct.

Despite these advancements, the Reflexion authors identify various limitations
of the pattern. Primarily, Reflexion is susceptible to “non-optimal local
minima solutions”. It also uses a sliding window for long-term memory, rather
than a database. This means that the volume of long-term memory is limited by
the token limit of the language model. Finally, the researchers identify that
while Reflexion surpasses other single-agent patterns, there are still
opportunities to improve performance on tasks that require a significant
amount of diversity, exploration, and reasoning.

AUTOGPT + P. AutoGPT + P (Planning) is a method that addresses reasoning
limitations for agents that command robots in natural language [1]. AutoGPT+P
combines object detection and Object Affordance Mapping (OAM) with a planning
system driven by a LLM. This allows the agent to explore the environment for
missing objects, propose alternatives, or ask the user for assistance with
reaching its goal.

AutoGPT+P starts by using an image of a scene to detect the objects present. A
language model then uses those objects to select which tool to use, from four
options: Plan Tool, Partial Plan Tool, Suggest Alternative Tool, and Explore
Tool. These tools allow the robot to not only generate a full plan to complete
the goal, but also to explore the environment, make assumptions, and create
partial plans.

However, the language model does not generate the plan entirely on its own.
Instead, it generates goals and steps to work aside a classical planner which
executes the plan using Planning Domain Definition Language (PDDL). The paper
found that “LLMs currently lack the ability to directly translate a natural
language instruction into a plan for executing robotic tasks, primarily due to
their constrained reasoning capabilities” [1]. By combining the LLM planning
capabilities with a classical planner, their approach significantly improves
upon other purely language model-based approaches to robotic planning.

As with most first of their kind approaches, AutoGPT+P is not without its
drawbacks. Accuracy of tool selection varies, with certain tools being called
inappropriately or getting stuck in loops. In scenarios where exploration is
required, the tool selection sometimes leads to illogical exploration
decisions like looking for objects in the wrong place. The framework also is
limited in terms of human interaction, with the agent being unable to seek
clarification and the user being unable to modify or terminate the plan during
execution.

Figure 4: A diagram of the AutoGPT+P method [1]

LATS. Language Agent Tree Search (LATS) is a single-agent method that
synergizes planning, acting, and reasoning by using trees [36]. This
technique, inspired by Monte Carlo Tree Search, represents a state as a node
and taking an action as traversing between nodes. It uses LM-based heuristics
to search for possible options, then selects an action using a state
evaluator.

When compared to other tree-based methods, LATS implements a self-reflection
reasoning step that dramatically improves performance. When an action is
taken, both environmental feedback as well as feedback from a language model
is used to determine if there are any errors in reasoning and propose
alternatives. This ability to self-reflect combined with a powerful search
algorithm makes LATS perform extremely well on various tasks.

However, due to the complexity of the algorithm and the reflection steps
involved, LATS often uses more computational resources and takes more time to
complete than other single-agent methods [36]. The paper also uses relatively
simple question answering benchmarks and has not been tested on more robust
scenarios that involve involving tool calling or complex reasoning.

##  4 Multi Agent Architectures

###  4.1 Overview

In this section, we examine a few key studies and sample frameworks with
multi-agent architectures, such as Embodied LLM Agents Learn to Cooperate in
Organized Teams, DyLAN, AgentVerse, and MetaGPT. We highlight how these
implementations facilitate goal execution through inter-agent communication
and collaborative plan execution. This is not intended to be an exhaustive
list of all agent frameworks, our goal is to provide broad coverage of key
themes and examples related to multi-agent patterns.

###  4.2 Key Themes

Multi-agent architectures create an opportunity for both the intelligent
division of labor based on skill and helpful feedback from a variety of agent
personas. Many multi-agent architectures work in stages where teams of agents
are created and reorganized dynamically for each planning, execution, and
evaluation phase [2, 9, 18]. This reorganization provides superior results
because specialized agents are employed for certain tasks, and removed when
they are no longer needed. By matching agents roles and skills to the task at
hand, agent teams can achieve greater accuracy and decrease time to meet the
goal. Key features of effective multi-agent architectures include clear
leadership in agent teams, dynamic team construction, and effective
information sharing between team members so that important information does
not get lost in superfluous chatter.

###  4.3 Examples

Embodied LLM Agents Learn to Cooperate in Organized Teams. Research by Guo et
al. demonstrates the impact of a lead agent on the overall effectiveness of
the agent team [9]. This architecture contains a vertical component through
the leader agent, as well as a horizontal component from the ability for
agents to converse with other agents besides the leader. The results of their
study demonstrate that agent teams with an organized leader complete their
tasks nearly 10% faster than teams without a leader.

Furthermore, they discovered that in teams without a designated leader, agents
spent most of their time giving orders to one another (~50% of communication),
splitting their remaining time between sharing information, or requesting
guidance. Conversely, in teams with a designated leader, 60% of the leader’s
communication involved giving directions, prompting other members to focus
more on exchanging and requesting information. Their results demonstrate that
agent teams are most effective when the leader is a human.

Figure 5: Agent teams with a designated leader achieve superior performance
[9]

.

Beyond team structure, the paper emphasizes the importance of employing a
“criticize-reflect” step for generating plans, evaluating performance,
providing feedback, and re-organizing the team [9]. Their results indicate
that agents with a dynamic team structure with rotating leadership provide the
best results, with both the lowest time to task completion and the lowest
communication cost on average. Ultimately, leadership and dynamic team
structures improve the overall team’s ability to reason, plan, and perform
tasks effectively.

DyLAN. The Dynamic LLM-Agent Network (DyLAN) framework creates a dynamic agent
structure that focuses on complex tasks like reasoning and code generation
[18]. DyLAN has a specific step for determining how much each agent has
contributed in the last round of work and only moves top contributors the next
round of execution. This method is horizontal in nature since agents can share
information with each other and there is no defined leader. DyLAN shows
improved performance on a variety of benchmarks which measure arithmetic and
general reasoning capabilities. This highlights the impact of dynamic teams
and demonstrates that by consistently re-evaluating and ranking agent
contributions, we can create agent teams that are better suited to complete a
given task.

AgentVerse. Multi-agent architectures like AgentVerse demonstrate how distinct
phases for group planning can improve an AI agent’s reasoning and problem-
solving capabilities [2]. AgentVerse contains four primary stages for task
execution: recruitment, collaborative decision making, independent action
execution, and evaluation. This can be repeated until the overall goal is
achieved. By strictly defining each phase, AgentVerse helps guide the set of
agents to reason, discuss, and execute more effectively.

As an example, the recruitment step allows agents to be removed or added based
on the progress towards the goal. This helps ensure that the right agents are
participating at any given stage of problem solving. The researchers found
that horizontal teams are generally best suited for collaborative tasks like
consulting, while vertical teams are better suited for tasks that require
clearer isolation of responsibilities for tool calling.

Figure 6: A diagram of the AgentVerse method [2]

MetaGPT. Many multi-agent architectures allow agents to converse with one
another while collaborating on a common problem. This conversational
capability can lead to chatter between the agents that is superfluous and does
not further the team goal. MetaGPT addresses the issue of unproductive chatter
amongst agents by requiring agents to generate structured outputs like
documents and diagrams instead of sharing unstructured chat messages [11].

Additionally, MetaGPT implements a ”publish-subscribe” mechanism for
information sharing. This allows all the agents to share information in one
place, but only read information relevant to their individual goals and tasks.
This streamlines the overall goal execution and reduces conversational noise
between agents. When compared to single-agent architectures on the HumanEval
and MBPP benchmarks, MetaGPT’s multi-agent architecture demonstrates
significantly better results.

##  5 Discussion and Observations

###  5.1 Overview

In this section we discuss the key themes and impacts of the design choices
exhibited in the previously outlined agent patterns. These patterns serve as
key examples of the growing body of research and implementation of AI agent
architectures. Both single and multi-agent architectures seek to enhance the
capabilities of language models by giving them the ability to execute goals on
behalf of or alongside a human user. Most observed agent implementations
broadly follow the plan, act, and evaluate process to iteratively solve
problems.

We find that both single and multi-agent architectures demonstrate compelling
performance on complex goal execution. We also find that across architectures
clear feedback, task decomposition, iterative refinement, and role definition
yield improved agent performance.

###  5.2 Key Findings

Typical Conditions for Selecting a Single vs Multi-Agent Architecture. Based
on the aforementioned agent patterns, we find that single-agent patterns are
generally best suited for tasks with a narrowly defined list of tools and
where processes are well-defined. Single agents are also typically easier to
implement since only one agent and set of tools needs to be defined.
Additionally, single agent architectures do not face limitations like poor
feedback from other agents or distracting and unrelated chatter from other
team members. However, they may get stuck in an execution loop and fail to
make progress towards their goal if their reasoning and refinement
capabilities are not robust.

Multi-agent architectures are generally well-suited for tasks where feedback
from multiple personas is beneficial in accomplishing the task. For example,
document generation may benefit from a multi-agent architecture where one
agent provides clear feedback to another on a written section of the document.
Multi-agent systems are also useful when parallelization across distinct tasks
or workflows is required. Crucially, Wang et. al finds that multi-agent
patterns perform better than single agents in scenarios when no examples are
provided [26]. By nature, multi-agent systems are more complex and often
benefit from robust conversation management and clear leadership.

While single and multi-agent patterns have diverging capabilities in terms of
scope, research finds that “multi-agent discussion does not necessarily
enhance reasoning when the prompt provided to an agent is sufficiently robust”
[26]. This suggests that those implementing agent architectures should decide
between single or multiple agents based on the broader context of their use
case, and not based on the reasoning capabilities required.

Agents and Asynchronous Task Execution. While a single agent can initiate
multiple asynchronous calls simultaneously, its operational model does not
inherently support the division of responsibilities across different execution
threads. This means that, although tasks are handled asynchronously, they are
not truly parallel in the sense of being autonomously managed by separate
decision-making entities. Instead, the single agent must sequentially plan and
execute tasks, waiting for one batch of asynchronous operations to complete
before it can evaluate and move on to the next step. Conversely, in multi-
agent architectures, each agent can operate independently, allowing for a more
dynamic division of labor. This structure not only facilitates simultaneous
task execution across different domains or objectives but also allows
individual agents to proceed with their next steps without being hindered by
the state of tasks handled by others, embodying a more flexible and parallel
approach to task management.

Impact of Feedback and Human Oversight on Agent Systems. When solving a
complex problem, it is extremely unlikely that one provides a correct, robust
solution on their first try. Instead, one might pose a potential solution
before criticizing it and refining it. One could also consult with someone
else and receive feedback from another perspective. The same idea of iterative
feedback and refinement is essential for helping agents solve complex
problems.

This is partially because language models tend to commit to an answer earlier
in their response, which can cause a ‘snowball effect’ of increasing diversion
from their goal state [34] . By implementing feedback, agents are much more
likely to correct their course and reach their goal.

Additionally, the inclusion of human oversight improves the immediate outcome
by aligning the agent’s responses more closely with human expectations,
mitigating the potential for agents to delve down an inefficient or invalid
approach to solving a task. As of today, including human validation and
feedback in the agent architecture yields more reliable and trustworthy
results [4, 9].

Language models also exhibit sycophantic behavior, where they “tend to mirror
the user’s stance, even if it means forgoing the presentation of an impartial
or balanced viewpoint” [20]. Specifically, the AgentVerse paper describes how
agents are susceptible to feedback from other agents, even if the feedback is
not sound. This can lead the agent team to generate a faulty plan which
diverts them from their objective [2]. Robust prompting can help mitigate
this, but those developing agent applications should be aware of the risks
when implementing user or agent feedback systems.

Challenges with Group Conversations and Information Sharing. One challenge
with multi-agent architectures lies in their ability to intelligently share
messages between agents. Multi-agent patterns have a greater tendency to get
caught up in niceties and ask one another things like “how are you”, while
single agent patterns tend to stay focused on the task at hand since there is
no team dynamic to manage. The extraneous dialogue in multi-agent systems can
impair both the agent’s ability to reason effectively and execute the right
tools, ultimately distracting the agents from the task and decreasing team
efficiency. This is especially true in a horizontal architecture, where agents
typically share a group chat and are privy to every agent’s message in a
conversation. Message subscribing or filtering improves multi-agent
performance by ensuring agents only receive information relevant to their
tasks.

In vertical architectures, tasks tend to be clearly divided by agent skill
which helps reduce distractions in the team. However, challenges arise when
the leading agent fails to send critical information to their supporting
agents and does not realize the other agents aren’t privy to necessary
information. This failure can lead to confusion in the team or hallucination
in the results. One approach to address this issue is to explicitly include
information about access rights in the system prompt so that the agents have
contextually appropriate interactions.

Impact of Role Definition and Dynamic Teams. Clear role definition is critical
for both single and multi-agent architectures. In single-agent architectures
role definition ensures that the agent stays focused on the provided task,
executes the proper tools, and minimizes hallucination of other capabilities.
Similarly, role definition in multi-agent architectures ensures each agent
knows what it’s responsible for in the overall team and does not take on tasks
outside of their described capabilities or scope. Beyond individual role
definition, establishing a clear group leader also improves the overall
performance of multi-agent teams by streamlining task assignment. Furthermore,
defining a clear system prompt for each agent can minimize excess chatter by
prompting the agents not to engage in unproductive communication.

Dynamic teams where agents are brought in and out of the system based on need
have also been shown to be effective. This ensures that all agents
participating in the planning or execution of tasks are fit for that round of
work.

###  5.3 Summary

Both single and multi-agent patterns exhibit strong performance on a variety
of complex tasks involving reasoning and tool execution. Single agent patterns
perform well when given a defined persona and set of tools, opportunities for
human feedback, and the ability to work iteratively towards their goal. When
constructing an agent team that needs to collaborate on complex goals, it is
beneficial to deploy agents with at least one of these key elements: clear
leader(s), a defined planning phase and opportunities to refine the plan as
new information is learned, intelligent message filtering, and dynamic teams
whose agents possess specific skills relevant to the current sub-task. If an
agent architecture employs at least one of these approaches it is likely to
result in increased performance compared to a single agent architecture or a
multi-agent architecture without these tactics.

##  6 Limitations of Current Research and Considerations for Future Research

###  6.1 Overview

In this section we examine some of the limitations of agent research today and
identify potential areas for improving AI agent systems. While agent
architectures have significantly enhanced the capability of language models in
many ways, there are some major challenges around evaluations, overall
reliability, and issues inherited from the language models powering each
agent.

###  6.2 Challenges with Agent Evaluation

While LLMs are evaluated on a standard set of benchmarks designed to gauge
their general understanding and reasoning capabilities, the benchmarks for
agent evaluation vary greatly.

Many research teams introduce their own unique agent benchmarks alongside
their agent implementation which makes comparing multiple agent
implementations on the same benchmark challenging. Additionally, many of these
new agent-specific benchmarks include a hand-crafted, highly complex,
evaluation set where the results are manually scored [2]. This can provide a
high-quality assessment of a method’s capabilities, but it also lacks the
robustness of a larger dataset and risks introducing bias into the evaluation,
since the ones developing the method are also the ones writing and scoring the
results. Agents can also have problems generating a consistent answer over
multiple iterations, due to variability in the models, environment, or problem
state. This added randomness poses a much larger problem to smaller, complex
evaluation sets.

###  6.3 Impact of Data Contamination and Static Benchmarks

Some researchers evaluate their agent implementations on the typical LLM
benchmarks. Emerging research indicates that there is significant data
contamination in the model’s training data, supported by the observation that
a model’s performance significantly worsens when benchmark questions are
modified [8, 38, 37]. This raises doubts on the authenticity of benchmark
scores for both the language models and language model powered agents.

Furthermore, researchers have found that “As LLMs progress at a rapid pace,
existing datasets usually fail to match the models’ ever-evolving
capabilities, because the complexity level of existing benchmarks is usually
static and fixed” [37]. To address this, work has been done to create dynamic
benchmarks that are resistant to simple memorization [38, 37]. Researchers
have also explored the idea of generating an entirely synthetic benchmark
based on a user’s specific environment or use case [14, 27]. While these
techniques can help with contamination, decreasing the level of human
involvement can pose additional risks regarding correctness and the ability to
solve problems.

###  6.4 Benchmark Scope and Transferability

Many language model benchmarks are designed to be solved in a single
iteration, with no tool calls, such as MMLU or GSM8K [3, 10]. While these are
important for measuring the abilities of base language models, they are not
good proxies for agent capabilities because they do not account for agent
systems’ ability to reason over multiple steps or access outside information.
StrategyQA improves upon this by assessing models’ reasoning abilities over
multiple steps, but the answers are limited to Yes/No responses [7]. As the
industry continues to pivot towards agent focused use-cases additional
measures will be needed to better assess the performance and generalizability
of agents to tasks involving tools that extend beyond their training data.

Some agent specific benchmarks like AgentBench evaluate language model-based
agents in a variety of different environments such as web browsing, command-
line interfaces, and video games [17]. This provides a better indication for
how well agents can generalize to new environments, by reasoning, planning,
and calling tools to achieve a given task. Benchmarks like AgentBench and
SmartPlay introduce objective evaluation metrics designed to evaluate the
implementation’s success rate, output similarity to human responses, and
overall efficiency [17, 30]. While these objective metrics are important to
understanding the overall reliability and accuracy of the implementation, it
is also important to consider more nuanced or subjective measures of
performance. Metrics such as efficiency of tool use, reliability, and
robustness of planning are nearly as important as success rate but are much
more difficult to measure. Many of these metrics require evaluation by a human
expert, which can be costly and time consuming compared to LLM-as-judge
evaluations.

###  6.5 Real-world Applicability

Many of the existing benchmarks focus on the ability of Agent systems to
reason over logic puzzles or video games [17]. While evaluating performance on
these types of tasks can help get a sense of the reasoning capabilities of
agent systems, it is unclear whether performance on these benchmarks
translates to real-world performance. Specifically, real-world data can be
noisy and cover a much wider breadth of topics that many common benchmarks
lack.

One popular benchmark that uses real-world data is WildBench, which is sourced
from the WildChat dataset of 570,000 real conversations with ChatGPT [35].
Because of this, it covers a huge breadth of tasks and prompts. While
WildBench covers a wide range of topics, most other real-world benchmarks
focus on a specific task. For example, SWE-bench is a benchmark that uses a
set of real-world issues raised on GitHub for software engineering tasks in
Python [13]. This can be very helpful when evaluating agents designed to write
Python code and provides a sense for how well agents can reason about code
related problems; however, it is less informative when trying to understand
agent capabilities involving other programming languages.

###  6.6 Bias and Fairness in Agent Systems

Language Models have been known to exhibit bias both in terms of evaluation as
well as in social or fairness terms [5]. Moreover, agents have specifically
been shown to be “less robust, prone to more harmful behaviors, and capable of
generating stealthier content than LLMs, highlighting significant safety
challenges” [25]. Other research has found “a tendency for LLM agents to
conform to the model’s inherent social biases despite being directed to debate
from certain political perspectives” [24]. This tendency can lead to faulty
reasoning in any agent-based implementation.

As the complexity of tasks and agent involvement increases, more research is
needed to identify and address biases within these systems. This poses a very
large challenge to researchers, since scalable and novel benchmarks often
involve some level of LLM involvement during creation. However, a truly robust
benchmark for evaluating bias in LLM-based agents must include human
evaluation.

##  7 Conclusion and Future Directions

The AI agent implementations explored in this survey demonstrate the rapid
enhancement in language model powered reasoning, planning, and tool calling.
Single and multi-agent patterns both show the ability to tackle complex multi-
step problems that require advanced problem-solving skills. The key insights
discussed in this paper suggest that the best agent architecture varies based
on use case. Regardless of the architecture selected, the best performing
agent systems tend to incorporate at least one of the following approaches:
well defined system prompts, clear leadership and task division, dedicated
reasoning / planning- execution - evaluation phases, dynamic team structures,
human or agentic feedback, and intelligent message filtering. Architectures
that leverage these techniques are more effective across a variety of
benchmarks and problem types.

While the current state of AI-driven agents is promising, there are notable
limitations and areas for future improvement. Challenges around comprehensive
agent benchmarks, real world applicability, and the mitigation of harmful
language model biases will need to be addressed in the near-term to enable
reliable agents. By examining the progression from static language models to
more dynamic, autonomous agents, this survey aims to provide a holistic
understanding of the current AI agent landscape and offer insight for those
building with existing agent architectures or developing custom agent
architectures.

## References

  * [1] Timo Birr, Christoph Pohl, Abdelrahman Younes and Tamim Asfour  “AutoGPT+P: Affordance-based Task Planning with Large Language Models” arXiv:2402.10778 [cs] version: 1  arXiv, 2024  URL: http://arxiv.org/abs/2402.10778
  * [2] Weize Chen et al.  “AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors” arXiv:2308.10848 [cs]  arXiv, 2023  URL: http://arxiv.org/abs/2308.10848
  * [3] Karl Cobbe et al.  “Training Verifiers to Solve Math Word Problems” arXiv:2110.14168 [cs]  arXiv, 2021  URL: http://arxiv.org/abs/2110.14168
  * [4] Xueyang Feng et al.  “Large Language Model-based Human-Agent Collaboration for Complex Task Solving”, 2024  arXiv:2402.12914 [cs.CL]
  * [5] Isabel O. Gallegos et al.  “Bias and Fairness in Large Language Models: A Survey” arXiv:2309.00770 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2309.00770
  * [6] Silin Gao et al.  “Efficient Tool Use with Chain-of-Abstraction Reasoning” arXiv:2401.17464 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2401.17464
  * [7] Mor Geva et al.  “Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies” arXiv:2101.02235 [cs]  arXiv, 2021  URL: http://arxiv.org/abs/2101.02235
  * [8] Shahriar Golchin and Mihai Surdeanu  “Time Travel in LLMs: Tracing Data Contamination in Large Language Models” arXiv:2308.08493 [cs] version: 3  arXiv, 2024  URL: http://arxiv.org/abs/2308.08493
  * [9] Xudong Guo et al.  “Embodied LLM Agents Learn to Cooperate in Organized Teams”, 2024  arXiv:2403.12482 [cs.AI]
  * [10] Dan Hendrycks et al.  “Measuring Massive Multitask Language Understanding” arXiv:2009.03300 [cs]  arXiv, 2021  URL: http://arxiv.org/abs/2009.03300
  * [11] Sirui Hong et al.  “MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework”, 2023  arXiv:2308.00352 [cs.AI]
  * [12] Xu Huang et al.  “Understanding the planning of LLM agents: A survey”, 2024  arXiv:2402.02716 [cs.AI]
  * [13] Carlos E. Jimenez et al.  “SWE-bench: Can Language Models Resolve Real-World GitHub Issues?” arXiv:2310.06770 [cs]  arXiv, 2023  URL: http://arxiv.org/abs/2310.06770
  * [14] Fangyu Lei et al.  “S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Models” arXiv:2310.15147 [cs]  arXiv, 2023  URL: http://arxiv.org/abs/2310.15147
  * [15] Fangru Lin et al.  “Graph-enhanced Large Language Models in Asynchronous Plan Reasoning” arXiv:2402.02805 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2402.02805
  * [16] Na Liu et al.  “From LLM to Conversational Agent: A Memory Enhanced Architecture with Fine-Tuning of Large Language Models” arXiv:2401.02777 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2401.02777
  * [17] Xiao Liu et al.  “AgentBench: Evaluating LLMs as Agents” arXiv:2308.03688 [cs]  arXiv, 2023  URL: http://arxiv.org/abs/2308.03688
  * [18] Zijun Liu et al.  “Dynamic LLM-Agent Network: An LLM-agent Collaboration Framework with Agent Team Optimization”, 2023  arXiv:2310.02170 [cs.CL]
  * [19] Yohei Nakajima  “yoheinakajima/babyagi” original-date: 2023-04-03T00:40:27Z, 2024  URL: https://github.com/yoheinakajima/babyagi
  * [20] Peter S. Park et al.  “AI Deception: A Survey of Examples, Risks, and Potential Solutions” arXiv:2308.14752 [cs]  arXiv, 2023  URL: http://arxiv.org/abs/2308.14752
  * [21] Greg Serapio-García et al.  “Personality Traits in Large Language Models”, 2023  arXiv:2307.00184 [cs.CL]
  * [22] Zhengliang Shi et al.  “Learning to Use Tools via Cooperative and Interactive Agents” arXiv:2403.03031 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2403.03031
  * [23] Noah Shinn et al.  “Reflexion: Language Agents with Verbal Reinforcement Learning” arXiv:2303.11366 [cs]  arXiv, 2023  URL: http://arxiv.org/abs/2303.11366
  * [24] Amir Taubenfeld, Yaniv Dover, Roi Reichart and Ariel Goldstein  “Systematic Biases in LLM Simulations of Debates” arXiv:2402.04049 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2402.04049
  * [25] Yu Tian et al.  “Evil Geniuses: Delving into the Safety of LLM-based Agents” arXiv:2311.11855 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2311.11855
  * [26] Qineng Wang et al.  “Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?” arXiv:2402.18272 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2402.18272
  * [27] Siyuan Wang et al.  “Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation” arXiv:2402.11443 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2402.11443
  * [28] Zhenhailong Wang et al.  “Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration”, 2024  arXiv:2307.05300 [cs.AI]
  * [29] Jason Wei et al.  “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” arXiv:2201.11903 [cs]  arXiv, 2023  URL: http://arxiv.org/abs/2201.11903
  * [30] Yue Wu, Xuan Tang, Tom M. Mitchell and Yuanzhi Li  “SmartPlay: A Benchmark for LLMs as Intelligent Agents” arXiv:2310.01557 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2310.01557
  * [31] Zhiheng Xi et al.  “The Rise and Potential of Large Language Model Based Agents: A Survey”, 2023  arXiv:2309.07864 [cs.AI]
  * [32] Shunyu Yao et al.  “ReAct: Synergizing Reasoning and Acting in Language Models” arXiv:2210.03629 [cs]  arXiv, 2023  URL: http://arxiv.org/abs/2210.03629
  * [33] Shunyu Yao et al.  “Tree of Thoughts: Deliberate Problem Solving with Large Language Models” arXiv:2305.10601 [cs]  arXiv, 2023  URL: http://arxiv.org/abs/2305.10601
  * [34] Muru Zhang et al.  “How Language Model Hallucinations Can Snowball” arXiv:2305.13534 [cs]  arXiv, 2023  URL: http://arxiv.org/abs/2305.13534
  * [35] Wenting Zhao et al.  “(InThe)WildChat: 570K ChatGPT Interaction Logs In The Wild”  In _The Twelfth International Conference on Learning Representations_ , 2024  URL: https://openreview.net/forum?id=Bl8u7ZRlbM
  * [36] Andy Zhou et al.  “Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models” arXiv:2310.04406 [cs]  arXiv, 2023  URL: http://arxiv.org/abs/2310.04406
  * [37] Kaijie Zhu et al.  “DyVal 2: Dynamic Evaluation of Large Language Models by Meta Probing Agents” arXiv:2402.14865 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2402.14865
  * [38] Kaijie Zhu et al.  “DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks” arXiv:2309.17167 [cs]  arXiv, 2024  URL: http://arxiv.org/abs/2309.17167


# Comparison of security margin estimation methods under various load
configurations

Hannes Hagmar, Robert Eriksson, and Le Anh Tuan  The work presented in this
paper has been financially supported by Energimyndigheten (Swedish Energy
Agency) and Svenska kraftnät (Swedish National Grid) within the SamspEL
program with project number 44358-1.

###### Abstract

The post-contingency loadability limit (PCLL) and the secure operating limit
(SOL) are the two main approaches used in computing the security margins of an
electric power system. While the SOL is significantly more computationally
demanding than the PCLL, it can account for the dynamic response after a
disturbance and generally provides a better measure of the security margin. In
this study, the difference between these two methods is compared and analyzed
for a range of different contingency and load model scenarios. A methodology
to allow a fair comparison between the two security margins is developed and
tested on a modified version of the Nordic32 test system. The study shows that
the SOL can differ significantly from the PCLL, especially when the system has
a high penetration of loads with constant power characteristics, or a large
share of induction motor loads with fast load restoration. The difference
between the methods is also tested for different contingencies, where longer
fault clearing times are shown to significantly increase the difference
between the two margins.

###### Index Terms:

Dynamic security margins, load modelling, security assessment, security margin
estimation

##  I Introduction

Electric power systems are generally operated according to the N-1 contingency
criterion, meaning that the system should be able to withstand the loss of any
single system component, such as a transmission line or a generating unit,
without losing stability. A system capable of handling this is said to be
secure [1]. However, even when the system is secure for a given operation
condition, system operators are also required to know how far the system can
move from its current operating point and still remain secure. To estimate
this, system operators continuously compute security margins, which in turn
represents the available transmission capacity in the system.

There exist two main approaches to compute the security margins of a power
system: the post-contingency loadability limit (PCLL) and the secure operating
limit (SOL) [2]. The PCLL is evaluated by estimating the loadability limit of
a post-contingency operating point, where a solution path is traced by
iteratively increasing the system stress until the system’s critical point is
reached. The characteristics of the iteratively increased system-stress in the
post-contingency setting are similar to that of the slow load restoration that
typically follows in a long-term voltage stability event. Thus, it is common
that the PCLL is also simply referred to as the voltage security margin (VSM).

An alternative measure of the security margin is the SOL, which refers to the
most stressed, pre-contingency operating state in which the system can
withstand a specified set of contingencies. The SOL can account for the
dynamic response after a contingency and provides a generally more accurate
measure of the actual security margin of the system. However, the SOL has been
comparatively less documented in the literature, likely due to the practical
difficulties required in its estimation. The SOL requires numerous time-domain
or quasi steady-state (QSS) simulations to trace the security limit for a set
of different contingencies, a task that is generally too time-consuming to
meet the near real-time monitoring requirements of system operators. In
applications where the SOL is mainly used in monitoring voltage stability, it
is also referred to as the dynamic voltage security margin (DVSM).

Several studies have attempted to provide solutions to the high computational
effort required in estimating the SOL. Methods using QSS simulations, as found
in [3], can reduce the simulation time significantly compared to full time-
domain simulations, but cannot fully incorporate the impact of short-term and
transient effects. In [4], a method that combined QSS and time-domain
simulations was proposed to include the impact of short-term effects. In [5,
6, 7], various machine learning approaches based on training neural networks
were proposed to allow real-time estimation of the DVSM. In [8], a combined
methodology based on validating the estimations of neural networks with actual
time-domain simulations was proposed to overcome the robustness issues that
are commonly related to machine learning methods.

Despite ongoing efforts in improving the computational efficiency of the SOL,
the circumstances when the SOL is to prefer to the PCLL have been relatively
unexplored in the literature. In [9], it was shown that if a system starts at
a stable equilibrium and is slowly stressed towards a critical point without
encountering oscillations or other limit-induced events (e.g. reactive power
limits for generators), static estimation methods are sufficient to locate the
exact critical point experienced by the dynamic system. Thus, in such
circumstances, the security margin computed by PCLL would essentially be equal
to the one computed using the method of the SOL. In [8], a theoretical
comparison between the DVSM and VSM was performed, where the difference
between the two measures was illustrated using a concept called "transient
$P\textrm{-}V$ curves". The study highlighted the importance of load
restoration dynamics on the difference between the two methods but provided no
numerical results. In [10], the SOL was numerically compared to another type
of security margin computed by static $V$-$Q$ curves, in which variations in
the reactive power injection at a bus would affect the voltage at that same
bus. The study concluded that the SOL (or generally a dynamic simulation
approach) is a superior method compared to $V$-$Q$ curves, but since the two
methods are so conceptually different, the results of the two methods could
not directly be compared. In [3], a SOL computed by QSS simulations was
compared to the PCLL, where primarily the impact of post-disturbance control
was investigated. However, as will later be discussed, QSS simulations cannot
fully incorporate the short-term effects after a disturbance and no variations
in the load composition was analyzed in the study.

To address the above-mentioned lack of an accurate comparison between the
methods, this study aims to provide a comprehensive analysis of the difference
between the PCLL and the SOL. The purpose is to isolate the root cause of the
difference between the two security margin methods, referring to that the SOL
could better account for the system’s dynamic response after a disturbance,
and to examine under what circumstances this causes the results of the methods
to differ. The study is, to the authors’ knowledge, the first in developing a
methodology which allows the two margins to be fairly compared for a large
range of different load configurations and disturbance scenarios.

The main contributions of this study are the following:

  * •

A methodology to allow a fair comparison between the PCLL and the SOL is
developed. The purpose of the developed methodology is to ensure that the
difference between the computed margins are due to actual differences of the
security margin approaches, rather than differences in how the simulations are
conducted.

  * •

An extensive numerical comparison between the SOL and the PCLL under a range
of both static and dynamic load model configurations is performed. Different
fault scenarios are examined and discussed in the study. The purpose is to
examine under what circumstances that the SOL are preferable to the PCLL.

The rest of the paper is organized as follows. In Section II, the security
margin definitions for the SOL and the PCLL are presented. In Section III, the
methodology used in computing the margins is presented along with the
simulation platform and the adaptations used to allow a fair comparison
between the security margins. Results and discussion are presented in Section
IV. Concluding remarks are presented in Section V.

##  II Security margin definitions

In this section, a theoretical comparison between PCLL and the SOL is
presented. The conceptual difference of computing the two security margins is
first presented, which is followed by a theoretical analysis of the
circumstances and the instability mechanisms that can cause the two methods to
differ.

###  II-A PCLL versus SOL

The security margin estimation processes for the PCLL and the SOL are
illustrated in Fig. 1. Pre-contingency and post-contingency $P\textrm{-}V$
curves are used where a receiving end voltage in a stressed area is a function
of an increasing (active) power transfer from the system to this receiving
end. An initial, unstressed, operating condition (OC) is the starting point
for the security margin estimation. The security margin is then defined as the
change in loading from the initial OC to the $N$-$1$ critical point. It should
be noted that in actual applications, the limit is often smaller due to the
other stopping criteria such as too low system voltages. However, for better
illustration purposes, the former limit is used here.

Figure 1: Security margin estimation for PCLL and SOL. 1’ and 2’ illustrates
the computation path for the PCLL; 1 and 2 illustrates the computation path
for the SOL.

In PCLL estimation, a post-contingency operating point is found by first
introducing a contingency on the initial OC, which is typically followed by
solving the resulting power flow study. This is illustrated in Fig. 1 by
moving along arrow $\boldsymbol{1^{\prime}}$. The security margin is then
traced along the solution path by iteratively and step-wise increasing the
system stress until the critical point is reached, moving along the arrow
$\boldsymbol{2^{\prime}}$. Parameterized continuation methods based on static
load flow solutions, generally referred to as continuation power flow (CPF),
are commonly used to avoid convergence problems close to the critical point of
the system [11]. The distance between the pre-contingency operating point and
the $N$-$1$ critical point represents the PCLL.

In the estimation of the SOL, the dynamic security of the system is tested
with an increasing stress level, illustrated by arrow $\boldsymbol{1}$ in Fig.
1 [3]. For every new stressed pre-contingency OC, the system response
following the disturbance is studied. The final, pre-contingency OC that is
tested and can provide a stable operating point is illustrated by moving along
arrow $\boldsymbol{2}$ in Fig. 1. The state transition following arrow
$\boldsymbol{2}$ can generally not be computed using static methods as it can
result in numerical divergence. Instead, methods based on dynamic (or QSS)
simulations are generally required. The increased loadability between the
initial OC and the most stressed pre-contingency OC that can still handle a
dimensioning contingency without causing instability represents the SOL.

###  II-B System dynamics and instability mechanisms

Loads are often recognized to maintain a constant power characteristics in a
long-term system perspective, but do not generally behave as such following a
disturbance. Assuming a sudden voltage change, loads will initially drop
according to their instantaneous characteristics [12]. Then, the impedance or
the drawn current is adjusted to restore the load to its original level; a
process that can be exemplified by the automatic changes in the slip of
induction motors or by changes in tap positions to increase the voltage for
loads behind load tap changers (LTCs). The overall load restoration after a
disturbance is generally assumed to act significantly slower than the dynamics
of other system components, such as the dynamics of generators and excitation
systems. The PCLL is based on this time-scale decomposition, where short-term
dynamics, such as the slip of induction motors, or generator and excitation
dynamics, are assumed to be in equilibrium. Using this assumption, the
loadability limit of the post-disturbance system can be found even though only
static estimation methods are used to trace the security margin.

In [12] and [8], the concept of transient $P\textrm{-}V$ curves was used to
allow visualization and analysis of short-term dynamics using $P\textrm{-}V$
curves. In the analysis, the post-disturbance $P$-$V$ curve is not fixed in
time, but is allowed to be affected by short-term system dynamics of, for
instance, excitation systems. Nor is the load curve fixed in time, which
allows the load restoration that follows after a disturbance to be
illustrated. In Fig. 2, transient $P\textrm{-}V$ curves are used to illustrate
a system that, when assuming the short-term dynamics are in equilibrium, could
appear to be secure. However, when the short-term dynamics are taken into
account, there is a loss of post-disturbance equilibrium of the short-term
dynamics, and the disturbance would in fact cause the system to become
unstable. The load restoration curves and the transient $P\textrm{-}V$ curves
are illustrated using different shades of grey, where a lighter shade
indicates closer in time after the disturbance. The time just after a
disturbance is indicated by $t_{1}$; $t_{2}$ relates to a short time after the
disturbance; $t_{3}$ relates to the time when all short-term dynamics are in
equilibrium. The load is assumed to have long-term constant power
characteristic, but just after a disturbance, the load will initially change
to a constant impedance characteristic. Then, by fast load restoration, the
load is quickly restored to the pre-disturbance level.

The initial OC is found in point $\boldsymbol{A^{\prime}}$. Just after the
disturbance, the short-term dynamics of system components such as generators
or excitation systems will not yet have stabilized, which has the effect of
shifting the post-disturbance $P$-$V$ curve to the left. As a result of the
initial load characteristics and the shifted transient $P$-$V$ curve, the
operating point moves along the arrow to operating point
$\boldsymbol{B^{\prime}}$. After this transition, both the load and the post-
disturbance $P$-$V$ curve are shifted towards their stable counterparts.
However, due to the fast load dynamics in this case, there exists no
intersection between the load curve and the post-disturbance $P$-$V$ curve at
$t_{2}$ (the area indicated with the red dotted circle), and without any
emergency control actions, the system stability would be lost. Thus, despite
the fact that the post-disturbance $P$-$V$ curve and the load characteristic
at $t_{3}$ still intersect in this case, the system would have become
unstable.

Figure 2: Transient $P\textrm{-}V$ curves illustrating a short-term
instability event [8]

Instability caused by the short-term dynamics that follows a disturbance can
generally be divided into three different instability mechanisms [2, 1]:

  * •

Loss of post-disturbance equilibrium of short-term dynamics: Typically
exemplified by the stalling of induction motors after a disturbance causing
the transmission impedance to increase. Due to the increased transmission
impedance, the mechanical and electrical torque curves of the motor may not
intersect, causing the system to lack a post-disturbance equilibrium, similar
to the case illustrated in Fig. 2.

  * •

Lack of attraction towards the stable post-disturbance equilibrium of short-
term dynamics: Typically exemplified by transient angle instability and the
loss of synchronism by one (or several) generators following a too slow fault
clearing.

  * •

Oscillatory instability of the post-disturbance equilibrium: Typically
exemplified by rotor angle stability, in which the equilibrium between
electromagnetic torque and mechanical torque of synchronous machines in the
system is disturbed. Instability may be caused by increasing angular swings of
some generators leading to their loss of synchronism with other generators
[1].

Typically, time-domain simulations are required to capture the short-term
dynamics after a large disturbance. SOLs computed using QSS simulations can
typically not account for the short-term dynamics that follows after a
disturbance, and are thus better suited to only monitor long-term voltage
instability phenomena. However, combinations of time-domain simulations and
QSS, as proposed in [4], can use time-domain simulations to model the system
during the short-term period following a disturbance, followed by QSS
simulations used to simulate the long-term interval after the short-term
effects are finalized. However, short-term instability may also be induced by
long-term dynamics, where the system degradation caused by long-term
instability eventually can trigger the above mentioned short-term events [2].
It should be noted that SOLs computed by combinations of time-domain
simulations and QSS, as proposed in [4], cannot capture this type of events.

##  III Methodology for security margin computations

In this section, the methodology used in the comparison between the PCLL and
the SOL is presented. The load models and a description of the test system are
presented along with the required adaptions. Finally, the methodology used to
allow a fair comparison of the PCLL and the SOL is presented.

###  III-A Load models

The power consumption of loads are generally affected by the system voltages
and different load models are often used to characterize this relationship. A
traditional load model used in both static and dynamic stability analysis is
the ZIP model, which is made up of three components: constant impedance ($Z$),
constant current ($I$), and constant power ($P$). The characteristics of the
ZIP model is given by [2]:

$\displaystyle
P=zP_{0}\left[a_{P}\left(\frac{V}{V_{0}}\right)^{2}+b_{P}\frac{V}{V_{0}}+c_{P}\right]$
(1a) $\displaystyle
Q=zQ_{0}\left[a_{Q}\left(\frac{V}{V_{0}}\right)^{2}+b_{Q}\frac{V}{V_{0}}+c_{Q}\right]$
(1b)

where $a_{P}+b_{P}+c_{P}$ = $a_{Q}+b_{P}+c_{Q}$ = $1$, $P_{0}$ and $Q_{0}$ are
the real and reactive powers consumed at a reference voltage $V_{0}$, given
that $z$ = $1$. $V$ is the actual voltage and $z$ is a variable indicating the
actual loading of the system [2]. The constants $a_{x}$, $b_{x}$, and $c_{x}$
represents the share of constant impedance, constant current, and constant
power of the load, respectively.

Although simple and widely used in security analysis [13], the ZIP model
cannot model any dynamic behaviour of the loads themselves. The significance
of induction motor loads and other fast-acting dynamic loads are often
highlighted in system stability studies. Induction motors (IM) are
characterized by fast load restoration dynamics (often in the time frame of a
second) and have a high reactive power demand. Induction motors are also prone
to stalling, which may cause the motor to draw high reactive currents from the
grid, resulting in a deteriorating effect on the system stability [2]. In
PSS®E, a complex load model (CLOD) can be used to represent a bundled mix of
loads with different dynamic load characteristics into a single model. In Fig.
3, a schematic of the CLOD model is presented. The transformer and feeders
connecting the system bus to the load bus are modelled as a single impedance.
At the load bus, different percentages of large and small IMs, discharge
lighting loads, transformer saturation, and constant power loads can be
modelled. The remaining part of the load is modelled as a polynomial load
where the voltage dependency of the active load is controlled through a
constant $K_{p}$. The performance curves of the two motor models, the
discharge lightning model, and the transformer saturation model is further
detailed in [14].

Figure 3: Overview of the complex load model (CLOD) [14].

###  III-B Simulation test system

All simulations have been tested on the slightly modified version of the
Nordic32 test system, detailed in [15]. The main characteristic of the system
is sensitivity towards long-term voltage instability, although the system can
exhibit other types of instabilities as well. A one-line diagram of the test
system is presented in Fig. 4. The security margins are computed by increasing
the loading in the area "Central", while the generation in the area "North" is
increased by a corresponding quantity. The starting point for all scenarios is
the secure "operating point B" as defined in [15].

Figure 4: One-line diagram of the modified Nordic32 system [15]

All simulations were carried out using PSS®E version 35.0. To ensure numerical
stability during the dynamical simulation runs, a short integration step of
0.0005 seconds was used in the simulations. In certain sensitive scenarios,
such as when the simulations resulted in a non-converging dynamic simulation,
the integration step was at times varied to provide a converging simulation.
To more easily allow direct adjustments of the generator power during
dynamical simulations, the governor models were changed to the more generic
governor model of IEESGO [14], illustrated in Fig. 5. To allow almost
instantaneous changes in the mechanical input to generators given the governor
reference changes, all controller lags, delays, and constant parameters was
chosen zero, except for the governor lag constant $T_{3}$ which was given a
value of 0.01.

Figure 5: Block diagram of the IEESGO Model [14]

###  III-C Methodology and adaptations

To compare the two conceptually different methods of PCLL and SOL is not
trivial, as one is computed using a static model of the system, while the
other is generally estimated using a dynamic model. To ensure that the
difference in the computed security margins was not caused by differences in
how the simulations were conducted, but rather by the fact that the SOL could
better account for the system’s dynamic response after a disturbance, a few
adaptations of the methods were required. Instead of using CPF methods to
compute the PCLL, we adopted a method that slowly ramps up the system stress
in a dynamic simulation setting; an approach similar to the one used in [15].
This approach allows the PCLL to be performed in a dynamic setting while
mimicking how the system stress would have been increased if it would have
been performed in a static setting. The advantage by adopting this methodology
is that the loading and the generation set points could be increased in the
exact same way for both the computation of the PCLL and the SOL.

####  III-C1 Steps for PCLL computation

The steps used in computing the PCLL were the following:

  * •

Initialization: The PCLL computation was initialized by applying a chosen
contingency in the system in a dynamic simulation from the base case. The
dynamic simulation ran until the system was fully stabilized.

  * •

Increase system stress: Once the system had stabilized after the initial
disturbance, the system stress was increased in small increments of 1 MW,
which was distributed among all the loads in the "Central" area. To reduce the
required simulation time, the system stress was for certain fault scenarios
(scenario A and B) initially increased in larger increments (5 MW) up until a
total system stress of 300 MW, since such low stress levels were found to not
cause instability in the system. The different fault scenarios are further
discussed in Section IV-A. Simultaneously, the governor reference for all the
generation units in the "North" area was increased to compensate for the
increased load level. The power factor of each load was kept constant. The
added load for both the PCLL and SOL were computed as a nominal load increase
at 1.0 pu voltage to ensure that the same amount of load was added for both
methods regardless of the current load voltage in the dynamic simulation.
Increased active line losses caused by the increased system stress were
automatically compensated by the generators’ governor systems, while the
reactive line losses were automatically compensated by the generators’
excitation systems.

  * •

Check stability criterion: After the increased system stress, the simulation
continued to run until the system either stabilized or until a stability
criterion was violated. The system was considered unstable if any bus voltage
in the system was lower than 0.9 pu. Although the modified Nordic32 test
system is characterized by sensitivity towards voltage instability, other
types of instability can violate the stability criterion. For instance,
transient angle instability can cause locally low voltages due to lost
synchronism of certain generators.

  * •

Re-iterate: The system stress was increased until the system eventually
violates the stability criterion. The difference in loading from the base case
to the final stable operating point in the stressed post-contingency system
represents the computed PCLL.

####  III-C2 Steps for SOL computation

The SOL was computed similarly to the PCLL, but by instead stressing the
system in its pre-contingency configuration and then introducing the
disturbance. The steps used in the computation of the SOLs were the following:

  * •

Initialize simulation and increase system stress: When the ZIP-model was used
to model the loads, the dynamic simulation was initialized at the start. The
system stress was then increased in its pre-contingency base case in small
increments of 1 MW, in the same way as was done for the PCLL computation in
its post-contingency configuration. The small step-size in system stress was
chosen to allow the illustration of the security margins using $P\textrm{-}V$
curves. In more general applications, faster methods such as the binary search
method [3] or the dual binary search method [8], can otherwise be used to
compute the SOL. Adjustments for CLOD-model: When using the CLOD-model to
model the loads in the system, the increased system stress was required to be
added in a static load flow scenario, which was then converted for dynamical
studies. This causes a small difference in how the generation in the "North"
area is increased. For the PCLL and the SOL case for the ZIP-model, the
generation is increased by changing the generators mechanical power by
incrementally increasing the governor reference. When the generation is
increased using the CLOD-model, the active generation is directly changed in
the static load flow scenario which creates a small difference of how the
system stress is increased.

  * •

Introduce disturbance and check stability: A disturbance was then applied in
the system. A final end time of the dynamic simulation of 1000 seconds was
used. The system was considered unstable if any bus voltage in the system was
lower than 0.9 pu at the end of the simulation. The simulation was also
stopped in advance if any bus voltage decreased below 0.7 pu (still allowing
the system to first stabilize for 20 seconds after the disturbance).

  * •

Re-iterate: If the system was stable, the system was reinitialized to the last
pre-contingency state, followed by increasing the system stress by an
additional 1 MW and applying the same disturbance. The SOL is then computed
from the difference in loading from the base case to the final stable
operating point in the stressed post-contingency system.

##  IV Simulation results and discussion

In this section, the results of the numerical comparison between the PCLL and
the SOL is presented. Three different contingency scenarios were tested. The
results of PCLL and the SOL computation are presented for each contingency
scenario and each load model configuration.

###  IV-A Contingency scenarios and loading scenarios

TABLE I: Computed PCLL and SOL for different loading and contingency scenarios |  | Constant |  |  |  |  |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
|  | MVA | I | Z |  |  | Scenario A |  |  | Scenario B |  |  | Scenario C
Scenario |  | (P/Q) | (P/Q) | (P/Q) |  |  | PCLL | SOL |  |  | PCLL | SOL |  |  | PCLL | SOL
number |  | [%] | [%] | [%] |  |  | [MW] | [MW] |  |  | [MW] | [MW] |  |  | [MW] | [MW]
1 |  | 100/0 | 0/0 | 0/100 |  |  | 372 | 142 |  |  | 372 | 5 |  |  | - | -
2 |  | 95/0 | 5/0 | 0/100 |  |  | 378 | 245 |  |  | 378 | 174 |  |  | - | -
3 |  | 90/0 | 10/0 | 0/100 |  |  | 386 | 311 |  |  | 386 | 282 |  |  | 150 | -
4 |  | 80/0 | 20/0 | 0/100 |  |  | 399 | 398 |  |  | 399 | 398 |  |  | 173 | 5
5 |  | 50/0 | 50/0 | 0/100 |  |  | 425 | 417 |  |  | 425 | 417 |  |  | 281 | 267
6 |  | 95/0 | 5/50 | 0/50 |  |  | 265 | 152 |  |  | 265 | 152 |  |  | - | -
7 |  | 80/0 | 20/50 | 0/50 |  |  | 393 | 363 |  |  | 393 | 255 |  |  | 126 | 9
8 |  | 50/0 | 50/50 | 0/50 |  |  | 418 | 411 |  |  | 418 | 411 |  |  | 259 | 259
9 |  | 0/0 | 100/0 | 0/100 |  |  | 450 | 441 |  |  | 450 | 440 |  |  | 336 | 268
10 |  | 0/0 | 50/0 | 50/100 |  |  | 473 | 463 |  |  | 473 | 440 |  |  | 375 | 290
11 |  | 0/0 | 20/0 | 80/100 |  |  | 492 | 480 |  |  | 492 | 475 |  |  | 393 | 301
12 |  | 0/0 | 0/0 | 100/100 |  |  | 497 | 493 |  |  | 497 | 489 |  |  | 408 | 305

The following contingency scenarios were examined:

  * •

Scenario A: A three-phased fault for 40 milliseconds, followed by tripping the
faulted line. The faulted line is the one connecting the two areas "North" and
"Central" between bus 4032 to bus 4044.

  * •

Scenario B: A longer three-phased fault for 100 milliseconds, followed by
tripping the faulted line. The faulted line is the one connecting the two
areas "North" and "Central" between bus 4032 to bus 4044.

  * •

Scenario C: Tripping the of generator "g16b" located at bus 4051 in the
"Central" area.

For each of the contingency scenarios, different combinations of the ZIP-load
were tested for both the PCLL and the SOL. In addition, the SOL was computed
for different compositions when the CLOD model was used to model the loads in
the system. The CLOD model was found to be numerically unstable for longer
fault clearing times. Thus, we only provide a comparison of the results for
Scenario A with a fault clearing time of 40 milliseconds.

###  IV-B Simulation results

The PCLL and SOL results for each scenario and each load configuration using
the ZIP-model are presented in Table I. The SOL results for scenario A and
different configurations of the CLOD model are presented in Table II. The
largest difference between the PCLL and SOL is found for cases with a high
share of constant power characteristic of the active part of the loads. For
instance, for scenario 1A with fully constant power characteristics for the
active part of the load and fully constant impedance characteristics for the
reactive part of the load, the SOL was only 142 MW, while the PCLL was found
to be 372 MW. The difference between the two security margin methods then
reduces rapidly with a decreasing level of constant power characteristics on
the active part of the load. Already at slightly lower levels of constant
power loads, for instance, in Scenario 4A, the difference between the SOL and
the PCLL becomes close to negligible. In Fig. 6, the post-disturbance
$P\textrm{-}V$ curves of the transmission side of bus 1041 are illustrated,
respectively, for Scenario 1A. The $P\textrm{-}V$ curves are computed by
sampling the voltage magnitude when the system had stabilized after each
dynamic simulation. Here, with a fully constant power characteristic of the
active part of the loads, the $P\textrm{-}V$ curves are almost identical for
both the PCLL and the SOL up until the collapse point for the SOL.

Figure 6: $P\textrm{-}V$ curves computed at bus 1041 for scenario 1A

The difference between the PCLL and the SOL is more significant for Scenario B
when a longer fault clearing time was used in the simulations. For instance,
in Scenario 1B, the SOL was estimated to only 5 MW, compared to 373 MW for the
PCLL. With reference to the discussion with the transient $P\textrm{-}V$
curves presented in Section II-B, a longer fault clearing time would have the
effect of shifting the post-disturbance $P$-$V$ curve for a longer time to the
left, causing the system to lack attraction towards a stable post-disturbance
equilibrium. Yet again, the difference between the two security margins
decreases rapidly as the share of constant active power loads decrease. For
instance, in Scenario 4B with an 80% share constant active power load, and the
remaining part of the active load being of constant current characteristics,
the SOL and the PCLL are almost identical. The post-disturbance $P\textrm{-}V$
curves of scenario 4B on the transmission side of bus 1041 are illustrated in
Fig. 7. The figure shows that although the computed $P\textrm{-}V$ curves of
the SOL is slightly below that of the PCLL, the two security margins find
almost the same critical point of the system.

For all cases, except when the load is of constant power characteristics, the
$P\textrm{-}V$ curves computed using the SOL are slightly below the ones
computed using the PCLL. Although the initial response of the excitation
systems used in the Nordic32 test system is fast, there is an integrating part
of the control system which takes a longer time until the voltage magnitudes
of the generators are restored to their pre-disturbance set-point (differing
slightly due to the droop in the automatic voltage regulation). In the PCLL
case, this voltage restoration is allowed to fully stabilize after the initial
disturbance before the system stress is added to the system. This is not the
case for the SOL, in which the system is stressed before the disturbance is
applied to the system. In turn, this causes LTCs and OELs to act earlier for a
lower level of system stress, causing the magnitude of the post-disturbance
voltages to be generally lower.

In Scenario C, the chosen contingency was the disconnection of the generator
"g16b", located in the "Central" area. This disturbance was found to more
severely affect the stability of the system, and several of the scenarios
resulted in negative and/or suddenly unstable simulations. Furthermore, the
computed security margins for this case were also found to be sensitive to
changes in the simulation setup, such as the used integration time step.
Again, the computed SOL was found to be generally smaller or equal than the
PCLL for most load configurations, but more care should be used when comparing
the results as the difference could be caused by numerical instability during
the dynamic simulations.

Figure 7: $P\textrm{-}V$ curves computed at bus 1041 for scenario 4B

In Table II, the computed SOLs for scenario A when using different
configurations of the CLOD model are presented. The scenarios are generated by
varying the load composition, consisting of large induction motors (LIMs),
small induction motors (SIMs), discharge lightning (DL), transformer
saturation (TS), constant power loads (MVA), and the remaining load (Kp) which
is of constant impedance characteristics. Unsurprisingly, the computed SOL was
the lowest when there was a large share of motor loads in the system. When the
loads were modelled with a too high share of motor loads, such as scenario
16A, the computed SOL for the base case was negative. There was a relatively
large difference between the computed SOL for scenario 20A with a 35% share of
LIM loads and 20% share small motor loads, and scenario 23A with a 20% share
of large motor loads and 35% share of SIM loads. LIM loads generally draws a
higher reactive current during instances of low system voltages than SIM
loads, which may have caused the computed SOL to differ from 90 MW for
scenario 20A to 196 MW for scenario 23A.

In all scenarios where the CLOD model was used, the system crashed during the
transient state just after the disturbance. In Fig 8, scenario 23A is
illustrated with varying levels of system stress. For the illustrated lower
system stress levels of 150 MW and 334 MW, the system is stable and is capable
of handling the dynamics following the disturbance. However, for a stress
level of 335 MW, the system crashes almost instantly during the transient
state of the disturbance. The CLOD models were found to be particularly
sensitive towards long fault clearing times, and the Nordic32 test system
consistently crashed when using a longer fault clearing time (such as 0.1
seconds). The difference between the two security margins is thus likely even
greater if breakers with longer fault clearing times can be assumed to
dominate the system.

TABLE II: SOLs for different load configurations of the CLOD model | Complex load (CLOD) model parameters | Scenario A
---|---|---
Scenario | LIM | SIM | DL | TS | MVA | Kp | SOL
number | [%] | [%] | [%] | [%] | [%] | [MW] |
16 | 35 | 35 | 5 | 5 | 10 | 2 | -
17 | 30 | 30 | 5 | 5 | 10 | 2 | 64
18 | 20 | 20 | 5 | 5 | 10 | 2 | 377
19 | 35 | 30 | 5 | 5 | 10 | 2 | 12
20 | 35 | 20 | 5 | 5 | 10 | 2 | 92
21 | 35 | 10 | 5 | 5 | 10 | 2 | 334
22 | 30 | 35 | 5 | 5 | 10 | 2 | 28
23 | 20 | 35 | 5 | 5 | 10 | 2 | 198
24 | 10 | 35 | 5 | 5 | 10 | 2 | 375
Figure 8: Voltage evolution for bus 1041 for Scenario 23A for different levels
of system stress

###  IV-C Discussion

The results in the previous section show that although the same operating
point has been used as a starting point for all scenarios, the PCLL and the
SOL differ significantly depending on the current load configuration and the
type of fault that is considered. The largest difference between the two
security margin methods was found when either the loads were of high constant
power characteristics or consisted of a large penetration of induction motor
loads. These results thus confirm the well-known fact that loads with fast
restoration dynamics (where a constant power characteristic can be considered
a theoretic extreme case) will deteriorate the system stability, and
illustrate how significant this impact may be on the computed security
margins. However, care should be taken when generalizing the results of this
study to real power systems with different characteristics. For instance,
although the difference between the SOL and the PCLL in this study was found
to be negligible whenever the share of constant power characteristic of the
active part of the loads was lower than 50 %, this is not necessarily the case
for other systems with different dynamics. System operators would thus be
required to perform a similar analysis on their specific systems to analyze
the difference between the PCLL and the SOL.

The stability assessment practice of many system operators is, to the authors’
best knowledge, to compute security margin estimations computed by PCLLs,
often in combination with dynamic security assessment (DSA). However, in DSA
only the current operating point’s capability to withstand a disturbance is
assessed, but it provides no information regarding the margin towards
instability. We believe that if system operators continue to rely on
conventional security margins computed by the PCLL, it is important to verify
the reliability of those security margins to avoid either overly optimistic
security margins or to avoid having to add unnecessary large reliability
margins to the computed security margins. To account for modelling
inaccuracies, transmission reliability margins are often added to ensure that
modelling inaccuracies will not cause the system to be operated unknowingly in
a non-secure operating state. Thus, if more accurate methods to determine
security margins are used, such as the SOL, these reliability margins may
theoretically be reduced and the system operators could more efficiently
utilize the existing transmission capacity.

Dynamic load modelling may also become increasingly important in the future,
as more loads are expected to be controlled through power electronically-
controlled interfaces. These types of loads, such as electric vehicles
chargers, inhibits very fast dynamic responses after disturbances [16].
Despite this, dynamic load models are still relatively unused in the industry.
In a large survey study from 2013 on international industry practice on power
system load modelling, it was shown that about 70% of system operators and
utilities around the world still used only static load models for power system
stability studies [17]. A drawback of using more advanced load models is that
the load composition is often partly unknown to system operators, and it is
thus more straightforward to use the simplified static load models. Another
drawback is the increase in computational requirement during simulations,
which reduces their applicability in real-time applications. However, although
complex load models do not necessarily need to be used in real-time
applications, sensitivity analyses can preferably be performed using these
models, so that the impact of various degrees of motor loads and other types
of loads on the stability of a system can be studied.

##  V Conclusions

In this paper, the PCLL and the SOL have been compared and studied under
various load configurations and disturbance scenarios. A methodology was
developed to allow a fair comparison between the two methods to ensure that
the difference in the computed security margins was due to actual differences
of the security margin approaches, and not caused by differences in the
simulation setups. The numerical comparison shows that the two methods differ
significantly under various load configurations and fault scenarios. The
largest difference between the two methods was found when the loads were of
high constant power characteristics or when the loads consisted of a large
share of induction motor loads. Furthermore, the fault clearing time is found
to be especially important and a longer fault clearing time caused the SOL to
become significantly smaller than the PCLL. The results highlight the
importance of load modelling and show that if a power system can be expected
to have a large share of loads with fast restoration dynamics, the
conventional method of using PCLL to compute the security margins can provide
overly optimistic values of the actual security margin.

## References

  * [1] P. Kundur _et al._ , “Definition and classification of power system stability ieee/cigre joint task force on stability terms and definitions,” _IEEE Trans. Power Syst._ , vol. 19, no. 3, pp. 1387–1401, Aug 2004.
  * [2] T. V. Cutsem and C. Vournas, _Voltage stability of electric power systems_. Boston: Kluwer Academic Publishers, 1998.
  * [3] T. Van Cutsem, C. Moisse, and R. Mailhot, “Determination of secure operating limits with respect to voltage collapse,” _IEEE Trans. Power Syst._ , vol. 14, no. 1, pp. 327–335, Feb 1999.
  * [4] T. Van Cutsem, M.-E. Grenier, and D. Lefebvre, “Combined detailed and quasi steady-state time simulations for large-disturbance analysis,” _International Journal of Electrical Power & Energy Systems_, vol. 28, no. 9, pp. 634 – 642, 2006.
  * [5] A. Sittithumwat and K. Tomsovic, “Dynamic security margin estimation using artificial neural networks,” in _IEEE Power Engineering Society Summer Meeting,_ , vol. 3, Chicago, IL, July 2002, pp. 1322–1327.
  * [6] N. Amjady, “Dynamic voltage security assessment by a neural network based method,” _Electric Power Systems Research_ , vol. 66, no. 3, pp. 215 – 226, 2003.
  * [7] M. V. Baghmisheh and H. Razmi, “Dynamic voltage stability assessment of power transmission systems using neural networks,” _Energy Conversion and Management_ , vol. 49, no. 1, pp. 1 – 7, 2007.
  * [8] H. Hagmar, R. Eriksson, and A. T. Le, “Fast dynamic voltage security margin estimation: concept and development,” _IET Smart Grid_ , vol. 3, no. 4, 04 2020.
  * [9] I. Dobson, “The irrelevance of load dynamics for the loading margin to voltage collapse and its sensitivities,” in _Bulk power system voltage phenomena - III: Voltage stability, security & control_, 1994, pp. 509–518.
  * [10] B. H. Chowdhury and C. W. Taylor, “Voltage stability analysis: V-Q power flow simulation versus dynamic simulation,” _IEEE Trans. Power Syst._ , vol. 15, no. 4, pp. 1354–1359, Nov 2000.
  * [11] V. Ajjarapu and C. Christy, “The continuation power flow: a tool for steady state voltage stability analysis,” _IEEE Trans. Power Syst._ , vol. 7, no. 1, pp. 416–423, Feb 1992.
  * [12] M. K. Pal, “Voltage stability conditions considering load characteristics,” _IEEE Trans. Power Syst._ , vol. 7, no. 1, pp. 243–249, Feb 1992.
  * [13] A. Arif _et al._ , “Load modeling—a review,” _IEEE Trans. Smart Grid_ , vol. 9, no. 6, pp. 5986–5999, 2018.
  * [14] _PSS®E 34.2.0 Program Application Guide: Volume II_ , Siemens Power Technologies International, Schenectady, NY, Apr. 2017.
  * [15] T. Van Cutsem _et al._ , “Test systems for voltage stability studies,” _IEEE Trans. Power Syst._ , vol. 35, no. 5, pp. 4078–4087, 2020.
  * [16] D. Mao, K. Potty, and J. Wang, “The impact of power-electronics-based load dynamics on large-disturbance voltage stability,” in _2018 IEEE Power Energy Society General Meeting (PESGM)_ , 2018, pp. 1–5.
  * [17] J. V. Milanovic _et al._ , “International industry practice on power system load modeling,” _IEEE Trans. Power Syst._ , vol. 28, no. 3, pp. 3038–3046, 2013.

Hannes Hagmar (S’17) received the M.Sc. degree in electric power engineering
from Chalmers University of Technology, Gothenburg, Sweden in 2016. Between
2016 to 2017, he worked at RISE Research Institutes of Sweden with research in
electric transmission systems and measurement technology. He is currently
pursuing a Ph.D. degree at Chalmers University of Technology. His research
interest includes power system dynamics and stability, integration of
renewables, and machine learning.
---
Robert Eriksson (SM’16) received the M.Sc. and Ph.D. degrees in electrical
engineering from the KTH Royal Institute of Technology, Stockholm, Sweden, in
2005 and 2011, respectively. He held an associate professor position at the
Center for Electric Power and Energy, DTU Technical University of Denmark,
from 2013 to 2015. He is currently with the Swedish National Grid (Svenska
kraftnät), Department of Power Systems. He is also an adjunct professor at the
KTH Royal Institute of Technology. His current research interests include
power system dynamics and stability, automatic control, and HVDC systems.
---
Le Anh Tuan (S’01–M’09) received a M.Sc. degree in energy economics from the
Asian Institute of Technology, Bangkok, Thailand, in 1997, and a Ph.D. degree
in power systems from Chalmers University of Technology, Gothenburg, Sweden,
in 2004. He is currently a Senior Lecturer with the Division of Electric Power
Engineering, Department of Energy and Environment, Chalmers University of
Technology. His current research interests include power system operation and
planning, power market and deregulation issues, grid integration of renewable
energy, and plug-in electric vehicles.
---