# An Open Optimal Power Flow Model for the Australian National Electricity
Market

###### Abstract

The Australian National Electricity Market (NEM) is a complex energy market
that faces challenges due to the increasing number of distributed energy
resources (DERs) and the transition to a net-zero emissions target. Power
system modelling plays a crucial role in addressing these challenges by
providing insights into different scenarios and informing decision-making.
However, accessing power system data containing sensitive information can be a
concern. Synthetic data offer a solution by allowing researchers to analyze
and develop new methods while protecting confidential information. This paper
utilizes an existing synthetic network model based on the NEM (‘S-NEM2300’-bus
system) to develop a benchmark for power system optimization studies. The
model is derived and enhanced using PowerModels.jl and MATPOWER data models,
and feasibility is ensured through power flow and optimal power flow studies.
The resulting benchmark model, called ‘S-NEM2000’-bus system, is validated and
enriched with additional parameters such a thermal limits, generation fuel
categories and cost models. The ‘S-NEM2000’-bus system is an _open_
333https://github.com/csiro-energy-systems/Synthetic-NEM-2000bus-Data dataset
which provides a valuable resource for optimization studies in the power
system domain.

Rahmat Heidari111R. Heidari and M. Amos are with the Energy Business Unit,
Commonwealth Scientific and Industrial Research Organisation (CSIRO),
Newcastle, NSW 2300, Australia, (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>, Matthew Amos11footnotemark: 1, Frederik Geth222F. Geth
is with the GridQube, Brisbane, QLD 4300, Australia, (e-mail:
<EMAIL_ADDRESS>
##### Disclaimer:

The open network data developed in this study, specifically the
‘S-NEM2000’-bus benchmark model, is a representation created for research
purposes and does not reflect the actual NEM. It is important to note that
this model is a work in progress and may not include all the features and
components present in the real NEM. Furthermore, the model can be further
improved by incorporating additional features and elements and refining
existing components. The ongoing development and growth of this work aim to
enhance the representation of the NEM and its associated power system for
future studies and analyses. The authors invite collaborators to reach out
through the issue tracker in the repository for any inquiries, feature
requests, or collaboration opportunities.

##  1 Introduction

The Australian National Electricity Market (NEM) is a complex energy market
that operates across five states and territories in Australia and regulates
and manages the supply and demand of electricity within the market. The NEM,
shown in Figure 1, is one of the largest interconnected power systems in the
world, covering 40,000 km of transmission lines and cables, and servicing
around 10 million
customers444https://www.aemo.com.au/-/media/Files/Electricity/NEM/National-
Electricity-Market-Fact-Sheet.pdf. The market operates on a real-time basis,
balancing the demand for electricity with the supply of electricity from a
range of sources, including large-scale generators, renewable energy systems,
and distributed energy resources (DERs).

Figure 1: One-line diagram of the Australian NEM [1].

The NEM is facing challenges due to the rising number of DER such as rooftop
solar, batteries, and electric vehicles. Challenges include integrating DERs
into the existing grid infrastructure, ensuring system stability and
reliability, and addressing grid congestion in areas with high penetration of
rooftop solar, which can cause voltage fluctuations and result in power
outages [2]. This requires significant investment in network infrastructure,
as well as changes to the regulatory framework to support the integration of
DERs. The transition to a net-zero emissions target in the NEM also presents
challenges, including the need for energy market reform and changes to
regulatory frameworks to incentivize renewable energy resources deployment and
emissions reduction. Additionally, the uncertainty surrounding government
policy and support for renewable energy can make it difficult for investors to
make long-term investment decisions.

Power system modelling is essential in dealing with the above-mentioned
challenges in the electricity sector. Modelling tools provide insights into
the impacts of different scenarios, such as the deployment of renewable energy
sources or grid upgrades, and inform decision-making related to planning,
design, operation, policy, regulation, and investment. This helps to ensure
the reliability, sustainability, and affordability of the power system.

Despite necessity of access to power system data that enables reliable
modelling and decision-making, power system data can contain confidential
information such as operational parameters, network topology, and customer
data. By using _synthetic data_ , this sensitive information can be protected
while still allowing researchers and engineers to analyse and develop new
methods. Synthetic data are beneficial in the power systems context as they
allow engineers to simulate and analyse power systems under different
scenarios, identify potential issues, and test new algorithms and techniques.
This helps to ensure the reliability and safety of the power system, which is
critical to the functioning of society as a whole.

Furthermore, openly accessible data, _open data_ , plays an important role in
power systems as it contributes to increased transparency, efficiency, and
innovation. Open data encourages collaboration and knowledge sharing among
different organizations and researchers, leading to the development of best
practices and the advancement of the power sector as a whole.

The authors of [3, 4] provided a comprehensive library of all publicly
available transmission network datasets (to date of the paper) under a common
data format to be used for AC power flow optimisation studies. To this end,
the authors updated the dataset by reconstructing the missing data and adding
key parameters such as generator cost functions and thermal limits, for
algorithmic benchmarking purposes. More recently, a detailed network model of
the European countries has been released in [5] to study market simulations of
different combinations of wind and transmission capacity that is installed in
the North Sea. The modelling framework and datasets are open which facilitates
a range of other studies to investigate the impact of large-scale projects on
the European system and electricity markets. An open grid data of the NEM
transmission network has been developed and released in [6] which includes
network and generation data sets, geospatial locations of network elements,
and is designed to interface with AEMO’s public database. This interface is
specifically advantageous since it allows historic regional load and generator
dispatch data to be integrated with the network model. The authors ensured
completeness and functionality of the dataset by assessing power flow models.

Recently a synthetic network model based on the NEM has been released [7, 8,
9]. This model, called S-NEM2300-bus benchmark model, has longitudinal
structure and is useful for EMT real-time digital simulations. The authors
developed this model based on a PSS/E model of the NEM and used statistical
methods to obfuscate generation, transmission and load parameters. The
resulting model is a relational database released as CSV files and can be used
in real-time EMT platforms.

In this work we use this S-NEM2300 network data to develop a benchmark for
power system _optimisation_ studies. We develop a data model to represent ac-
feasible steady state power flow behaviour of the synthetic NEM model by
parsing, cleaning and enhancing the input relational data in csv format
(extract from Hypersim simulations) to PowerModels.jl [10] and MATPOWER [11]
data models. PowerModels.jl [10] is a Julia/JuMP package to study steady-state
power network optimization problems. It provides capabilities such as parsing
and modifying network data as well as a common platform for computational
evaluation of emerging power network formulations and algorithms. PowerModels’
network data model is organized as data dictionaries and the key names are
designed consistent with MATPOWER’s [11] file format which is a familiar data
model for power system researchers, with some exceptions on the having
separate components for loads and shunts, and design of time series simulation
as “multi-network”.

To ensure feasibility of the converted data we conduct a series of power flow
and optimal power flow studies as described in Section 3.3. The ac-feasible
converted data is then validated by comparing power flow results with the
steady state results in the original data release. The converted synthetic
network data is then complemented and extended by adding thermal limits and
generation fuel type and cost models to serve as a benchmark model for
optimisation studies.

###  1.1 Motivation

The S-NEM2300 network is developed as a benchmark model based on the NEM which
is compatible for real-time simulations. We aim to convert this data to a
useful data for optimal power flow studies. The original data comes with the
following limitations:

  * •

It is generated based on a snapshot of the actual NEM at a Summer day of 2018.
As such, it is likely that not all generators were scheduled at the time,
hence not represented in the synthetic model.

  * •

Generators are classified as thermal or hydro, but further details regarding
fuel type and cost data are not taken into account.

  * •

Line and transformer thermal ratings are not identified.

The main motivation to develop a benchmark model is studying the following
applications on the NEM-level network:

  * •

We aim to develop the data model to represent ac-feasible steady state power
flow behaviour of the S-NEM2300 model. That is, the AC formulations are
feasible for the developed network data, which ensures feasibility of other
formulations such as current-voltage (IV), Quadratic Convex (QC), and ‘DC’.

  * •

Impact of DER integration on the grid can be studied in a large scale network
with similar properties the the NEM.

  * •

Conducting optimal power flow studies such as unit commitment, economic
dispatch, and market studies.

  * •

Conducting security constrained optimal power flow studies.

  * •

Development of hybrid ac-dc network by adding the NEM HVDC lines in the
synthetic model and investigating security of the network against credible
contingencies.

##  2 Original Synthetic NEM (S-NEM2300) Network Data

We first revisit the main properties of the synthetic NEM data released in
[8]. Table 1 and Figure 2 illustrate the main network components and the areas
of the network based on synthetic coordinates available in the dataset. More
properties of the network such as area interconnectors, graph properties,
structural properties and area diameters are provided in [8].

Table 1: Original synthetic NEM network number of components and areas [8].

Component | Count
---|---
Buses | 2340
Lines | 1821
Two-winding transformers | 1418
Three-winding transformers | 113
Generators | 265
Shunts | 301
Loads | 1696

Area | State | Bus ID range
---|---|---
1000 | New South Wales | 1000-1999
2000 | Victoria | 2000-2999
3000 | Queensland | 3000-3999
4000 | South Australia | 4000-4999
5000 | Tasmania | 5000-5999

Figure 2: One-line graph diagram of the synthetic NEM data [8].

##  3 Data Model Development Framework

This section describes data model cleaning and derivation process and the
methodology to develop an ac-feasible network data. The input data is in CSV
format which we first parse into PowerModels data model that can then readily
be serialized into MATPOWER data format555command export$\\_$matpower(file,
data).

###  3.1 Data Models

####  3.1.1 Hypersim

Opal-RT Hypersim666https://www.opal-rt.com/systems-hypersim/ is a powerful
real-time simulation platform that can be used for power system simulations.
It is a software solution that allows engineers to create a virtual
environment that mimics the physical power system, enabling accurate modeling
and testing of complex power systems.

####  3.1.2 MATPOWER

The MATPOWER data model format is a standardized data format used by the
MATPOWER software package [11] for representing power system data in a MATLAB
file. It provides an established, flexible format that researchers know, which
makes it easier to analyse, compare, and share data in the power system
research community.

####  3.1.3 PowerModels.jl

The PowerModels.jl package internally uses a dictionary to store network data
with strings as keys so that it can be serialized to JSON for algorithmic
exchange [10]. A detailed description of the PowerModels data model is
provided in the documentation777https://lanl-
ansi.github.io/PowerModels.jl/stable/. PowerModels’ data model can be
instantiated by reading MATPOWER case studies, and PowerModels can serialize
data to both MATPOWER and JSON.

###  3.2 Data Model Derivation

We parse and clean the input relational data into the PowerModels data model
with bus, branch, load, gen, switch, storage, shunt and dcline as dictionary
keys. Each network element is indexed and the data is stored in per unit
values. Transformers are stored as branches with an identifier flag.

The data cleaning process consists of the following steps:

  * •

Identifying how to accurately convert each element data from the relational
dataset to the PowerModels format. For most network elements such as buses,
lines, shunts, load and generators, the cleaning is fairly straightforward.
For two-winding and three-winding transformers, however, the cleaning requires
implementing an equivalent circuit, which explained later in this section.
Note that the input data does not contain any dc line, so in this paper we
follow the same approach, and develop a model with dc line and buses in a
future work.

  * •

Regional areas and interconnector branches are identified and flagged.

  * •

Per unit conversion and unit transformations: We convert all data to per unit
w.r.t their individual base values as given in the original data set (per unit
conversion) and then update the per unit values to be on the base value of the
cleaned data model (unit transformation).

####  3.2.1 Two-Winding Transformer

In the input data, two-winding transformer models include primary, secondary,
and magnitization impedance values. This model represents a T-section circuit
as shown in Figure 3 on the left. The branch model in PowerModels data format,
however, is in $\Pi$-section Figure 3 on the right. We therefore perform a T
to $\Pi$ circuit conversion using equations in (1) and calculate the
transformer tap ratio from primary and secondary per unit voltage values.

$V_{1}$$Z_{1}$$Z_{2}$$V_{2}$$Z_{3}$

$V_{1}$$Z_{B}$$V_{2}$$Z_{A}$$Z_{C}$

Figure 3: Two winding transformer equivalent circuits: T model (left) and
$\Pi$ model (right).

$\displaystyle Z_{A}=\frac{Z_{1}Z_{2}+Z_{2}Z_{3}+Z_{1}Z_{3}}{Z_{2}},\quad
Z_{B}=\frac{Z_{1}Z_{2}+Z_{2}Z_{3}+Z_{1}Z_{3}}{Z_{3}},\quad
Z_{C}=\frac{Z_{1}Z_{2}+Z_{2}Z_{3}+Z_{1}Z_{3}}{Z_{1}}$ (1)

####  3.2.2 Three-Winding Transformer

An equivalent circuit of a three-winding transformer is shown in Figure 4.
This circuit is implemented in PowerModels format as three branches
representing three two-winding transformers that are connected to a common
auxiliary node in between. The primary, secondary and tertiary impedance
values are available from input data and the magnetization impedance is
considered to be at the primary side, i.e. the primary two winding
transformer. To accurately capture the actual three winding transformer tap
ratios, each two-winding transformer tap ratio is calculated from its per unit
voltage divided by the auxiliary node per unit voltage which is assumed to be
1.

$V_{1}$$R_{1}$$L_{1}$$R_{2}$$L_{2}$$V_{2}$$R_{3}$$L_{3}$$V_{3}$$R_{0}$$L_{0}$
Figure 4: Three winding transformer equivalent circuit.

###  3.3 Methodology

While the original network data is implemented and tested in Hypersim,
cleaning and derivation of a different data model is still challenging as the
cleaned data may cause infeasibility in optimal power flow (OPF) studies, as
OPF needs to satisfy bounds such as voltage and current limits, and avoid
voltage collapse. A technique to analyse power flow infeasibility is to extend
traditional power-balance constrained OPF to include nodal slack variables
that indicate constraint violations when there is no feasible solution. The
infeasibility analysis technique is such that if the network is feasible, the
slack variables have values of zero at convergence. Otherwise, nonzero slack
variables can be used to identify and localize infeasibility for planning
insight or corrections during operation. This method was first introduced in
[12] for transmission systems and then extended in other work such as [13,
14]. Lately this technique has been applied to distribution systems in [15].

In this work we apply the infeasibility analysis method used in [14] in an
algorithmic way to identify different local constraint violations and the
physical meaning behind it. Figure 5 shows the flowchart of the algorithm
implemented to identify infeasibility issues in the derivation process.
Starting from input files, the cleaned data is first tested by running a
linearized ‘DC’ OPF with slack nodal power variables to identify whether
active power balance at all nodes are maintained. Nodes with significant
imbalance are investigated as to what active elements could be the problem.
The next step is to identify reactive power balance at the nodes which is the
outcome of an AC-OPF with nodal slack power variables. Similar to the previous
step, the nodes with substantial imbalance are investigated and the derivation
process is modified until active and reactive power balance at all nodes are
maintained. If the slack AC-OPF renders perfectly balanced network, the
derivation could conclude at this point, but in case of existing imbalance in
this step, the next two steps prove effective. The next step is to test
generation dispatch setpoints, which is achieved by comparing AC and
linearized ‘DC’ PF solutions of the network with slack nodal power variables.
If the generator setpoints mismatch, slack power variables are introduced to
generators to identity which generators cause the most mismatch and the
derivation process is modified accordingly.

Figure 5: Model conversion algorithm consisting of a series of relaxed power
flow and optimal power flow evaluations.

###  3.4 Validation of the Derived Synthetic NEM (S-NEM2000) Network Data

Figure 6 demonstrates how the power flow results of the derived network data
compares to the steady state results of the original network. As can be seen,
the derived model accurately tracks the original model in voltage magnitude
and phase angle, and generators’ active power setpoints. The slight mismatch
in generators’ reactive power setpoints is mainly due to approximations made
in generator models which ignore internal impedance values.

Figure 6: Validation of the derived model power flow results.

##  4 Thermal Limits

The S-NEM2300 network data was developed to serve for phasor and EMT
simulation studies. To enhance the network data to be useful for
_optimisation_ studies we need to add reasonable network constraints such as
line and transformer thermal limits.

We adopt the data driven approach described in [3] built on publicly available
datasets. As noted in [3], these datasets reflect many of the statistical
features that are common on real networks. Section 4.1 provides an overview of
the reference datasets and the derivation and validation of the thermal limit
models.

To apply the thermal limit models to the synthetic NEM model, however, we
first review the properties of the S-NEM2300 data. The network data is then
modified to reasonably resemble the transmission systems by reducing network
through joining short lines, and removing ideal lines and circuit breakers.
The data driven thermal limit models are then applied to the modified
synthetic NEM network data which we call S-NEM2000. We next run an algorithm
that links and updates thermal limits of the connecting lines and transformers
so that it more closely represents a real power system set of constraints. The
properties of the modified network and linked thermal limits are validated and
discussed.

###  4.1 A Thermal Limit Model

In practice, line thermal limits are straightforward to calculate if conductor
type and line length are available. In this synthetic dataset, however, we
only have line impedance values and the connecting bus nominal voltages. With
this limited information we use a linear regression model proposed in [3] to
determine line thermal limits based on the values of line reactance $X$, line
resistance $R$ and nominal voltage $v$. For the elements that any of these
values are not available or not applicable (in case of ideal lines and
transformers), a reasonable upper bound is derived.

####  4.1.1 Reviewing the Reference Data

We use two sources of transmission thermal limits for this benchmark study:
the Polish transmission system data [11], and the Irish transmission
transmission system data [16]. Although some transformer data is also
available in these datasets, we focus the data driven study only on the
available transmission line data. In order to stay within reasonable range of
thermal limits and line types, we only consider lines with $X/R$ ratio below
$20$ and normalised thermal rating (thermal rating divided by nominal voltage,
that is, current rating) above $0.005$. Figure 7 shows the correlation between
line thermal limits and the nominal voltage. As stated in [3], the significant
variance at each voltage level which even overlaps with the neighbouring
voltage levels replicates a realistic behaviour in real networks where a given
nominal voltage may contain low and high capacity lines.

Figure 7: Polish network (left) and EIRGrid network (right) thermal limits and
voltage levels correlation.

####  4.1.2 Statistical Model

The statistical model proposed in [3] derives thermal limits in power flow
(MVA) rather than current flow (kA), which means that identical conductors can
have different MVA limits based on their nominal voltage levels, but
converting back to kA they should have similar thermal limits. The model also
utilizes the ratio of line reactance to resistance to de-correlate the line
conductor type with line length. Figure 8 indicates a clear correlation
between $X/R$ ratio and the kA thermal limits.

Figure 8: Polish network (left) and EIRGrid network (right) normalised thermal
limits and X/R ratio correlation.

We next combine the two datasets and fit the following linear regression model
on the log-log scale

$y=e^{a}x^{k}$ (2)

for which we derive the following values for $a$ and $k$,

$a=-5.1407,\quad k=0.6078.$ (3)

The regression fit model and its parameters are shown in Figure 9 which is a
fairly crude but effective fit of the data for our purposes. In [3] the values
for $a$ and $k$ are derived as $-5.0886$ and $0.4772$, respectively. Therefore
the value for $a$ we derived is close to the one in [3], but the value of $k$
is slightly different, which could be due to having a more recent Irish
dataset, inclusion of more points from the reference datasets and/or the
filters applied to the data.

Figure 9: Linear log-log model for combined Polish and EIRGrid networks.

The model (2) is used to estimate per unit thermal limits $S^{u}$ for the
lines that have all three parameters $X,R,v$ on both sides of the line as,

$S^{u}=ve^{-5.1407}\left(\frac{X}{R}\right)^{0.6078}.$ (4)

####  4.1.3 Upper Bound

The statistical model (4) is only applicable when all three $X,R,v$ are
available and both sides of the element have the same nominal voltage. It is
therefore not possible to apply this model to ideal lines with $R=0$ and
transformers with different nominal voltage at either side. In these cases,
the authors in [3] propose calculating a reasonable theoretical upper bound
for the thermal limit that is close to the theoretical maximum, but not to the
extent that it deactivates the thermal limit constraints. The upper bound
accurately indicates the line’s true throughput limitation and is calculated
from [3]

$(S^{u}_{ij})^{2}=(v_{i}^{u})^{2}Y_{ij}^{2}\big{(}(v_{i}^{u})^{2}+(v_{j}^{u})^{2}-2v_{i}^{u}v_{j}^{u}cos(\theta_{ij}^{\Delta})\big{)}$
(5)

where the index $ij$ means the line/transformer between buses $i$ and $j$,
$v_{i}^{u}$ is the voltage magnitude upper bound at bus $i$, $Y_{ij}$ is the
line/transformer’s admittance, and $\theta_{ij}^{\Delta}$ is
line/transformer’s phase angle difference bound. The thermal limit upper bound
can be calculated given that reasonable voltage bounds are available
throughout the network and with the assumption of
$\theta_{ij}^{\Delta}=15^{\circ}$.

####  4.1.4 Validating the Statistical and Upper Bound Models

Figure 10 shows correlation of the proposed statistical and upper bound
thermal limit values to the real thermal limits given in the reference
datasets of Polish and EIRGrid transmission systems. The figure on the
statistical model shows a reasonable correlation of the model and the actual
data as the data forms a stretch of points along the $x=y$ line which
indicates perfect predictions. The upper bound, on the other hand, shows
points below the $x=y$ line affirming the optimistic high bound on the thermal
capacity by design, while the large upper bound limits indicate lines with
small reactance values which represent ideal conductors.

Figure 10: Comparison of the statistical model (left) and the thermal limit
upper bound calculation (right) to the given MVA limits for Polish and EIRGrid
networks.

###  4.2 Reviewing the S-NEM2300 Data

Figure 11 shows the correlation between normalised thermal limits and the
$X/R$ ratios in S-NEM2300 network. Note that all thermal limits are set to the
nominal MVA of 100 MVA for all lines and transformers and only the $X/R$
values of non-ideal lines are shown. From the figure it is evident the network
data requires some modifications to show a correlation behaviour similar to
reference datasets in Figure 8.

Figure 11: S-NEM2300 network normalised thermal limits and X/R ratio. Note
that X/R ratio renders numerous infinity values corresponding to ideal lines
which are not represented in this figure.

We perform two basic network reduction techniques to modify and prepare the
network data for the thermal limit models application:

  * •

we remove the lines that represent ideal lines or circuit breakers, filtered
as lines with

    * –

small impedance ($<0.01$ p.u.) and small shunt admittance ($<0.01$ p.u.),
since in the synthetic data there exist constant parameter (CP) and decoupling
lines with negligible impedance but non-negligible shunt admittance values,
or,

    * –

high $X/R$ ratio ($>100$) and small shunt admittance ($<0.01$ p.u.);

  * •

we join the cascading lines connected through degree 2 buses to ensure a
uniform thermal limit over linear paths in the network.

Figure 12 shows how the reduced network bus voltage magnitudes and phase
angles results still closely match the original network results. In fact, we
used this power flow validation to specify and visualise a small variation of
voltage magnitude and angle w.r.t the S-NEM2300 results in order to determine
the empirical thresholds for impedance, shunt admittance, and $X/R$ ratio
values of the lines to be removed.

Figure 12: Validation of S-NEM2000 after network reduction.

###  4.3 Adding Thermal Limits and Validations

In the next step we add line and transformer thermal limit using the
statistical and upper bound thermal limit models where applicable. We also set
lower and upper bounds of 30 and 1500 MVA for the thermal limits on
transformers according to NEM equipment ratings [17]. Figure 13 shows line
thermal limits across different voltage levels and correlation of normalised
thermal limits and $X/R$ ratio. The left figure shows that the lines with
higher nominal voltage have higher thermal limits which is a true behaviour of
real systems. The perfect prediction of line thermal limits in the right
figure indicates that no ideal lines exists in the data any more. Note that
the transformer thermal limits are derived from the upper bound model and are
not represented in Figure 13.

Figure 13: S-NEM2000 thermal limit vs voltage levels (left) and normalised
thermal limits and X/R ratio (right).

The estimated thermal limits, though seem to have been predicted reasonably
well as seen in Figure 13, should be tested against power flow results of the
modified network data. Figure 14 on the left shows line and transformer
loading w.r.t their thermal limits. We note that with these thermal limits a
total of 9 line flow violations occur in steady state conditions. Since the
number and magnitude of violations are manageable, in the next step we
increase the thermal limits of the violating lines appropriately to ensure
that the network is secure under normal operation. Figure 14 on the right
shows line and transformer loading w.r.t their thermal ratings while Figure 15
shows how the thermal limits have changed with respect to Figure 13.

Figure 14: S-NEM2000 line and transformer loading: before modification (left),
after modification(right).

Figure 15: S-NEM2000 modified thermal limit vs voltage levels (left) and
normalised thermal limits and X/R ratio (right).

###  4.4 S-NEM2000 Network Model Components

After cleaning the S-NEM2300 network data, reducing the lines, and fixing some
minor issues with generators, the network data components count is listed in
Table 2, which compared to the ones reported in Table 1, the main differences
are the number of buses and lines due to removing degree-2 buses and joining
the connecting lines. Note that the three-winding transformers are modelled as
three two-winding transformers in the released version of the S-NEM2000
network data.

Table 2: S-NEM2000 network number of components.

Component | Count
---|---
Buses | 2000
Lines | 1324
Two-winding transformers | 1418
Three-winding transformers | 113
Generators | 265
Shunts | 301
Loads * | 1702

* Some generators have negative injection.
They are divided into a generator
with zero injection, and a load.

##  5 Generation Models

The canonical objective in OPF is generation cost minimization. Such an
application, and related ones such as market clearing and unit commitment,
depend on detailed generator specifications such as fuel type, active and
reactive power capability curves, ramp up/down rates, minimum on/off times,
startup/shutdown costs, and fuel efficiency. To assign the appropriate
properties to generators in the synthetic model, we first notice that most
generator properties are directly linked to its mechanical design, which in
turn is determined by the fuel type. We therefore develop data driven models
to assign generator fuel categories, and then, using publicly available data
sets we derive generator properties.

To assign generator fuel categories and properties we use publicly available
generation datasets provided by AEMO [18, 19] and GeoScience Australia (GA)
[20]. We categorize the generators fuel type so that the synthetic network
model is a reasonable representative of the actual NEM generation fuel mix.

###  5.1 Reviewing the Reference Data

The reference data we use to assign synthetic generator types is GA major
power stations list [20] which contains generators’ fuel type, registered
capacity, voltage level, status, and more. We only consider operational
generators and the ones with a single fuel source. The reference dataset [20]
classifies generation fuel categories into black/brown coal, gas, distillate,
biomass, biogas, water, solar, wind, battery, fuel oil and coal seam methane.

We first ignore generators using coal seam methane and fuel oil as there are
only a few of them with very limited capacity. Next, inspired by Figure 16
that shows generation fuel mix of the NEM for the past year [21], we ignore
distillate, biomass, biogas and battery power stations as well. Figures 17 and
18 show the count and capacity of generators throughout the NEM categorized by
fuel type and state. For the S-NEM2000 model we target generation fuel type
allocation to represent the generation fuel mix as shown in these two figures.

Figure 16: NEM generation fuel mix for the period of May 2022 to May 2023
[21].  Figure 17: Generators count categorized by state and fuel type in the
NEM. Figure 18: Generation capacity categorized by state and fuel type in the
NEM.

Upon determining generation fuel categories, we assign generator properties
and fuel cost data from Fuel and Technology Cost Review [19] which contains
parameters such as start up and shut down costs, no-load, fixed and variable
operation costs, minimum up/down times, ramp up/down rates, and more
operational properties.

###  5.2 Reviewing the S-NEM2000 Data

The S-NEM2300 model classifies generators into synchronous generators and
network sources, and does not take into account any fuel category and cost
data. However, the authors in [8] state that synchronous generations include
fossil-fuel generators as well as hyrdo and wind powered generators whereas
the network source category consists of ‘other’ types of generators. We base
our classification on these assumptions and choose the network sources to only
include solar farms. It is also worth noting that the reactive power
compensators such as synchronous var generators and synchronous condensers can
be part of the network sources as well, particularly by noting that some of
the networks sources only provide reactive power and no active power to the
network.

##### Remark:

The S-NEM2300 network data models the Basslink HVDC interconnector between the
mainland NEM and Tasmania as two network sources connected to each sub-
network. However, since both network sources are indexed and placed as
Tasmania elements, we assign the generator (and the generator bus),
representing Tasmania in the mainland, to the Victoria area.

Figure 19 shows how the total count and capacity of generators across
different states are compared between the NEM and the S-NEM2300.

Figure 19: Comparison of generators in the NEM and Synthetic NEM per state
with respect to: total generators count (left) and total generation capacity
(right).

###  5.3 Generation Fuel Category Classification

In effort to classify synthetic generators by fuel category, a data-driven
classification approach was taken. Within this approach the registered
capacity of generators from GA major power stations list [20] were used to
train an Sklearn Random Forest classification model [22, 23] to predict the
fuel type of a given generation. Given that in the reference dataset some fuel
types are overrepresented compared to others, we used oversampling of the
under-represented classes by generating samples based on nearest neighbours
using Synthetic Minority Over-sampling Technique (SMOTE) [24]. Using SMOTE to
balance the classes before classifying, the trained classification model was
then used with the maximum generation capacity of synthetic generators within
the S-NEM2000 dataset to predict the fuel type of the generators in each state
separately.

To stay true to the S-NEM2300 model, the classifier uses the synthetic
synchronous generators list to identify generators with black/brown coal,
natural gas, hydro and wind fuel categories and uses the network source list
to assign solar generation units888 An important point that is missing from
many datasets is the identification of synchronous condensers and other var
compensator units. The S-NEM2300 model does not mention var compensators as
part of the generator categories, but close observation of the data reveals
that the network sources contains several generation units with zero or very
small active and substantial reactive power contribution to the network. One
could assume these units as var compensator units and assign the reminder as
solar generation units. . Nonetheless, we pre-allocate the the network source
representing Basslink in NEM mainland as a hydro generator, due to hydro being
the main fuel category in Tasmania, and similarly the network source
representing Basskink in Tasmania as a natural gas fueled generator, due to
natural gas being the only fossil-fuel present in Tasmania fuel
categories999These two network sources can be removed in future work when HVDC
lines are added to the network..

Figure 20 shows comparison of the generator capacities categorized by fuel
type between the NEM and the S-NEM2000. Quantiles comparison shows that the
classification has rendered a reasonable traction of generator capacities for
each fuel type.

To further assess the performance of classification, the predicted generation
type was used to aggregate the generation capacity and the count of the
synthetic generators in each state to compare to the aggregate generation
capacity and count of generators within the NEM [20]. These aggregate
comparisons can be seen in Figures 21 and 22. These figures show how the total
generation capacity and count of each state compare between the classified
synthetic model and the reference data. The aggregate generation capacity and
count categories in both network models follow a similar pattern while the
synthetic network has a lower total capacity level. The shortage generation is
because the reference data is an updated list of all existing generators in
the NEM, while the S-NEM2300 model was developed from a PSS/E model of the
network representing a 2018 Summer day. The synthetic model was developed to
replicate that specific moment in time, which caused the exclusion of several
generation units that were not in operation at the time. It is therefore
realistic to see a lower generation capacity level in the synthetic network
model. More observations from the Figures 21 and 22 are as follows:

  * •

The total generation capacity and generators count for the whole NEM,
classified by fuel type, are often lower in the S-NEM2000, except for solar
which is due to consideration of all ‘other’ generation units such as var
compensators and battery storage.

  * •

Tasmania generation capacity and count are represented quite well.

  * •

Queensland is represented reasonably well in terms of generator count and
capacity.

  * •

Generation capacity and count in New South Wales, Victoria and South Australia
have been represented fairly reasonable for most fuel types and less accurate
for some others.

Overall, the classification demonstrates a reasonable representation of fuel
types in each state and across the whole NEM. One could also further modify
generator capacities to match the actual aggregated value to be a closer
representation of the actual system.

Figure 20: Comparing NEM and S-NEM2000 w.r.t generator count by fuel type.
Figure 21: Comparing NEM and S-NEM2000 w.r.t generator count by fuel type and
state. Figure 22: Comparing NEM and S-NEM2000 w.r.t generator capacity by fuel
type and state.

###  5.4 Generation Cost and Operation Data

After generator fuel type classification we use AEMO’s Fuel and Technology
Cost Review [19] to assign cost and operational models for different
generators. Table 3 reports the main parameters identified from the reference
file and assigned to the fuel categories.

Table 3: S-NEM2000 generator cost and operational data. | Black coal | Brown coal | Natural Gas | Water | Wind | Solar
---|---|---|---|---|---|---
Min Gen ($\%$ of nameplate capacity) | 40 | 60 | 0 | 0 | 0 | 10
Minimum On/Off Time (Hours) | 8/8 | 16/16 | 1/1 | 1/1 | 1/1 | 1/1
No Load Fuel Consumption |  |  |  |  |  |
($\%$ of Full Load Fuel Consumption) | 10 | 10 | 30 | 0 | 0 | 0
Auxiliary load |  |  |  |  |  |
($\%$ of nameplate capacity) | 6 | 10 | 3 | 1 | 1 | 8
Ramp Up Rate (MW/h) | 120 | 130 | 600 | 7630 | 900 | 10000
Ramp Down Rate (MW/h) | 150 | 140 | 600 | 1840 | 600 | 10000
Thermal Efficiency ($\%$, HHV sent-out) | 35.4 | 23.5 | 32 | 100 | 100 | 100
Maintenance (days/yr) | 20 | 20 | 5 | 7 | 3 | 10
Fixed Op Cost ($\$$/MW/year) | 53400 | 147200 | 14200 | 56700 | 43000 | 64000
Variable Op Cost ($\$$/MWh sent-out) | 1 | 1 | 8.3 | 7 | 10 | 15.2
No Load Costs ($\$$/MW, 2014-15) | 1500 | 1500 | 500 | 500 | 2000 | 500
No Load Recurring Cost ($\$$/MW/year) | 500 | 500 | 200 | 200 | 1000 | 200
Cold/Warm/Hot Start-up Costs ($\$$/MW) | 350 | 350 | 100 | 5 | 5 | 5
Warm Start-up Costs ($\$$/MW) | 120 | 120 | 100 | 3 | 3 | 3
Hot Start-up Costs ($\$$/MW) | 40 | 40 | 100 | 2 | 2 | 2

##  6 Proof of Concept Study

In addition to the integrated NEM data (snem2000.m), we divide it into two
sub-networks 1) mainland (snem1803.m) and 2) Tasmania (snem197.m). In this
section we run optimal power flow on S-NEM2000 and its mainland and Tasmanis
sub-networks for AC power flow relaxations and compare objective values and
runtime, which is similar to how the results in [4] are presented. We use
PowerModels.jl version 0.19.8 and the interior point solver Ipopt [25] with
MUMPS linear solver to solve the optimal power flow problems. The computation
is performed on a PC with 6-Core Intel(R) Core(TM) i7 @ 2.60GHz processor with
32 GB of RAM.

Table 4: Quality and runtime results of AC power flow formulations. Network | Formulation | Objective value ($\$$) | Gap ($\%$) | Runtime (s)
---|---|---|---|---
Whole NEM | NLP | 98322.1 | - | 19.75
(snem2000.m) | SOC | 90424.6 | 8.03 | 18.83
| DC | 87396.6 | 11.11 | 1.55
Mainland | NLP | 98320.6 | - | 17.44
(snem1803.m) | SOC | 90423.1 | 8.03 | 16.54
| DC | 87395.0 | 11.11 | 2.54
Tasmania | NLP | 1.50165 | - | 0.64
(snem197.m) | SOC | 1.50067 | 0.06 | 0.72
| DC | 1.47406 | 1.84 | 0.06

##  7 Conclusion and Future Work

In this study, we utilized an existing synthetic representation of the NEM and
developed the S-NEM2000 benchmark for power system optimization studies. By
converting the synthetic network data into the PowerModels.jl and MATPOWER
data models, we enabled feasibility and validation checks through power flow
and optimal power flow studies. The resulting benchmark model includes
enhanced features such as thermal limits and generation fuel type and cost
models, providing a valuable resource for optimization studies in the power
system domain. We release the whole of NEM data (snem2000.m) under the
creative commons ‘CC BY 4.0’101010https://creativecommons.org/licenses/by/4.0/
license at https://github.com/csiro-energy-systems/Synthetic-NEM-2000bus-Data.
The authors also intend to publish this data set in Power Grid Library Optimal
Power Flow benchmarks.

However, it is important to note that the synthetic model used in this study
is not an exact replica of the actual NEM, and there is still room for
improvement. Future work should focus on incorporating additional components,
such as High-Voltage Direct Current (HVDC) lines, which play a significant
role in interconnecting regions within the NEM. The inclusion of HVDC lines
would enable more accurate simulations and analyses of power flow and
optimization scenarios. Furthermore, expanding the benchmark model to include
detailed load and generation profiles is another important avenue for future
work. Incorporating realistic and time-varying data on load and generation
patterns would provide a more comprehensive representation of the NEM and
enable more accurate assessments of system performance and optimization
strategies. Additionally, for the data-driven generation classification
approach, using additional generator properties, for example voltage, could
improve the classification results.

The ongoing development and growth of this work will involve continuous
refinement and expansion of the S-NEM2000 model, incorporating additional
components and data sources to enhance its representation of the real-world
system. These advancements will contribute to furthering our understanding of
the NEM and supporting effective decision-making in the context of power
system optimization and planning.

## Acknowledgement

We would like to express our gratitude to Dr. Felipe Arraño-Vargas and Dr.
Georgios Konstantinou who released the original data used in this study, and
for their ongoing support and advice on gaining insights from the data. Their
invaluable contribution and commitment to open data have facilitated our
research and enabled us to develop the S-NEM2000 benchmark for power system
optimization studies.

We extend our appreciation to AEMO and GeoScience Australia for providing the
publicly available generation datasets that formed the basis of our analysis.
Their efforts in collecting and maintaining comprehensive data on the NEM have
been instrumental in enhancing our understanding of the power system domain
and played a crucial role in assigning appropriate properties to the synthetic
generators in our model and ensuring its fidelity to the NEM.

## References

  * [1] Australian Energy Market Commission (AEMC). National Electricity Market (NEM). Accessed: 2023-05-23. [Online]. Available: https://www.aemc.gov.au/energy-system/electricity/electricity-system/NEM
  * [2] F. Arraño-Vargas, Z. Shen, S. Jiang, J. Fletcher, and G. Konstantinou, “Challenges and mitigation measures in power systems with high share of renewables - the australian experience,” _Energies_ , vol. 15, no. 2, 2022\. [Online]. Available: https://www.mdpi.com/1996-1073/15/2/429
  * [3] C. Coffrin, D. Gordon, and P. Scott, “NESTA, the NICTA energy system test case archive,” pp. 1–26, 2014. [Online]. Available: http://arxiv.org/abs/1411.0359
  * [4] S. Babaeinejadsarookolaee, A. Birchfield, R. D. Christie, C. Coffrin, C. DeMarco, R. Diao, M. Ferris, S. Fliscounakis, S. Greene, R. Huang, C. Josz, R. Korab, B. Lesieutre, J. Maeght, T. W. K. Mak, D. K. Molzahn, T. J. Overbye, P. Panciatici, B. Park, J. Snodgrass, A. Tbaileh, P. V. Hentenryck, and R. Zimmerman, “The power grid library for benchmarking ac optimal power flow algorithms,” 2021.
  * [5] A. Tosatto, X. M. Beseler, J. Østergaard, P. Pinson, and S. Chatzivasileiadis, “North sea energy islands: Impact on national markets and grids,” _Energy Policy_ , vol. 167, p. 112907, 2022.
  * [6] A. Xenophon and D. Hill, “Open grid model of Australia’s national electricity market allowing backtesting against historic data,” _Scientific Data_ , vol. 5, pp. 1–21, 2018.
  * [7] F. Arraño-Vargas and G. Konstantinou, “Modular design and real-time simulators toward power system digital twins implementation,” _IEEE Trans. Indust. Informatics_ , vol. 19, no. 1, pp. 52–61, 2023.
  * [8] F. Arraño-Vargas, “Power System Digital Twins and Real-Time Simulations in Modern Grids,” Ph.D. dissertation, University of New South Wales, 2022.
  * [9] F. Arraño-Vargas and G. Konstantinou, “Synthetic Grid Modeling for Real-Time Simulations,” _IEEE ISGT Asia_ , pp. 1–5, 2021.
  * [10] C. Coffrin, R. Bent, K. Sundar, Y. Ng, and M. Lubin, “PowerModels.jl: An open-source framework for exploring power flow formulations,” in _Power Syst. Comp. Conf._ , June 2018, pp. 1–8.
  * [11] R. D. Zimmerman, C. E. Murillo-Sánchez, and R. J. Thomas, “MATPOWER: Steady-state operations, planning, and analysis tools for power systems research and education,” _IEEE Trans. Power Syst._ , vol. 26, no. 1, pp. 12–19, 2011.
  * [12] T. Overbye, “A power flow measure for unsolvable cases,” _IEEE Trans. Power Syst._ , vol. 9, no. 3, pp. 1359–1365, 1994.
  * [13] M. Jereminov, D. M. Bromberg, A. Pandey, M. R. Wagner, and L. Pileggi, “Evaluating feasibility within power flow,” _IEEE Trans. Smart Grid_ , vol. 11, no. 4, pp. 3522–3534, 2020.
  * [14] S. Li, A. Pandey, A. Agarwal, M. Jereminov, and L. Pileggi, “A lasso-inspired approach for localizing power system infeasibility,” 2019.
  * [15] E. Foster, T. McNamara, A. Pandey, and L. Pileggi, “Actionable three-phase infeasibility optimization with varying slack sources,” 2023.
  * [16] EIRGrid. All-island ten-year transmission forecast statement. Accessed: 2023-05-23. [Online]. Available: https://www.eirgridgroup.com/site-files/library/EirGrid/All-Island-Ten-Year-Transmission-Forecast-Statement-2020.pdf
  * [17] Australian Energy Market Operator. Transmission equipment ratings. Accessed: 2023-05-23. [Online]. Available: https://aemo.com.au/en/energy-systems/electricity/national-electricity-market-nem/data-nem/network-data/transmission-equipment-ratings
  * [18] ——. Register for the national electricity market (nem). Accessed: 2023-05-23. [Online]. Available: https://aemo.com.au/en/energy-systems/electricity/national-electricity-market-nem/participate-in-the-market/registration
  * [19] ——. Fuel and technology cost review – data (acil allen). Accessed: 2023-05-23. [Online]. Available: https://aemo.com.au/energy-systems/major-publications/integrated-system-plan-isp/national-transmission-network-development-plan-ntndp/ntndp-database
  * [20] Geoscience Australia. Foundation electricity infrastructure WFS. Accessed: 2023-05-23. [Online]. Available: https://pid.geoscience.gov.au/dataset/ga/145105
  * [21] D. McConnell, S. Holmes à Court, S. Tan, and N. Cubrilovic, “An open platform for national electricity market data,” 2022, this work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit https://creativecommons.org/licenses/by/4.0/. [Online]. Available: https://opennem.org.au/energy/nem/?range=1y&interval=1w
  * [22] L. Buitinck, G. Louppe, M. Blondel, F. Pedregosa, A. Mueller, O. Grisel, V. Niculae, P. Prettenhofer, A. Gramfort, J. Grobler, R. Layton, J. VanderPlas, A. Joly, B. Holt, and G. Varoquaux, “API design for machine learning software: experiences from the scikit-learn project,” in _ECML PKDD Workshop: Languages for Data Mining and Machine Learning_ , 2013, pp. 108–122.
  * [23] L. Breiman, “Random forests,” _Machine learning_ , vol. 45, no. 1, pp. 5–32, 2001.
  * [24] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic minority over-sampling technique,” _J. Artificial Intelligence Res._ , vol. 16, pp. 321–357, June 2002. [Online]. Available: https://doi.org/10.1613%2Fjair.953
  * [25] A. Wächter and L. T. Biegler, “On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming,” _Math. Prog._ , vol. 106, no. 1, pp. 25–57, 2006. [Online]. Available: https://doi.org/10.1007/s10107-004-0559-y