# OrientDream: Streamlining Text-to-3D Generation with
Explicit Orientation Control

Yuzhong Huang1,2,$*$ Zhong Li1 Zhang Chen1 Zhiyuan Ren 1
Guosheng Lin 3 Fred Morstatter 2 Yi Xu1

1 OPPO US Research Center 2 University of Southern California
3 Nanyang Technological University

###### Abstract

†† ∗ Work done while Yuzhong was an intern at OPPO US Research Center.

In the evolving landscape of text-to-3D technology, Dreamfusion [22] has
showcased its proficiency by utilizing Score Distillation Sampling (SDS) to
optimize implicit representations such as NeRF. This process is achieved
through the distillation of pretrained large-scale text-to-image diffusion
models. However, Dreamfusion encounters fidelity and efficiency constraints:
it faces the multi-head Janus issue and exhibits a relatively slow
optimization process. To circumvent these challenges, we introduce
OrientDream, a camera orientation conditioned framework designed for efficient
and multi-view consistent 3D generation from textual prompts. Our strategy
emphasizes the implementation of an explicit camera orientation conditioned
feature in the pre-training of a 2D text-to-image diffusion module. This
feature effectively utilizes data from MVImgNet, an extensive external multi-
view dataset, to refine and bolster its functionality. Subsequently, we
utilize the pre-conditioned 2D images as a basis for optimizing a randomly
initialized implicit representation (NeRF). This process is significantly
expedited by a decoupled back-propagation technique, allowing for multiple
updates of implicit parameters per optimization cycle. Our experiments reveal
that our method not only produces high-quality NeRF models with consistent
multi-view properties but also achieves an optimization speed significantly
greater than existing methods, as quantified by comparative metrics.

|
---|---
a chimpanzee dressed like Henry VIII king of England | a small tiger dressed with sash
|
a majestic giraffe with a long neck | a kangaroo wearing boxing gloves

Figure 1:  3D objects generated by OrientDream pipeline using text input.  Our
OrientDream pipeline successfully generates 3D objects with high-quality
textures, effectively free from the view inconsistencies commonly referred to
as the Janus problem. We also present visualizations of geometry and normal
maps alongside each result for further detail and clarity.

##  1 Introduction

The demand for 3D digital content creation has surged recently, driven by
advancements in user-end platforms ranging from industrial computing to
personal, mobile, and virtual metaverse environments, impacting sectors like
retail and education. Despite this growth, 3D content creation remains
challenging and often requires expert skills. Integrating natural language
into 3D content creation could democratize this process. Recent developments
in text-to-2D image creation techniques, especially in diffusion models [26],
have shown significant progress, supported by extensive online 2D image data.
However, text-to-3D generation faces challenges, notably the lack of
comprehensive 3D training data. Although quality 3D datasets exist [3, 6],
they are limited in scale and diversity. Current research, such as in Lin et
al. [14], focusing on specific 3D object categories like human faces, lacks
the versatility needed for broader artistic applications.

Recent advancements, particularly in DreamFusion [39], have showcased that
pre-trained 2D diffusion models, which utilize 2D image data, can effectively
serve as a 2D prior in the optimization of 3D Neural Radiance Fields (NeRF)
representations through Score Distillation Sampling (SDS). This approach
aligns NeRF-rendered images with photorealistic images derived from text
prompts, facilitating a novel method for text-to-3D generation that bypasses
the need for actual 3D content data. However, this technique faces notable
challenges. The lack of relative pose information in the 2D prior supervision
leads to the ”multi-head Janus issue,” as evident in Fig. 2, where the 3D
objects display multiple heads or duplicated body parts, detracting from the
content’s quality. Additionally, the process is hindered by the
synchronization required between 2D diffusion iteration and NeRF optimization,
resulting in a relatively slow optimization process, averaging several hours
per scene. While subsequent research, including Magic3D[14] and Wang et al.
[38], has improved aspects like quality and diversity, the Janus issue and the
efficiency of the optimization process remain largely under-explored areas.

In this paper, we introduce a novel text-to-3D method capable of generating
multi-view consistent 3D content within a reasonable time frame. Key to our
approach is the fine-tuning of a 2D diffusion model with explicit orientation
control to ensure multi-view aware consistency during Score Distillation
Sampling (SDS) optimization. This is achieved by leveraging MVImgNet [40], a
comprehensive multi-view real object dataset encompassing 6.5 million frames
and camera poses. We integrate encoded camera orientation alongside the text
prompt, enabling the orientation conditioned 2D diffusion model to provide
multi-view constrained 2D priors for supervising NeRF reconstruction.

To accelerate the optimization process, we decouple the back-propagation steps
between NeRF parameter updates and the diffusion process. This is implemented
using the DDIM solver [32], which directly resolves the diffusion step to
$X_{T-step}$ instead of the conventional $X_{T-1}$. This approach allows for
multiple implicit parameter updates per optimization cycle, enhancing
efficiency.

Our experiments demonstrate that our method effectively circumvents the multi-
head Janus issue while maintaining high quality and variety, outperforming
other state-of-the-art methods in terms of time efficiency. As shown in Fig.
1. In summary, our contributions are threefold:

Figure 2: Illustration of the Janus Problem: This figure showcases typical
Janus issue manifestations, where the bunny is depicted with three ears, the
pig with two noses, the eagle with a pair of heads, and the frog anomalously
having three back legs but only one front leg.

  * •

In our paper, we introduce a text-to-3D generation method that effectively
reduces the multi-head Janus issue and enhances 3D content generation
efficiency while maintaining quality and variety.

  * •

We have developed an advanced 2D diffusion model with explicit orientation
control, using MVImgNet data for integrating camera orientation with text
prompts. This ensures consistent multi-view accuracy and provides a precise 2D
prior for NeRF reconstruction.

  * •

Our novel optimization strategy significantly speeds up the NeRF
reconstruction process. By decoupling NeRF parameter updates from the
diffusion process and employing the DDIM solver, we achieve faster and more
efficient multiple parameter updates in each optimization cycle.

##  2 Related Works

###  2.1 NeRFs for Image-to-3D

Neural Radiance Fields (NeRFs), as proposed by Mildenhall et al. [19], have
revolutionized the representation of 3D scenes through radiance fields
parameterized by neural networks. In this framework, 3D coordinates are
denoted as $\mathbf{p}=[x,y,z]\in\mathcal{P}$, and the corresponding radiance
values are represented as $\mathbf{d}=[\sigma,r,g,b]\in\mathcal{D}$.

NeRFs are trained to replicate the process of rendering frames similar to
those produced from multi-view images, complete with camera information.
Traditional NeRFs employ a simple mapping of locations $\mathbf{p}$ to
radiances $\mathbf{d}$ via a function parameterized by a Multilayer Perceptron
(MLP). However, recent advancements in NeRF technology have seen the
introduction of spatial grids that store parameters, which are then queried
based on location. This development, as seen in works by Müller et al. [20],
Takikawa et al. [35], and Chan et al. [5], integrates spatial inductive biases
into the model.

We conceptualize this advancement as a _point-encoder_ function
$\mathbf{E}_{\theta}:\mathcal{P}\to\mathcal{E}$, with parameters $\theta$
encoding a location $\mathbf{p}$ prior to processing by the final MLP
$\mathbf{F}:\mathcal{E}\to\mathcal{D}$. This relationship is mathematically
formulated as:

$\mathbf{d}=\mathbf{F}\left(\mathbf{E}_{\theta}\left(\mathbf{p}\right)\right)$
(1)

###  2.2 Text-to-Image Generation

The widespread availability of captioned image datasets has facilitated the
creation of potent text-to-image generative models. Our model aligns with the
architecture of recent large-scale methods, as seen in Balaji et al.[4],
Rombach et al.[26], and Saharia et al.[27]. We focus on score-matching, a
process where input images are modified by adding noise, as outlined by Ho et
al.[11] and Song et al. [33], and then predicted by the Denoising Diffusion
Model (DDM).

A key feature of these models is their ability to be conditioned on text,
enabling the generation of corresponding images through classifier-free
guidance, a technique described by Ho et al. [10].

In our approach, we employ pre-trained T5-XXL [25] and CLIP [24] encoders to
produce text embeddings. These embeddings are then used by the DDM, which
conditions them via cross-attention with latent image features. Importantly,
we repurpose the text token embeddings—denoted as $\boldsymbol{c}$—for
modulating our Neural Radiance Field (NeRF), integrating textual information
directly into the 3D generation process.

###  2.3 Text-to-3D Generation

In the realm of Text-to-3D Generation, recent methodologies primarily rely on
per-prompt optimization for creating 3D scenes. Methods such as those proposed
in Dreamfusion [22] and Wang et al. [37] utilize text-to-image generative
models to train Neural Radiance Fields (NeRFs). This process involves
rendering a view, adding noise, and then using a Denoising Diffusion Model
(DDM), conditioned on a text prompt, to approximate the noise. The difference
between the estimated noise and actual noise is used to iteratively update the
parameters of the NeRF.

However, these 2D lifting methods, while impressive in text-to-3D tasks,
inherently suffer from 3D inconsistencies, leading to the well-known Janus
Problem, as depicted in Fig. 2. Various attempts have been made to mitigate
this issue. PerpNeg [2] endeavors to enhance view-dependent conditioning by
manipulating the text embedding space. Other strategies focus on stabilizing
the Score Distillation Sampling (SDS) process, such as adjusting noise
strength [13] and gradient clipping [12]. A popular alternative involves
leveraging 3D datasets, either by training an additional 2D guidance model on
3D data [23] or by training a diffusion model to produce multi-view sub-images
in a single 2D image [30, 16, 21]. However, due to the smaller size and lower
texture quality of 3D datasets, outputs from these methods tend to mirror the
average quality found in these datasets. [30], outputs from these methods lean
towards the average quality in the 3D dataset.

Figure 3:  Overview of camera orientation conditioned Diffusion Model: This
figure illustrates the core components of our innovative model within the
OrientDream pipeline. It highlights how we integrate encoded camera
orientations with text inputs, utilizing quaternion forms and sine encoding
for precise view angle differentiation. The model, fine-tuned on the real-
world MVImgDataset, demonstrates our approach to enhancing 3D consistency and
accuracy in NeRF generation, surpassing common limitations found in models
trained solely on synthetic 3D datasets.

##  3 Method

Our proposed method is structured into three critical sections, each essential
to our innovative approach in text-to-3D generation: First, Section 3.1 delves
into the camera orientation conditioned diffusion model. Here, we introduce a
diffusion model that adeptly integrates camera orientation, significantly
enhancing the precision and consistency of our 3D content generation from
various viewpoints. Second, Section 3.2 covers the Text-to-3D Generation
process. In this part, we detail how our method effectively translates text
descriptions into detailed 3D models. We focus on overcoming specific
challenges, such as the multi-head Janus issue, and on enhancing the overall
efficiency of the generation process. Lastly, Section 3.3 presents our
Decoupled Back Propagation approach. This section is dedicated to outlining
our novel strategy for optimizing the NeRF model. We emphasize our method’s
capacity to improve both the speed and the diversity of 3D generation by
employing an advanced back-propagation technique.

###  3.1 Camera Orientation Conditioned Diffusion Model

Score distillation-based text-to-3D methods, such as those proposed in Poole
et al.[22], Lin et al.[14], and Wang et al. [37], operate under the assumption
that maximizing the likelihood of images rendered from arbitrary viewpoints of
a Neural Radiance Field (NeRF) is tantamount to maximizing the NeRF’s overall
likelihood. While this assumption is rational, the 2D diffusion models these
methods employ lack 3D awareness, often resulting in inconsistent and
distorted geometry in the generated NeRF. To address this limitation and
ensure the reconstructed NeRF maintains 3D consistency, our approach
integrates 3D awareness into the 2D image diffusion generation model.

Prior works, including those by Poole et al.[22] and Wang et al.[37], have
attempted to incorporate a sense of 3D awareness by using text prompts that
vaguely describe the camera viewpoint (for example, ’side view’). However,
this method is inherently flawed due to its ad-hoc nature. The ambiguity
inherent in using text prompts that can represent a wide range of different
pose values makes NeRF generation susceptible to geometric inconsistencies. To
overcome this, we propose an approach that explicitly controls our 2D
diffusion image generation model by inputting encoded camera orientations.

In our proposed method, we utilize a standard Latent Diffusion model [18] with
camera orientation control, which can be mathematically represented as
follows:

$p(\mathbf{x}|\theta)=\prod_{t=1}^{T}p(\mathbf{x}|\theta,t,c^{\prime}),$ (2)

where $p(\mathbf{x}|\theta)$ denotes the data distribution parametrized by
$\theta$, $\mathbf{x}$ represents the data (e.g., an image), and $t$ signifies
the number of diffusion steps. The conditional distribution of the data at
step $t$ is given by $p(\mathbf{x}|\theta,t,c^{\prime})$, where $c^{\prime}$
is a concatenated feature combining the CLIP feature $c$ from the text input
and the encoded orientation $SE(q)$, both having the same shape. Our method
involves adding $p$ to $c$ as a residual, resulting in $c^{\prime}=c+p$.

To encode camera orientation, we first convert the camera orientation to a
quaternion form $q$ due to the numerical discontinuity of the rotation matrix.
To facilitate smooth interpolation between camera poses, we apply a sine
encoding $\left\\{{{SE}}_{i}(q),i=1,...,7\right\\}$, where $i$ represents the
dimension. The sine encoding process is as follows:

$\text{{SE}}_{i}(q)=\sin\left(q\cdot i\right)$ (3)

This encoding allows an intermediate quaternion between two poses $q_{1}$ and
$q_{2}$ to be represented as a simple linear interpolation, which is
straightforward for the network to learn and generalize:

$\text{{SLerp}}(q_{1},q_{2},t)=\frac{{\sin((1-t)\omega)}}{{\sin(\omega)}}q_{1}+\frac{{\sin(t\omega)}}{{\sin(\omega)}}q_{2}$
(4)

where $\omega$ is the angle between the two quaternions, determined as
$\omega=\cos^{-1}(q_{1}\cdot q_{2})$.

We evaluated our camera encoding scheme against alternative approaches like
relative position encoding [31], rotary embeddings [34], and the plain
rotation matrix, concluding that the quaternion form with sine encoding most
effectively distinguishes between view angles.

With the above setup, we fine-tune the diffusion model as shown Fig. 3, and
using the following loss function:

$\displaystyle\min_{\theta};\mathbb{E}{z\sim\mathcal{E}(x),t,\epsilon\sim\mathcal{N}(0,1)}||\epsilon-\epsilon{\theta}(x_{t},t,c,SE(q))||_{2}^{2}.$
(5)

Here, $x_{t}$ is the noisy image generated from random noise $\epsilon$, $t$
is the timestep, $c$ is the condition, $\mathbf{p}$ is the camera condition,
and $\epsilon_{\theta}$ is the view-conditioned diffusion model.

After training the model $\epsilon_{\theta}$, the inference model $f$ can
generate an image through iterative denoising from a Gaussian noise image, as
described in Rombach et al. [26]. This process is conditioned on the
parameters $(t,c,p)$.

Fine-tuning pre-trained diffusion models in this manner equips them with a
generalized mechanism for controlling camera viewpoints, allowing for
extrapolation beyond the objects encountered in the fine-tuning dataset.
Unlike methods that rely on 3D datasets, such as those discussed in Liu et
al.[15] and Shi et al.[30], our model benefits from fine-tuning on the real-
world captured MVImgDataset [40]. This approach avoids the common issues of
reduced content richness and texture quality associated with models trained
exclusively on 3D datasets.

Camera poses in MVImgNet [40] are estimated using COLMAP [29]. Despite being
captured by different annotators, the normalization term used in COLMAP
ensures that the camera poses are canonical across different captures.

Figure 4:  Overview of Text-to-3D Generation Methodologies: This figure
succinctly illustrates our dual approach in applying multi-view diffusion
models for 3D generation. It highlights the use of multi-viewpoint images for
sparse 3D reconstruction, alongside our focus on employing an orientation-
conditioned diffusion model for Score Distillation Sampling (SDS). We figure
showcases our innovative SDS adaptation, where traditional models are replaced
with our orientation conditioned diffusion model, seamlessly integrating
camera parameters and text prompts for enhanced 3D content generation.

###  3.2 Text to 3D generation

We explore two primary methodologies for applying a multi-view diffusion model
to 3D generation. The first method involves using images generated from
multiple viewpoints as inputs for sparse 3D reconstruction. This technique,
while effective, requires high-quality images and poses, which has inspired
our work described in Section 3.3.

The second method leverages the multi-view diffusion model as a prior for
Score Distillation Sampling (SDS). This approach is more robust to
imperfections in images and tends to produce superior output quality. In our
work, we focus on this latter method as described in Fig.4, modifying existing
SDS pipelines by replacing the Stable Diffusion model with our pose-
conditioned diffusion model. Instead of utilizing direction-annotated prompts
as in Dreamfusion [22], we employ original prompts for text embedding
extraction and incorporate camera parameters as inputs.

Furthermore, we use our orientation-conditioned diffusion model to initialize
the variational model in Variational Score Distillation (VSD), since both
models use camera pose as a conditioning element. This integration allows for
a more nuanced and accurate 3D generation process, capitalizing on the
strengths of our developed diffusion model.

###  3.3 Decoupled Back Propagation

Figure 5: Decoupled Sampling: We summarizes our approach to enhance NeRF
optimization speed, highlighting the shift from uniform sampling to targeted
reduction in $T$ steps and the use of the DDIM solver for efficient
computation, thereby improving both the diversity and texture quality in 3D
model generation.

The current methodologies still grapple with slow optimization speeds. As
highlighted in Prolificdreamer [38], the mode-seeking nature of Score
Distillation Sampling (SDS) can lead to over-saturation and reduced diversity.
Prolificdreamer suggests incorporating variational constraints to address
this, but this addition further decelerates the learning process.

Recent advancements in Ordinary Differential Equation (ODE) samplers for
diffusion models, like DDIM [32] and DPM-Solver [17], offer rapid evaluation
of 2D diffusion models by potentially solving the ODE in a single step.

In our approach to decoupled back-propagation, we implement a series of
strategic steps to enhance the efficiency of the process. Initially, we move
away from uniform sampling, opting instead to strategically decrease the
choice of $T$. Following this, as shown in Fig. 5, we sample a random camera
pose and render a corresponding 2D view of the NeRF. Then, using the DDIM
solver, we directly compute $X_{T-step}$, diverging from the traditional
method of calculating $X_{T-1}$. This step size is not static; it’s a
hyperparameter that we methodically reduce during the course of training.
Finally, once $X_{T-step}$ is determined, we proceed to optimize the NeRF at
this specific camera location, employing a Mean Squared Error (MSE) loss for
this purpose.

The DDIM solver formula is expressed as:

$x_{t}=x_{0}+\sqrt{2\beta}\sum_{i=1}^{t}\sigma_{i}\epsilon_{i}$ (6)

where $x_{t}$ denotes the image at time $t$, $x_{0}$ signifies the initial
image, $\beta$ is the parameter governing the diffusion process, $\sigma_{i}$
represents the standard deviation of the noise at step $i$, and $\epsilon_{i}$
is the noise sample at the same step.

This method circumvents the problematic SDS loss and instead employs standard
MSE loss to train NeRF, which results in improved diversity and texture
quality. Towards the end of training, as the step size becomes smaller, the
quality aligns closely with that of a 2D diffusion solver.

A significant advantage of this method is the considerable acceleration of the
training process. The stable diffusion 2.1 model having 890 million
parameters, are substantial in size compared to the much smaller NeRF models,
which have only about 2.3 million parameters.

Decoupled back propagation requires fewer evaluations of the diffusion model,
thus reducing the frequency of optimizing NeRF from different viewing angles.
This efficiency is achievable only when the 2D diffusion model can precisely
generate views from specified locations, making the most of its advanced
capabilities.

SD + Text Suffix |  |
---|---|---
MVDream |  |
Our |  |
| An astronaut riding a horse | A bald eagle carved out of wood
SD + Text Suffix |  |
MVDream |  |
Our |  |
| A bull dog wearing a black pirate hat | A DSLR photo of a ghost eating a hamburger
Figure 6: Qualitative Results of Orientation-Controlled Image Generation: We
showcases the comparative results of our method against the text suffix-based
approach and MVDream [30]. While the text-based method yields images with
inconsistent orientations, both MVDream and our model exhibit precise control
over viewing angles. Notably, our method outperforms MVDream by generating
photorealistic images with richer textures and greater variety, highlighting
its effectiveness in producing high-quality, orientation-specific imagery.

##  4 Experiments

In this section, we begin by reviewing the training details (Section 4.1). We
then assess our orientation conditioned diffusion model in the context of 2D
image generation (Section 4.2), followed by an evaluation of its performance
in 3D object generation using a 2D to 3D lifting method (Section 4.3).
Subsequently, we provide a quantitative evaluation of the generation quality
and speed (Section 4.4) . Due to space limit, ablation study of Decoupled Back
Propagation is left in Appendix.

###  4.1 Training Details

Our model, an orientation-controlled diffusion variant, builds upon the Stable
Diffusion 2.1 framework. It is fine-tuned using the MVImgNet dataset [40],
which comprises 6.5 million frames across 219,188 videos featuring objects
from 238 classes. The fine-tuning process is carried out over 2 epochs. For
the 2D to 3D lifting aspect, we adopt methodologies from Dreamfusion [22, 36].
We have implemented our approach within the Threestudio [1] framework, which
is also employed for replicating baseline models.

###  4.2 Orientation-Controlled Image Generation

Our orientation-controlled diffusion model is specifically trained on an
orientation-annotated dataset. To verify its effectiveness, we initially focus
on generating open-domain objects at specified camera orientations.

We utilized the same prompts as those in MVDream [30]. For each prompt, four
images are generated, each with the same elevation but differing in azimuth by
90 degrees. Both MVDream [30] and our method employ orientation quaternions
for this purpose.

As a comparative measure, we also incorporate a text-based view conditioning
method. This approach involves adding text suffixes like front view, side
view, back view to the text prompt, corresponding to the camera orientation.
This method is a common feature in text-to-3D pipelines, as seen in works like
Threestudio and Stable-Dreamfusion [1, 36].

In Figure 6, we display the results of these generative approaches. The text
suffix-based method tends to produce images in various orientations, rather
than consistent ones, underscoring its limitations. Conversely, both MVDream
and our model demonstrate reliable control over viewing orientation. However,
while MVDream tends to produce 2D images with simplified textures and limited
variety, our method achieves photorealistic imagery. This distinction is
crucial as it directly impacts the quality of text-to-3D generation.

MVDream generate 4-views simultaneously, while our method generate 4-views
independently, it’s not an apple-to-apple comparison to evaluate 4-view
consistency. Considering text-to-3d training require around 10,000 iterations,
consistency within a 4-view pair is not critical. On the other hand, our
method has clear advantage on texture quality and content richness.

|  0∘ |  45∘ |  90∘ |  135∘ |  |  0∘ |  45∘ |  90∘ |  135∘
---|---|---|---|---|---|---|---|---|---

Dreamfusion |  |
---|---|---
ProlificDreamer |  |
MVDream |  |
Our |  |
Our-Normal |  |
| A colorful toucan with a large beak | a bald eagle carved out of wood

Figure 7: Qualitative experiment of text to 3D generation results. Each row
represents a method and each column represents a azimuth angle used to render
the object.

###  4.3 Orientation-Controlled Diffusion for Text to 3D

Following the approach in Dreamfusion [22, 36], we employ the NeRF
architecture with Instant-NGP representation [20] for lifting 2D features to
3D. Our comparison includes prominent baselines like Dreamfusion [22],
Prolificdreamer [38], and MVDream [30]. In Figure. 7, we display our text-
to-3D generation results. Dreamfusion [22] tends to suffer from the Janus
problem, as seen in the duplication of heads in both the toucan and eagle
models. Prolificdreamer [38] achieves impressive texture quality in individual
views, but viewed as a coherent 3D object, it exhibits an even more pronounced
Janus issue. MVDream [30] maintains good 3D consistency, yet it falls behind
in terms of content richness and texture quality compared to other methods.
Our method stands out by being free from the Janus problem and providing
exceptional detail and texture quality.

###  4.4 Quantitative Evaluation

Evaluating the quality of text-to-3D generation is challenging due to the lack
of ground truth 3D scenes corresponding to text prompts. CLIP R-Precision [22]
is one metric, assessing retrieval accuracy through joint embedding of 2D
images and text. However, high image-text matching scores don’t necessarily
imply image consistency.

A-LPIPS [12], calculating average LPIPS [41] between adjacent images, is
another metric. But frame-wise similarity doesn’t guarantee view consistency,
and the Janus problem, characterized by repetitive patterns, could
misleadingly improve this metric. An object appearing identical from every
angle would score highly on A-LPIPS, despite lacking view diversity.

Method |  R-Precision $\uparrow$ | $\text{A-LPIPS}_{\text{Alex}}\downarrow$ | $\text{Zero123}_{grad}\downarrow$
---|---|---|---
Dreamfusion[22] | 58.8% | 0.0250 | 0.0873
Prolificdreamer[38] | 63.4% | 0.0688 | 0.1412
MVDream[30] | 48.4% | 0.0311 | 0.0092
Ours | 55.5% | 0.0415 | 0.0107

Table 1: Comparision of R-Precision, A-LPIPS, and
$\text{Zero123}_{\text{grad}}$. Results are calculated on 64 prompts dataset.

To address these limitations, we developed a 3D topology-aware metric,
leveraging models like Zero123[15], MVDream[30], and our OrientDream. We
sample k uniformly spaced camera poses from an upper hemisphere around each
scene. For adjacent image pairs, we compute the norm of the gradient predicted
by a pre-trained Zero123 [15] model. The average gradient across all frames
and scenes is defined as $\text{Zero123}_{grad}$.

For systematic evaluation, we used the first 64 prompts from Dreamfusion’s
[22] online gallery of 397 prompts. The results, shown in Table.1, indicate
that our model achieves comparable $\text{Zero123}_{grad}$ to MVDream, while
outperforming in R-Precision and A-LPIPS. This suggests our model maintains
similar 3D awareness but offers superior texture quality.

We additionally quantified the occurrence of the Janus problem in our
generated scenes. Instances with anomalies such as extra or missing parts —
like an additional leg, a missing leg, or multiple heads — were identified as
manifestations of the Janus problem. The results of this counting are detailed
in Table. 2.

Method |  # of Janus Scenes$\downarrow$ |  % of Janus Scenes $\downarrow$
---|---|---
Dreamfusion[22] | 42 | 65.625%
Prolificdreamer[38] | 46 | 71.875%
MVDream[30] | 3 | 4.688%
Ours | 5 | 7.813%
Table 2: Counting of Janus Scenes

Speed Analysis: Generation speed is a critical factor affecting usability. As
illustrated in Table. 3, we present the average time required to generate
outputs for 64 prompts. Our method demonstrates a relatively faster generation
speed compared to other methods, owing to our efficient decoupled optimization
process.

Method |  Time Per Scene $\downarrow$
---|---
Dreamfusion [22] | 90mins
Stable-Dreamfusion [36] w/ Instant-NGP [20] | 11mins
Prolificdreamer[38] | 210mins
MVDream[30] | 90mins
Our-SDS | 14 mins
Our-Decoupled | 5mins
Table 3: Generation Speed Analysis

###  4.5 Evaluation of Orientation Conditioned Image Generation.

2D lifting-based text-to-3D methods are heavily dependent on the gradients
generated by their 2D priors. Consequently, in order to understand the
disparities among various text-to-3D methods, our initial focus is on the
quantitative comparison and understanding of 2D diffusion models.

We employ three metrics: Fréchet inception distance (FID) [9], Inception Score
(IS) [28], and CLIP score [8]. FID and IS are used to measure image quality
and CLIP score is used to measure text-image similarity. Results are presented
in Table. 4. Scores are calculated over 397 text prompts from Dreamfusion’s
[22] online gallery, and 10 images are generated for each prompt. The original
StableDiffusion[18] achieves the best score across three metrics. MVDream[30]
get much worse FID after fine-tuning on Objaverse[7]. This partly explains why
it experiences a reduction in texture quality and content diversity in
generated 3D content. For comparison, our method achieves an FID of 16.93,
much closer to the level of the original stable diffusion. This finding
partially reveals why our model could produce photorealistic 3D models, and
show the benefits of fine-tuning on MVImgNet[40].

Model |  FID$\downarrow$ |  IS$\uparrow$ |  CLIP$\uparrow$
---|---|---|---
SD[18] | 12.63 | $\textbf{15.75}\pm 2.87$ | $\textbf{35.70}\pm 1.90$
MVDream[30] | 32.06 | $13.68\pm 0.41$ | $31.31\pm 3.12$
OrientDream | 16.93 | $13.28\pm 0.88$ | $34.91\pm 3.91$
Table 4: Quantitative evaluation on image synthesis quality.

##  5 Conclusion

In summary, our proposed approach makes a modest yet meaningful contribution
to the field of text-to-3D generation. The camera orientation conditioned
diffusion model, a key aspect of our research, marks an advancement in
generating 3D content with greater accuracy and consistency. We’ve
circumvented significant challenges, such as the multi-head Janus issue, and
refined the process of transforming text descriptions into 3D models. The
introduction of the decoupled back propagation method represents our
commitment to improving the efficiency and diversity of 3D model generation.
We are hopeful that our work will provide a valuable foundation for future
research in various applications of 3D technology.

## References

  * [1] Threestudio project.  https://github.com/threestudio-project/threestudio.  Accessed: 2023-11-17.
  * Armandpour et al. [2023] Mohammadreza Armandpour, Huangjie Zheng, Ali Sadeghian, Amir Sadeghian, and Mingyuan Zhou.  Re-imagine the negative prompt algorithm: Transform 2d diffusion into 3d, alleviate janus problem and beyond.  _arXiv preprint arXiv:2304.04968_ , 2023.
  * Bai et al. [2023] Haoran Bai, Di Kang, Haoxian Zhang, Jinshan Pan, and Linchao Bao.  Ffhq-uv: Normalized facial uv-texture dataset for 3d face reconstruction.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 362–371, 2023.
  * Balaji et al. [2022] Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, Bryan Catanzaro, et al.  ediffi: Text-to-image diffusion models with an ensemble of expert denoisers.  _arXiv:2211.01324_ , 2022.
  * Chan et al. [2022] Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al.  Efficient geometry-aware 3d generative adversarial networks.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022.
  * Choi et al. [2020] Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha.  Stargan v2: Diverse image synthesis for multiple domains.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 8188–8197, 2020.
  * Deitke et al. [2022] Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi.  Objaverse: A universe of annotated 3d objects.  _arXiv preprint arXiv:2212.08051_ , 2022.
  * Hessel et al. [2022] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi.  Clipscore: A reference-free evaluation metric for image captioning, 2022.
  * Heusel et al. [2018] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium, 2018.
  * Ho and Salimans [2022] Jonathan Ho and Tim Salimans.  Classifier-free diffusion guidance.  _arXiv:2207.12598_ , 2022.
  * Ho et al. [2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  _Advances in Neural Information Processing Systems_ , 2020.
  * Hong et al. [2023] Susung Hong, Donghoon Ahn, and Seungryong Kim.  Debiasing scores and prompts of 2d diffusion for view-consistent text-to-3d generation, 2023.
  * Huang et al. [2023] Yukun Huang, Jianan Wang, Yukai Shi, Xianbiao Qi, Zheng-Jun Zha, and Lei Zhang.  Dreamtime: An improved optimization strategy for text-to-3d content creation, 2023.
  * Lin et al. [2022] Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin.  Magic3d: High-resolution text-to-3d content creation.  _arXiv:2211.10440_ , 2022.
  * Liu et al. [2023a] Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick.  Zero-1-to-3: Zero-shot one image to 3d object.  _arXiv:2303.11328_ , 2023a.
  * Liu et al. [2023b] Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang.  Syncdreamer: Generating multiview-consistent images from a single-view image, 2023b.
  * Lu et al. [2022] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu.  DPM-solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps.  In _Advances in Neural Information Processing Systems_ , 2022.
  * Metzer et al. [2023] Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or.  Latent-nerf for shape-guided generation of 3d shapes and textures.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023.
  * Mildenhall et al. [2021] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng.  Nerf: Representing scenes as neural radiance fields for view synthesis.  _Communications of the ACM_ , 2021.
  * Müller et al. [2022] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller.  Instant neural graphics primitives with a multiresolution hash encoding.  _arXiv:2201.05989_ , 2022.
  * Ouyang et al. [2023] Yichen Ouyang, Wenhao Chai, Jiayi Ye, Dapeng Tao, Yibing Zhan, and Gaoang Wang.  Chasing consistency in text-to-3d generation from a single image, 2023.
  * Poole et al. [2022] Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall.  Dreamfusion: Text-to-3d using 2d diffusion.  _arXiv:2209.14988_ , 2022.
  * Qian et al. [2023] Guocheng Qian, Jinjie Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li, Hsin-Ying Lee, Ivan Skorokhodov, Peter Wonka, Sergey Tulyakov, and Bernard Ghanem.  Magic123: One image to high-quality 3d object generation using both 2d and 3d diffusion priors, 2023.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.  Learning transferable visual models from natural language supervision.  2021\.
  * Raffel et al. [2020] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _JMLR_ , 2020.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022.
  * Saharia et al. [2022] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al.  Photorealistic text-to-image diffusion models with deep language understanding.  _arXiv:2205.11487_ , 2022.
  * Salimans et al. [2016] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen.  Improved techniques for training gans.  In _Proceedings of the 30th International Conference on Neural Information Processing Systems_ , page 2234–2242, Red Hook, NY, USA, 2016. Curran Associates Inc.
  * Schönberger and Frahm [2016] Johannes Lutz Schönberger and Jan-Michael Frahm.  Structure-from-motion revisited.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016.
  * Shi et al. [2023] Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang.  Mvdream: Multi-view diffusion for 3d generation, 2023.
  * Singer et al. [2022] Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al.  Make-a-video: Text-to-video generation without text-video data.  _arXiv:2209.14792_ , 2022.
  * Song et al. [2020a] Jiaming Song, Chenlin Meng, and Stefano Ermon.  Denoising diffusion implicit models.  _arXiv:2010.02502_ , 2020a.
  * Song et al. [2020b] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.  Score-based generative modeling through stochastic differential equations.  _arXiv:2011.13456_ , 2020b.
  * Su et al. [2021] Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu.  Roformer: Enhanced transformer with rotary position embedding.  _arXiv:2104.09864_ , 2021.
  * Takikawa et al. [2022] Towaki Takikawa, Alex Evans, Jonathan Tremblay, Thomas Müller, Morgan McGuire, Alec Jacobson, and Sanja Fidler.  Variable bitrate neural fields.  In _ACM SIGGRAPH 2022 Conference Proceedings_ , 2022.
  * Tang [2022] Jiaxiang Tang.  Stable-dreamfusion: Text-to-3d with stable-diffusion, 2022.  github.com/ashawkey/stable-dreamfusion.
  * Wang et al. [2022] Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich.  Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation.  _arXiv:2212.00774_ , 2022.
  * Wang et al. [2023] Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu.  Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation.  _arXiv:2305.16213_ , 2023.
  * Xu et al. [2023] Jiale Xu, Xintao Wang, Weihao Cheng, Yan-Pei Cao, Ying Shan, Xiaohu Qie, and Shenghua Gao.  Dream3d: Zero-shot text-to-3d synthesis using 3d shape prior and text-to-image diffusion models.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023.
  * Yu et al. [2023] Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Tianyou Liang, Guanying Chen, Shuguang Cui, and Xiaoguang Han.  Mvimgnet: A large-scale dataset of multi-view images.  In _CVPR_ , 2023.
  * Zhang et al. [2018] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang.  The unreasonable effectiveness of deep features as a perceptual metric.  In _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 586–595, 2018.


# Identification of Substation Configurations in Modern Power Systems using
Artificial Intelligence

Dulip Madurasinghe, _Student Member, IEEE_
Real-Time Power and Intelligent Systems Laboratory  Holcombe Department of
Electrical
and Computer Engineering
Clemson University
Clemson, SC 29634, USA
<EMAIL_ADDRESS>Ganesh K. Venayagamoorthy, _Fellow, IEEE_
Real-Time Power and Intelligent Systems Laboratory This work is supported in
part by the US National Science Foundation (NSF) under grants 2131070 and the
Duke Energy Distinguished Professor Endowment Fund. Any opinions, findings and
conclusions or recommendations expressed in this material are those of the
author(s) and do not necessarily reflect the views of National Science
Foundation and Duke Energy. Holcombe Department of Electrical
and Computer Engineering
Clemson University
Clemson, SC 29634, USA
<EMAIL_ADDRESS>
###### Abstract

Power system transmission network topology is utilized in energy management
system applications. Substation configurations are fundamental to transmission
network topology processing. Modern power systems consisting of renewable
energy sources require reliable and fast network topology processing due to
the variable nature of wind and solar power plants. Currently used
transmission network topology processing, which is based on the relay signals
communicated through SCADA is not highly reliable or highly accurate.
Substation configuration identification (SCI) for different substation
arrangements including main and transfer bus arrangement (MTBA), ring bus
arrangement (RBA), and single bus arrangement (SBA) is investigated.
Synchrophasor measurement based SCI for functional arrangements (FA) using
artificial intelligence (AI) approaches is proposed in this paper. This method
improves monitoring FA. Typical results for MTBA, RBA and SBA substation
configuration identification is presented. A modified two-area four-machine
power system model with two grid connected solar PV plants consisting of MTBA,
RBA and SBA is simulated on real-time digital simulator. AI based SCI is shown
to accurately identify all possible FAs for the three substation arrangements
under any operating condition.

###### Index Terms:

Substation Configurations, Artificial Intelligence, Phasor Measurement Units,
Transmission Network Topology

##  I Introduction

Power system is a large, geographically distributed single circuit that supply
power to the consumers from the generation stations. Modern power system is
highly dynamic due to the nature of renewable energy sources (RES),
distributed energy resources, etc. Thus, modern power system control requires
high resolution real-time system monitoring and processing capability. The
control center is responsible for the reliable and secure power system
operation. Thus, control center requires the knowledge of each and every
component’s connectivity to the transmission network. Power system
transmission network topology processing (TNTP) is used in the control center
to understand the power system transmission network component connectivity.
TNTP defines the node-breaker model (NBM) based on status of breakers located
in substations [1]. The NBM is converted to a dynamic bus-branch model (BBM)
to be used in the energy management system (EMS) applications. Substation Bus
sections or branches with very low or zero impedance are eliminated under this
conversion to facilitate efficient numerical calculations [2].

BBM derived by the TNTP is used by EMS online applications including state
estimation [2], [3], [4], power flow [5]. At the same time, reliable BBM is
important for energy market application such as locational marginal pricing
[6], security constrained optimal dispatch and optimized renewable
integration, unit commitment [7] . Power system planning division uses TNTP to
conduct contingency analysis and stability analysis [2], where online BBM can
be used to decrease the gap between planning simulation results and the
practical results [2].

Currently, TNTP uses the relay signals collected by the remote terminal units
in the substation about breaker status, which communicated to the control
center via SCADA to derive the power system TNTP [2]. Thus, the substation
configuration is the fundamental for TNTP. The existing TNTP is not precise
[8]. At the same time, breaker malfunction or false data injection by a man-
in-the-middle attack [9] can easily compromised the derived TNTP in the
control center. Which cause erroneous in results obtained by the applications
based on TNTP. There has been numerous research focus on this problem. As
described in [10], graph methods can be integrated to derive the changes in
the TNTP. As proposed in [11], novel wireless sensor network can be establish
to identify the branch status, which can be used to derive the TNTP. Thus, an
improved TNTP or an validation for the existing TNTP is required. This can be
done either communicating the breaker status through a secure communication
channel or data-driven TNTP based on the measurements communicated through a
secure communication channel respectively. Although implemented PMUs has the
capability of integrating the digital relay signals of breakers status to it’s
data frame [12], implementing such infrastructure in the existing
synchrophasor network will be expensive. Thus, a redundant TNTP that use the
measurements communicated through secure communication channel with only
addition of computational nodes to the system without modification to the
infrastructure is ideal.

To enhance the situational awareness of the power system, Phasor Measurement
Units (PMUs) are introduced [13] in the substations to enable high resolution
real-time measurements. PMUs measure phasor quantities at a higher rate.
Typically, 30Hz reporting frequency is used. This data set is a representation
of the power system state, which open possibilities to investigate the system
online and implement TNTP based solely on the measurements. Although there are
proposed methods to improve the TNTP by introducing measurement based TNTP
[14]\- [15] most of this are lacking in-depth investigation of substation
configurations. The substation configuration identification is studied in
[16], [17], which are still rely on the breaker signals, which is again
depends on the existing method of TNTP. This study is an effort to investigate
the different substation configurations. Substation configuration
identification (SCI) is the first level of TNTP.

Figure 1: Substation Circuit Arrangements: a) Main and Transfer Bus
Arrangement (MTBA), b) Ring Bus Arrangement (RBA) and c) Single Bus
Arrangement (SBA).

High voltage substations are established in critical locations of the power
system transmission network to enable control of power flow and monitoring of
system states to supply power securely. Connectivity of power system critical
components such as generators, transmission lines, transformers, loads, etc.
can be controlled in the substations by operating the breakers accordingly.
Power system measurement instrumentation such as PMUs are deployed in the
substations to monitor branch current, bus-section voltage, power flow and
frequency. There are several substation arrangements [18]. In this study,
three such substation arrangements are considered. They are main and transfer
bus arrangement (MTBA), ring bus arrangement(RBA) and single bus arrangement
(SBA) as shown in Fig. 1. Selection of the particular substation arrangement
to extend in the power system transmission network is based on the type of
connections, level of required reliability and the expenditure capability.
Furthermore explanation on selected three substation arrangement types are
explained in section II.

The demand side power requirement is becoming complex. The utility grid
requires modernization to be compatible with complex demand requirement. TNTP
is one application that will come under this modernization [19]. The future
power grid will have higher number of switching operations and frequent
topology changes. This is driven by the rapid integration of renewable source
based generation which is intermittent and the accumulation of commercial
micro-grids, which islanding will be a standard frequent operation. Artificial
intelligence (AI) algorithms can be used to establish a real-time TNTP [15].
Furthermore, AI techniques can be used to provide enhanced situational
awareness with PMU data [20], [21]. In this study, AI based SCI for MTBA, RBA
and SBA is proposed. Use of data-driven AI based approach is new to SCI. The
authors considered this approach due to following considerations,

  * •

Future power system transmission network is expected to handle more frequent
topology changes. Thus, more intelligent AI based approach is preferred.

  * •

The proposed AI based approach is scalable to handle substation expansions
with growing power system complexity.

  * •

Operation and management of modern power systems will be more dependent on
real-time complex data and thus, AI based approaches are more robust for
substation configuration identification.

The SCI is based on either knowledge based logical decision making (LDM) or
neural network (NN). An LDM or NN based SCI for each substation type is
selected based on complexity of the arrangement and efficiency of the
algorithm.

SCI is tested using PMU data from a real-time simulation of MTBA, RBA and SBA.
Section II, elaborates on the substation configurations. Section III, explains
the AI based SCI methodology. In section IV, results of all possible
substation configurations for the MTBA, SBA and RBA is presented. Section V
concludes the study by elaborating on the findings and future directions of
the study.

##  II Substation Configuration

A substation consists of branches, bus-sections, protection equipment
(relays), measurement instruments (PMUs) and switching equipment (Breakers,
Isolators). Each of these equipment is required in the power system control.
TNTP is defined based on the status of each breaker established in all
substations. The breaker status is identified using the relays and the relay
signals are communicated to the control center, which is eventually used to
established the TNTP. The substation configurations can be analyzed as
follows,

  1. 1.

Component Arrangement (CA).

  2. 2.

Functional Arrangement (FA).

CA explains the physical connectivity of each branch and bus-section of the
substation. CA is the NBM in the substation context. Each breaker status is
used to define the CA of the substation. FA is explained by the fundamental
electrical circuit nodal analysis theorems. FA is the BBM in the substation
context. The FAs are a subset of the CAs.

Since the power system TNTP is highly dependent on the reliable relay signal
communication, an fail-safe redundancy or a validation is recommended. Use of
synchrophasor network measurements to establish an intelligent data-driven BBM
is ideal to overcome this issue. A novel BBM can be defined by integrating all
substation FAs. This novel BBM can be used as a replacement or to validate the
current BBM, which is derived using TNTP based on relay signals communicated
to the control center [2]. Thus, extension of the proposed FA for SCI in this
study can be used to improve the reliability of the existing BBM of the
network and the EMS applications. The remaining subsections describe the MTBA,
RBA and SBA arrangements and elaborate on FAs.

###  II-A Main and Transfer Bus Arrangement (MTBA)

The MTBA substation arrangement design to connect all the branches in between
main bus and transfer bus. At any point of active operation the main bus can
be taken out of service and the substation operation can be continued without
interruptions using only the transfer bus. Although the transfer bus is
typically used in a main bus maintenance, it can be used as a redundancy for
the main bus in the substation under emergency operation. PMUs are installed
to measure the bus voltages. MTBA can have $2^{n}$ (where n = number of
breakers in the substation) CAs and only two FAs. A load transfer substation
of MTBA with two branches is shown in the Fig. 1 (a).

###  II-B Ring Bus Arrangement (RBA)

The RBA consists of bus-sections connected through circuit breakers. RBA is a
reliable bus arrangement. Any single breaker malfunction does not affect the
substation operation. Combination of multiple circuit breakers operations are
used to disconnect and reconnect high voltage power system components such as
generators, transmission lines, transformers, etc. to the substation and
change the power routing pattern. The common practice by the utilities to
measure each current through connected branches and the voltages of the bus
sections at the connected node of the ring bus arranged substation. Thus,
current phasor measurements of each branch connecting to the RBA substation
and voltage phasor measurements of each bus section is available at the
substation and the control center. RBA have $2^{n}$ (where n = number of bus
sections or breakers in the substation) CAs and ($2^{n}-n-1$) FAs. The RBA
test case of this study, is a four-node RBA substation with PMU measurement
points is shown in the Fig. 1 (b). An example of CAs and FAs of a four-node
RBA is shown in Fig. 2.

Figure 2: Four-node RBA. Functional arrangement of the component arrangements
in (a) - (e) is shown in (f).

###  II-C Single Bus Arrangement (SBA)

The SBA is the fundamental substation arrangement. Reliability is very low due
to lack of redundancy under breaker failure or bus fault. All the branches are
directly connected to a single bus through circuit breakers. PMUs are
installed to measure all branch currents and the bus voltage. Thus, both
branch currents and the bus voltage is available at the control center. SBA is
a special case, where each FA, and corresponding CA is unique. SBA have
$2^{n}$ (where n = number of breakers or branches in the substation) CAs and
FAs. In this study, SBA substation test case has 6 branches connected as shown
in Fig. 1 (c).

##  III AI Based Substation Configuration Identification

In this study, two AI based algorithms are considered to identify the
functional arrangements to derive substation configuration, namely, logical
decision making (LDM) and neural network (NN). For each type of substation
arrangement, either LDM or NN is selected considering the complexity of the
problem and the arrangement based on the computation efficiency. Proposed
MTBA, SBA and RBA SCIs are expected be reliable, intelligent and easy to
implement. The SCI can be developed using knowledge based LDM algorithm that
follows the flow diagrams shown in Figs. 4, 6 and 8 or using NNs that are
trained offline and use at any substation similarly arranged, despite of the
powers system state. Training data set generation for NN approach can be done
based on the simple logical algorithm that follows the highlighted section
that is shown in Figs. 4, 6 and 8. Thus, the proposed SCI development is only
requires basic substation information such as substation type, number of
branches. Thus, NN based SCIs can be implemented and trained offline with a
generic binary training data set and used in the field. The proposed SCI
approach can be summarized as shown in Fig. 3.

Figure 3: Procedure of the proposed substation configuration identification
approach.

###  III-A Main and Transfer Bus Arrangement (MTBA)

MTBA substation with two branches is considered as the test case, which is
shown in the Fig. 1 (a). The PMUs are in place to measure main and transfer
bus voltages as shown in orange circles in the Fig. 1 (a). The two possible
FAs of any MTBA substation can be inferred by using the main and transfer bus
voltages. The intelligent decision procedure is shown in the Fig. 4 flow
diagram. A LDM algorithm that checks equality of main and transfer bus
voltages can be developed by following the flow diagram shown Fig. 4 or a
single binary neuron can be trained using the pre-generated training data by
following the flow diagram shown in Fig. 4 as shown in Fig. 5 to identify the
FA. The NN is for MTBA is a single binary neuron. This is a simple linear
separable problem. Therefore a NN is not needed. Although for comparison of
LDM vs NN purpose, a single neuron NN is developed.

Figure 4: A flow diagram for substation configuration identification of
functional arrangement of MTBA type substation. Figure 5: The single binary
neuron structure based substation configuration identification developed for
MTBA substation shown in Fig. 1 (a).

###  III-B Ring Bus Arrangement (RBA)

RBA substation with four-nodes is considered as the test case, which is shown
in the Fig. 1 (b). The PMUs are in place to measure each bus-section voltages
as shown in orange circles in the Fig. 1 (b). Each FA of the RBA substation
can be inferred based on the circuit nodal analysis using only voltage phasor
measurements of the bus sections. In the substation each bus section voltage
phasor is compared with neighbor bus section voltage to identify the nodal
connectivity which is separated by the breaker. Bus-section voltage
measurements are considered as the inputs in the RBA to identify FAs as shown
in the Fig. 6 flow diagram.

Figure 6: A flow diagram for substation configuration identification of
functional arrangement of RBA type substation.

X and Y are consecutive nodes in the RBA. The quantities shown are PMU
measurements. For RBA substation arrangement an LDM can be setup by following
the procedure shown in the Fig. 6 or a feed forward multilayer perceptron
(MLP) neural network can be trained as shown in Fig. 7using the generic pre-
generated training data as shown in TABLE I which was derived by following the
flow diagram shown in Fig. 6 to get the FAs. LDM or MLP can be selected based
on the computation efficiency. Furthermore, for RBA, a single LDM or MLP is
tasked with identifying the connectivity of two consecutive bus-sections
(nodes). Since the approach identifying the FA of the substation, all single
breaker operations are considered similar to the the FA shown in Fig. 2. (f).
Each LDM or MLP takes in the two consecutive bus section voltage phasor
measurements and compare them and converts into logical values in pre-
processing. The binary outputs of the pre-processing is used as the inputs to
the LDM or MLP training data generation. The output is the connectivity in
between the respective two bus-sections. All node connectivity statuses
combined together to get the FA of the RBA substation as shown in Fig. 7.
Equation (1) shows the MLP that derive the connection between X and Y bus-
sections (nodes) of RBA.

Figure 7: The neural network structure of the MLP based substation configuration identification developed for a four-node RBA substation shown in Fig. 1 (b). TABLE I: Training Dataset of MLP for X and Y node connectivity of RBA substation. Inputs | Output
---|---
C | A | B | Node X and Y Connection Status
1 | 1 | 1 | 1
1 | 0 | 0 | 0
0 | 1 | 1 | 0
0 | 1 | 0 | 0
0 | 0 | 1 | 0
0 | 0 | 0 | 0

$Y_{X,Y}=f(A,B,C,B_{1},[W_{1}],B_{2},[W_{2}])$ (1)

###  III-C Single Bus Arrangement (SBA)

SBA substation with six branches considered as the test case, which is shown
in Fig. 1 (c). Branch current phasor measurements (circled in purple) and bus
voltage phasor measurements (circled in orange) are in place as shown in Fig.
1 (c). All FA configurations of the SBA can be derived considering circuit
nodal analysis using branch currents and the bus voltage. In the SBA
substation, each branch current availability is considered to identify the
branch connectivity to the bus which is connected through the breaker. It is
important to defined the branch current availability rule based on the system
expert knowledge about the current measurements for a no-load line vs
disconnected branch. Combining the branch connection status with the bus
voltage to identify the single bus isolation to confirm the substation is in
service or not will be used to derive the Fa for the SBA. The SBA SCI method
identify FAs as shown in Fig. 8 flow diagram.

Figure 8: A flow diagram for substation configuration identification of
functional arrangement of SBA type substation.

For SBA substation arrangement, an LDM can be setup by following the procedure
shown in the Fig. 8 or a MLPs can be trained as shown in Fig. 9 using the
generic pre-generated training data as shown in TABLE II by following the flow
diagram shown in Fig. 8 to identify FAs. Furthermore for SBA, each branch is
considered by a single LDM or MLP, which takes bus voltage phasor and branch
current phasor as input to identify the branch status and eventually derive
the FAs as explained by the Fig. 9. MLP module use to identify the
connectivity of $i^{th}$ branch of SBA can be summarized into (2).

Figure 9: The neural network structure of the MLP based substation configuration identification developed for six branch SBA substation shown in Fig. 1 (c). TABLE II: Training dataset of MLP for branch $i$ connectivity of SBA substation. Inputs | Output
---|---
B | A | Branch $i$ Connection Status
1 | 1 | 1
0 | 1 | 0
0 | 0 | 0

$Y_{i}=f(A,B,B_{1},[W_{1}],B_{2},[W_{2}])$ (2)

##  IV Results & Discussion

Figure 10: Test Case: Modified Two-Area Four-Machine Power System with
renewable energy sources.

A modified two-area four-machine power system model is shown in Fig. 10. The
model consists of four conventional power plants at substation 5G, 6G, 10G and
11G and two RES based grid connected solar power plants at substation 12 and
13 instead of single generators in the original model [22]. The loads are
distributed through load distributions substations 5L, 6L, 7L, 9L, 10L and
11L. Substation 5L is configured as MTBA, substation 6 configured as SBA and
substation 5 is configured as RBA as shown in Fig. 1. The simulation of the
power system is carried out on the real-time digital simulator (RTDS). All
measurements are obtained using PMUs.

MTBA, RBA and SBA substation arrangements are simulated on the RTDS simulator.
The simulation PMU measurements is used to test the LDM and NN based SCIs
proposed in section III. The SCI methods are developed in MATLAB. All possible
substation FAs derived from the SCIs are cross-checked with the actual
substation FAs in the test cases including unstable power system operation
caused by some substation configurations, due to exceeding the (n-1)
contingency level. Under all these operating conditions, substation
configurations are checked. The SCI accuracy is not affected by the state of
the power system. The TABLE III shows the SCI accuracy for the MTBA, RBA and
SBA substation types. Computation time for LDM and NN for all three substation
types estimated for single PMU frame processing is shown in TABLE IV. The
timing values are in the TABLE IV are based on the same computational platform
(The testing platform is a consist of Intel Xeon(R) Gold 3.3GHz with 63.7GB
RAM). The computational timing is not a stable metric to estimate the
computational efficiency, since it is based on the computational platform.
Thus, a time ratio index is introduced. The time ratio index is the ratio
between the LDM computation time and the NN computation time, which normalize
the dependency on the computational platform. Based on the time ratio index,
it is identified that for MTBA substation arrangement efficiency is close in
both LDM and NN and for RBA and SBA substation types NN is efficient.

TABLE III: Substation Configuration Identification Accuracy of the LDM and NN approaches for the test cases MTBA. RBA and SBA substations shown in Fig. 1. | Substation
---
Type
| Substation
---
ID
| Number of
---
Breaker
CAs | FAs | SCI Accuracy
MTBA | 5L | 3 | 8 | 2 | 100%
RBA | 5 | 4 | 16 | 11 | 100%
SBA | 6 | 6 | 64 | 64 | 100%
TABLE IV: Computation performance of the LDM and NN approaches for the test cases MTBA, RBA and SBA substations shown in Fig. 1 Substation Type |  Substation ID | Computation Time (us) |  Time Ration Index (t(LDM)/t(NN)
---|---|---|---
LDM | NN
MTBA | 5L | 27.98 | 26.73 | 1.04
RBA | 5 | 77.35 | 51.56 | 1.50
SBA | 6 | 130.45 | 84.67 | 1.54
Figure 11: Identified functional arrangements using proposed substation
configuration identification approaches for the MTBA substation shown in Fig.
1 (a).

The FAs for MTBA are shown in Fig. 11. There are two FAs. (i) case connects
both Main and Transfer bus together and the (ii) case separate main and
transfer bus and connect all the branches to the transfer bus, which isolate
the main bus from the system.

Figure 12: Identified functional arrangements using proposed substation
configuration identification approaches for the RBA substation shown in Fig. 1
(b).

The FAs for RBA substation are shown in Fig. 12. There are total of twelve FAs
possible for a four-node RBA substation. Which can be described under five
categories as shown in Fig. 12. (i) category shows all branches are connected
together through the RBA substation. This scenario can be occurred under five
different CAs as shown, where all four breakers are closed or in any single
breaker opened cases, which result in the same FA. (ii) category shows a
single branch is disconnected from the substation. There are four possible CAs
and respective four FAs in four-node RBA substation. Any two consecutive
breakers can be opened to disconnect a single branch from the RBA substation.
This characteristic of the RBA substation improve the substation reliability,
where single breaker malfunction won’t affect the power system operation.
(iii) category shows system separation conducted by the RBA substation. This
characteristics can be used for islanding part of the power system or limit
cascading power system failures. In the four-node RBA, there are two possible
system separation CAs and respective two FAs. These two separation can be done
either opening only breakers ”a” and ”c” or breakers ”b” and ”d”. (iv)
category shows two consecutive branches were disconnected from the system in
four-node RBA substation by opening three consecutive breakers. There are four
possible CAs and respective four FAs. (v) category shows the four-node RBA
substation is isolated from the power system by disconnecting all branches
from the substation. This scenario can be occurred under major maintenance or
a post blackout scenario. There is a single CA resulted under this category
and the a respective single FA.

Figure 13: Identified functional arrangements using proposed substation
configuration identification approaches for the SBA substation shown in Fig. 1
(c).

The FAs for SBA substation are shown in Fig. 13. Under SBA, all branches are
connected through a common node. SBA arrangement is used in non-priority
substation due to lack of reliability. There are 64 different CAs and
respective 64 FAs in the six-branch SBA test case. As mentioned in the Section
II - subsection C, SBA CAs are identical to FAs. These 64 FAs can be
considered under seven categories as shown in Fig. 13. (i) category shows all
the branches connected to the substation single bus. There is one possible FA.
(ii) category shows single breaker is opened, which result in single branch
disconnected from the substation. There are six possible FAs. (iii) category
shows any two breakers are opened simultaneously, which result in two branches
disconnected from the substation. There are 15 possible FAs. (iv) category
shows any three breakers are opened simultaneously, which result in three
branches disconnected from the substation. There are 20 possible FAs. (v)
category shows any four breakers are opened simultaneously, which result in
four branches disconnected from the substation. There are 15 possible FAs.
(vi) category shows any five breakers are opened, which result in five
branches disconnected from the substation. There are six possible FAs. (vii)
category shows all six breakers are opened, which result in substation
isolation from the power system. There is one possible FA.

##  V Conclusion

Artificial intelligence based approaches for power system substation
configuration identification for main and transfer bus arrangement, ring bus
arrangement, and single bus arrangement has been presented in this paper. AI
based approaches for substation configuration identifications were developed
offline with the knowledge of the physical structure of the substation
arrangement type and number of branches or bus-sections (nodes). The AI based
substation configuration identifications were implemented and tested using
data from phasor measurement units collected on a real-time simulation of a
modified two-are four-machine power system model with two grid connected solar
PV plants. The substation functional arrangements were identified accurately
and fast for different power system operating states. Ongoing studies include
extension of AI approaches for more complex substation arrangements and
integrating the substation configuration identification into the transmission
network topology processor for energy management system applications.

## References

  * [1] Power World Corporation, “Unified Power System Models through Integrated Topology Processing (ITP),” https://www.powerworld.com/files/PowerWorld-Integrated-Topology-Processing1.pdf, [Last visited: 18-Feb-2022].
  * [2] S. Grijalva, “Direct utilization of planning applications in the real-time environment,” in _2008 IEEE/PES Transmission and Distribution Conference and Exposition_ , 2008, pp. 1–6.
  * [3] M. Farrokhabadi and L. Vanfretti, “State-of-the-art of topology processors for ems and pmu applications and their limitations,” in _IECON 2012 - 38th Annual Conference on IEEE Industrial Electronics Society_ , 2012, pp. 1422–1427.
  * [4] L. Zhang, A. Bose, A. Jampala, V. Madani, and J. Giri, “Design, testing, and implementation of a linear state estimator in a real power system,” _IEEE Transactions on Smart Grid_ , vol. 8, no. 4, pp. 1782–1789, 2017.
  * [5] B. Park, J. Holzer, and C. L. DeMarco, “A sparse tableau formulation for node-breaker representations in security-constrained optimal power flow,” _IEEE Transactions on Power Systems_ , vol. 34, no. 1, pp. 637–647, 2019\.
  * [6] T. Wu, Z. Alaywan, and A. Papalexopoulos, “Locational marginal price calculations using the distributed-slack power-flow formulation,” _IEEE Transactions on Power Systems_ , vol. 20, no. 2, pp. 1188–1190, 2005.
  * [7] A. V. Ramesh, X. Li, and K. W. Hedman, “An accelerated-decomposition approach for security-constrained unit commitment with corrective network reconfiguration,” _IEEE Transactions on Power Systems_ , vol. 37, no. 2, pp. 887–900, 2022.
  * [8] M. Kezunovic, “Monitoring of power system topology in real-time,” in _Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS’06)_ , vol. 10, 2006, pp. 244b–244b.
  * [9] G. Liang, S. R. Weller, J. Zhao, F. Luo, and Z. Y. Dong, “The 2015 ukraine blackout: Implications for false data injection attacks,” _IEEE Transactions on Power Systems_ , vol. 32, no. 4, pp. 3317–3318, July 2017.
  * [10] G. Liu, C. Yuan, X. Chen, J. Wu, R. Dai, and Z. Wang, “A high-performance energy management system based on evolving graph,” _IEEE Transactions on Circuits and Systems II: Express Briefs_ , vol. 67, no. 2, pp. 350–354, 2020\.
  * [11] M. A. Judge, A. Manzoor, F. Ahmed, S. Kazmi, Z. A. Khan, U. Qasim, and N. Javaid, “Monitoring of power transmission lines through wireless sensor networks in smart grid,” in _Innovative Mobile and Internet Services in Ubiquitous Computing_ , L. Barolli and T. Enokido, Eds. Cham: Springer International Publishing, 2018, pp. 162–170.
  * [12] K. D. Jones, J. S. Thorp, and R. M. Gardner, “Three-phase linear state estimation using phasor measurements,” in _2013 IEEE Power Energy Society General Meeting_ , 2013, pp. 1–5.
  * [13] J. De La Ree, V. Centeno, J. S. Thorp, and A. G. Phadke, “Synchronized phasor measurement applications in power systems,” _IEEE Transactions on Smart Grid_ , vol. 1, no. 1, pp. 20–27, June 2010.
  * [14] M. Delimar, I. Pavic, and Z. Hebel, “Artificial neural networks in power system topology recognition,” in _The IEEE Region 8 EUROCON 2003. Computer as a Tool._ , vol. 2, 2003, pp. 287–291 vol.2.
  * [15] D. M. V. Kumar and S. C. Srivastava, “Power system network topology processing based on artificial neural networks,” _Electric Machines & Power Systems_, vol. 26, no. 3, pp. 249–263, 1998. [Online]. Available: https://doi.org/10.1080/07313569808955820
  * [16] G. Korres, P. Katsikas, and G. Chatzarakis, “Substation topology identification in generalized state estimation,” _International Journal of Electrical Power & Energy Systems - INT J ELEC POWER ENERG SYST_, vol. 28, pp. 195–206, 03 2006.
  * [17] M. Heidarifar and H. Ghasemi, “A network topology optimization model based on substation and node-breaker modeling,” _IEEE Transactions on Power Systems_ , vol. 31, no. 1, pp. 247–255, 2016.
  * [18] J. D. McDonald, _Electric Power Substations Engineering_ , third edition ed. CRC Press, 2012.
  * [19] NERC, “Node-Breaker Modeling Representation,” https://www.nerc.com/comm/PC/Model\%20Validation\%20Working\%20Group\%20MVWG\%202013/December\%202016\%20-\%20Node\%20Breaker\%20Model\%20Representation\%20Webinar.pdf, [Last visited: 15-April-2022].
  * [20] Q. Wang and S. Bu, “Deep learning enhanced situation awareness for high renewable-penetrated power systems with multiple data corruptions,” _IET Renewable Power Generation_ , vol. 14, no. 7, pp. 1134–1142, 2020.
  * [21] G. K. Venayagamoorthy, “Computational approaches for bad data handling in power system synchrophasor networks,” _IFAC Proceedings Volumes_ , vol. 47, no. 3, pp. 11 269–11 274, 2014, 19th IFAC World Congress. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1474667016434075
  * [22] P. Kundur, “Power system stability and control,” McGraw-Hill, 1994.