Application of the Quantum Potential Neural Network to multi-electronic atoms

Hector H. Corzo,a,∗ Arijit Sehanobish,b,∗ and Onur Karac

Received Date, Accepted Date
First published on the web Date

DOI: 10.1039/b000000x

In this report, the application of the Quantum Potential Neural Network (QPNN)
framework to many electron atomic systems is presented. For this study, full
configuration interaction (FCI) one–electron density functions within
predefined limits of accuracy were used to train the QPNN. The obtained
results suggest that this new neural network is capable of learning the
effective potential functions of many electron atoms in a completely
unsupervised manner, and using only limited information from the probability
density. Using the effective potential functions learned for each of the
studied systems the QPNN was able to estimate the total energies of each of
the systems (with a maximum of 10 trials) with a remarkable accuracy when
compared to the FCI energies.

††footnotetext: a Center for Chemical Computation and Theory
University of California, Merced<EMAIL_ADDRESS>b
Yale University; E-Mail<EMAIL_ADDRESS>c
Hindsight Technology Solutions E-Mail<EMAIL_ADDRESS>∗
Equal Contribution

##  1 Introduction

The Quantum Potential Neural Network (QPNN) introduced by Sehanobish et al.
QPNN is a new type of neural network which is capable of learning reliable
effective potential functions for a wide variety of quantum systems in a
completely unsupervised manner. This new neural network is not only an
unprecedented example of unsupervised learning in quantum mechanics, but the
theoretical framework in which this neural network was built upon makes the
QPNN scheme for computing effective potential functions easily expandable and
applicable to any dimension of a mathematical space; which was demonstrated in
the original publication by Sehanobish et al. By using limited information
obtained from the system’s probability distribution, the QPNN is able to shape
an effective potential function that may be used to describe the system’s
underlying quantum phenomena. This approach for defining effective potential
functions to describe quantum systems rather than handcrafting solutions for
the Schrödinger equation uses a neural network that is developed on the
intrinsic formalism for solving inverse problems. In particular, the QPNN was
constructed based on the formulation of the inverse Schrödinger equation
aster2018parameter, jensen2018numerical, vogel2002computational,
beals2009strings and designed to learn potential functions directly from
observables in an unsupervised manner. Thus, the QPNN can generate simpler and
succinct functions that can be used to define effective Hamiltonians for the
description of a variety of quantum systems using only a small portion of the
available information known about the system. Since these effective
Hamiltonians can be generalized to obtain other observables, QPNN can provide
unique insights into complex quantum phenomena where only a small amount of
information is available and can be generalized as a low computational
approximation to compute observable of systems that are outside the training
set.

In this work the scope and applicability of the QPNN is extended to effective
potential functions for multi-electronic atoms. After a brief review of the
QPNN formalism and loss function, we describe the generalities of the
electronic structure method applied to define the densities that were used for
the different experiments. Subsequently, the validity of the QPNN loss
function for the calculation of $N$–electron systems is evaluated by
performing different experiments that corroborate the use of the loss
function. Finally, we provide a glance as to what can be expected for the QPNN
in the near future.

##  2 Quantum Potential Neural Network (QPNN)

The characterization of several phenomena and systems in quantum mechanics and
electronic structure theory, in many cases, require knowledge of the $n$ state
wave–function $\psi_{n}(\vec{x})$ which arises from the solution of the
time–independent Schrödinger equation,

$\hat{\rm H}\psi_{n}(\vec{x})=E\psi_{n}(\vec{x}),$ (1)

where $E$ indicates the energy of the system, $n$ the quantum state, and
$\hat{\rm H}$ is the Hamiltonian operator of the system, which is an Hermitian
operator acting on an infinite dimensional space of $L^{2}$ functions. The
Hamiltonian operator, $\hat{\rm H}$, is expressed as the sum of the kinetic
($\hat{\rm T}$) and potential energy ($\hat{\rm V}$) operators for all
particles in the quantum system,

$\hat{\rm H}=\hat{\rm T}+\hat{\rm V}.$ (2)

Generally, the kinetic energy operator contained in $\hat{\rm H}$ only depends
on the second derivatives of the wave–function, with respect to its spatial
coordinates. Thus, the definition of the time–independent Schrödinger equation
to describe a wide variety of quantum systems strongly depend on the
formulation of the Hamiltonian, and in particular, the rationale of the
potential used to describe the system. Thus equation 2 may be expressed as

$\leavevmode\nobreak\ \hat{\rm
H}=-\frac{\hbar^{2}}{2m}\frac{\partial^{2}}{\partial\vec{x}^{2}}+\hat{\rm
V}(\vec{x})\equiv-\frac{\hbar^{2}}{2m}\nabla_{\vec{x}}^{2}+\hat{\rm
V}(\vec{x}).$ (3)

An alternative way to characterize quantum systems is presented by rethinking
a quantum event as a solution of a inverse problem, e.g. finding an effective
potential or function that contains all the important physical constrains that
generated the observed outcomes of the quantum phenomena. Inverse problems
like this one are at the core of quantum mechanics and electronic structure
theories, for example, in the field of density functional theory (DFT),
jensen2018numerical, parr1995density, Burke_2005 density–to–potential
inversions are fundamental for the definition of new approximations. Moreover,
great amount of what is known about the structure of matter has come from
solving scattering problems which are mathematically defined as inverse
problems. zakhariev2012direct, jensen2018numerical, vogel2002computational
With this in mind, Sehanobish and coworkers QPNN formulated and designed the
Quantum Potential Neural Network (QPNN). For time–independent quantum systems,
the QPNN uses the probability density ($|\psi(\vec{x})|^{2}$) of the quantum
system to define an approximated wave–function,
$|\psi|=\sqrt{|\psi(\vec{x})|^{2}}$. Although this approximated probability
amplitude does not contain all the information, and in particular the phase
knowledge, found in the analytical probability amplitude, the QPNN can use
random sets of points from this approximated probability amplitude to
construct an effective potential function that accurately describes the
quantum system under study. The success of the QPNN for deducting reliable
time–independent effective potential functions is due to the definition and
efficient implementation of a new loss function ($L_{TISE}(\theta)$) that
obeys the time–independent Schrödinger equation (Equation 1) and effectively
demands energy conservation for time–independent systems. This loss function
reads

$L_{TISE}(\theta)=\bigg{\lvert}\bigg{\rvert}D\left(-\frac{\hbar^{2}}{2m}\frac{\Delta_{\vec{x}}|\psi|}{|\psi|}+U_{\theta}(\vec{x})\right)\bigg{\lvert}\bigg{\rvert}_{2}^{2},$
(4)

where $D$ is the total derivative operator (with respect to $\vec{x}$) acting
on multi-variate function
$-\frac{\hbar^{2}}{2m}\frac{\Delta_{\vec{x}}|\psi|}{|\psi|}+U_{\theta}(\vec{x})$,
$\lvert\rvert\cdot\lvert\rvert_{2}$ is the Frobenius norm and $U_{\theta}$ the
parametric function that delineates the effective potential for the given
quantum system. In the QPNN protocol, the $L_{TISE}(\theta)$ learns the
parametric function $U_{\theta}$ in an unsupervised manner. Because
$U_{\theta}$ is given by a differential equation, an initial condition is
introduced which ensures the uniqueness of the solution via the
Picard–Lindelöf theorem. Thus the loss function may be rewritten as

$L(\theta)=L_{TISE}(\theta)+(U_{\theta}(\vec{x})-y)^{2},$ (5)

where $\vec{x}$ is some point in the domain of the function and $y$ is the
expected ground truth value for the true potential at that point. It is
important to point out that although using the approximated probability
amplitude $|\psi|$, instead of the analytical one ($\psi$), to solve for the
potential may lead to the incorrect potential at finitely many points where
$\psi$ changes signs, this obstacle, however, has not been found for the QPNN
approach.

##  3 Virial Theorem

In this section, we revisit our formulation of QPNN. We use the virial theorem
to redefine a new loss function. The virial theorem in our context states

######  Theorem 1 (Virial).

$2\ \mathbb{E}(\hat{T})+\mathbb{E}(\hat{V})=0$

We define a new loss function that obeys the virial theorem. For a mini-batch
$\\{x_{1},\cdots x_{n}\\}$, we implement the virial theorem via the following
equation.

$\frac{1}{n}\sum_{i=1}^{n}\left(-\frac{\hbar^{2}}{m}(\Delta_{\vec{x}})(x_{i})|\psi|+U_{\theta}(\vec{x}_{i})\right)=0$
(6)

Let us call the left hand side of the equation 6 $L_{\text{virial}}(\theta)$
We define a new loss function by the equation:

$L(\theta)=L_{TISE}(\theta)+L_{\text{virial}}(\theta)$ (7)

Note that in this formulation we do not explicitly need an initial condition
but the virial theorem essentially guarantees us an unique solution.

##  4 Correlated electronic densities for atomic systems

Non–relativistic selected configuration interaction (SCI) wave–functions
bunge2018present, bunge2006select provide reliable and stable electronic
densities for atomic systems over a wide range of values of the radial
coordinate. These wave–functions are defined by a finite expansion

$\Psi=\sum_{K,p}\phi_{K}^{p}a_{Kp},$ (8)

where $\phi_{K}^{p}a_{Kp}$ are the configuration state functions expressed as
linear combination of Slater determinants,

$\phi_{K}^{p}=\sum_{q=1}^{p}D_{Kq}c^{p}_{kq}.$ (9)

These configuration state functions are successively orthonormalized $L^{2}$
and $S^{2}$ symmetric projections of the N–electron Slater determinants
($D_{k,i}$) made up of orthonormal spin-orbitals,

$\phi_{K}^{p}=O(L^{2},S^{2})\sum_{i=1}^{p}D_{Kq}b^{p}_{kq}.$ (10)

The spin–orbitals, $\psi(i,l,m_{l},m_{s})$, are linear combinations of
Slater–type orbitals (STO’s) times a spherical harmonic, times a spin
function:

$\psi(i,l,m_{l},m_{s})=R_{il}(r)Y_{lm_{l}}(\theta,\phi)\xi_{m_{s}}(\sigma_{s})$
(11)

where

$R_{il}(r)=\sum_{j}S_{jl}(r)a_{jli}$ (12)
$S_{jl}(r)=N_{jl}r^{n_{j}-1}Exp[-Z_{jl}r].$ (13)

The one–electron reduced density matrix, then, is calculated in terms of a
spin–orbital base according to

$\gamma_{(}x,x^{\prime})=\sum_{k,l}\psi^{\star}_{k}(x^{\prime})\psi_{l}(x)\gamma_{kl},$
(14)

that after unitary transformation and diagonalization yields

$\gamma_{(}x,x^{\prime})=\sum_{k}\chi^{\star}_{k}(x^{\prime})\chi_{k}(x)n_{k},$
(15)

where $\chi_{k}$ are the natural spin–orbitals and $n_{k}$ the occupation
numbers. Thus, the density in terms of the natural radial orbitals
$\chi_{il}(r)$ may be written as

$\rho(r)=\sum_{i}f(i,l,ms)n_{il}\chi^{2}_{il}(r),$ (16)

where $f(i,l,ms)$ represent the $LS$ symmetry factors and the natural radial
orbitals that are defined as

$\chi^{2}_{il}(r)=\sum_{j}S_{jl}(r)c_{ji}.$ (17)

For this study the ground state densities of three multi-electronic atoms, Li
(fig. 1), Be (fig. 2), and Ne (fig. 3) were obtained with an average of 484 CI
term expansion for the interacting and non–interacting spaces, and a STOs
basis with harmonics in a $0\leq l\leq 5$ range. The densities for these
atomic systems were defined in terms of the STO’s and the density matrix using
the Slater–Condon rules for monoelectronic operators and only the $LS$
eigenfunctions that correspond to the leading configuration state function.
For each of the atoms, only the configurations that contribute to their ground
states were considered to generate their density.

Fig. 1: Lithium Density for the ${}^{2}S$ state used to train QPNN. Fig. 2:
Beryllium Density for the ${}^{1}S$ state used to train QPNN. Fig. 3: Neon
Density for the ${}^{1}S$ state used to train QPNN.

##  5 Model Architecture and Training Details

The QPNN model is a $3$ layer feedforward network with hidden sizes of
$64,128$ and $128$ with $\tanh$ non–linearity in between the hidden layers. A
residual connection between the second and third layer was used. This residual
connection helps to secure faster training, stable gradients and also results
in a smooth loss landscape. li2018visualizing The inputs to the model are the
$\vec{x}$ spatial coordinates. For the network training, $5000$ of these
coordinates were randomly selected from the domain of definition of each
atomic system. The model was trained for $500$ epochs with Adam optimizer
kingma2017adam and a learning rate of $1\text{\times}{10}^{-3}$. For stable
training, we used a $L^{2}$ regularization on the weights of the network with
the weight tuning parameter of $1\text{\times}{10}^{-4}$. Finally to use the
Picard–Lindelöf theorem, we use a gradient clip that ensures the Lipschitzness
of our learned function. Recall that a function
$f:\mathbb{R}^{n}\rightarrow\mathbb{R}^{m}$ is Lipschitz iff

$||f(x)-f(y)||\leq K||x-y||.$ (18)

In fact, by Rademacher’s Theorem, an almost everywhere differentiable function
is Lipschitz continuous iff it has a bounded total derivative.

The network was implemented using Python and PyTorch. All derivatives are
calculated using the Autograd library in PyTorch.

##  6 Experiments

We validated the QPNN model on the Lithium, Beryllium and Neon atoms. For all
the systems, the domain of definition for the spatial coordinate is
$[0.1,10]$. As explained in equation 5, an appropriate initial condition is
added for each system to ensure that an unique potential function is learned.
The learned effective potential functions for Li(fig. 4), Be(fig. 5) and
Ne(fig. 6) atoms present a maximum whose height and width correlate with the
number of electrons in the atom. The larger the number of electrons the taller
and more narrow the maximum in the potential curve.

Fig. 4: Lithium ${}^{2}S$ state learned potential. Fig. 5: Beryllium ${}^{1}S$
state learned potential. Fig. 6: Neon ${}^{1}S$ state learned potential.

Using the learned effective potential functions and partial irregular
information from the approximate wave–functions, the total energy for each of
the system can be estimated. Table 1 shows the full SCI energies ($E_{FCI}$)
for each of the atoms studied and the energies estimated by the QPNN model
over $10$ trials. In general, the energy computed by QPNN presented a
remarkable accuracy when compared to the ground true full SCI energies (which
are by definition equivalent to the full CI (FCI) energies for each the
atomsbunge2018present).

Table 1: FCI and QPNN Learned Energies in a.u. System | $E_{FCI}$ | $E_{QPNN}$
---|---|---
Lithium Atom | $-7.477487$ | $-7.5431$
Beryllium Atom | $-14.66101$ | $-14.6142$
Neon Atom | $-128.888004$ | $-128.8584$

The behavior of the energy as a function of the trials for each of the atoms
is presented in figure 7, 8 and 9 for the Lithium, Beryllium and Neon atom
respectively. From the three atomic system studied with the QPNN framework,
the largest error in energy correspond to the Lithium atom. This fact is quite
interesting, because unlike the Be and Ne atoms with closed–shell reference
configurations for the ground state, $1s^{2}2s^{1}$ and $1s^{2}2s^{2}2p^{6}$
respectively, the reference configuration for the ground state of the Li atom
is $1s^{2}2s^{1}$ which correspond to an open–shell configuration with a
unpaired electron in a s shell. It is well known that computing the energy of
open–shell systems tend to be more challenging due to their electronic
structure; characterized by having different number or $\alpha$ and $\beta$
electrons and different considerations for the exchange and correlation
electronic effects. Table 2 shows the quantitative results for the energies
computed by the QPNN model, for the closed–shell systems, the energies
computed by the QPNN enclose a remarkable accuracy when compared to the ground
true energies. Which suggest that by considering a larger pool of test
subjects to train the neural network, the energy estimations may improve and
interpolations (and possible extrapolation) to the energy of systems outside
these pool of test subjects may be possible. In other words, if larger set of
atoms is used to train the QPNN the accuracy of the energies predicted by the
QPNN may improve since the QPNN may have a larger amount and variety (atoms
have different number of protons and electrons which change their information
pattern) of information where to learn from. Thus, by introducing a new
training paradigm and reformulating the neural network protocol, the QPNN will
be able to forecast details of the elements not included in their training
set. Atoms share between them a specific information structure or periodic
pattern. This periodic arrangement between atoms and their properties
(information) was what helped Dmitri Mendeleev to developed his extended
version of the periodic table. Using this same idea, the QPNN could be trained
using the information of a given set of atoms to 1) improve the accuracy of
the energies predicted by the QPNN, in particular the energies of open–shell
systems. 2) Evaluate if with a larger training set information about atoms, in
particular their energies, outside this training set can be interpolated or
extrapolated in a unsupervised manner, i.e., Can the QPNN make valid
predictions about observables of atomic systems in an unsupervised manner?

Fig. 7: Lithium ${}^{2}S$ state learned energy. Fig. 8: Beryllium ${}^{1}S$ state learned energy. Fig. 9: Neon ${}^{1}S$ state learned energy. Table 2: A Quantitative Analysis for the QPNN System |  | RMSE between True and Learned
---
Energies
Lithium Atom |  $7\text{\times}{10}^{-2}$ $\pm$ $4\text{\times}{10}^{-2}$
Beryllium Atom |  $1\text{\times}{10}^{-2}$ $\pm$$2\text{\times}{10}^{-2}$
Neon Atom |  $1\text{\times}{10}^{-2}$ $\pm$$9\text{\times}{10}^{-3}$

##  7 Conclusion

The application of QPNN was extended to multi–electronic atoms. It is shown
that the original formulation of the loss function for the time–independent
systems implemented in the QPNN is valid for $N$–electron atomic systems. As
in the case of systems with analytical time–independent Schrödinger equation
solution, the QPNN is capable of learning reliable effective potential
functions in an unsupervised manner. The effective potentials learned by the
QPNN using the SCI probability densities for each of the atoms are able to
produce accurate total energy that are comparable to the FCI energies. This
present the possibility of defining a low dimensional and succinct function
capable to predict highly accurate energies for atomic and molecular systems.
The energy accuracy presented by the learned potential also opens the
possibility to define new density functional approximations using the QPNN
into a new end–to-end manner. A future improvement on the QPNN may involve the
consideration of symmetry eigenfunctions and constrains related open–shell and
degenerate configuration in the network loss function as well as a larger
training set. These improvement in the training and formulation of the loss
function could result in a machine learning framework capable of compute
observables such as the energy of atomic or molecular systems without the
consideration of one particle orbitals approximation and with the correct
state description for the final state. The results found with the QPNN also
open the possibility of defining a new approximation with a low computational
cost that can compute the total energy of the system by inferring and
reconstructing part of the essential information that is lost by describing a
quantum system by using only the probability density. New development and
improvements to the presented QPNN framework for the modeling of atomic
systems are under way.

##  Appendix A Appendix

###  A.1 Electron Density

Mathematically, the function $\rho(r)$ exhibits non-analytical behaviour in
the form of a discontinuity of its gradient exactly at the origin r = 0, (i.e.
the positions of the nuclei) which results poles in the potential energy
function at these positions (V = $-\infty$)hoggan2013proceedings. The
electronic wave–function must satisfy the cusp condition (which result in
spikes of $\rho(r)$ here) in the neighbourhood of each of the nuclei
electronic wave–function, where $\rho(r)\sim e^{−2Zr}$. Note: How sharp the
spikes observed around the cusp depends on the charge of the nucleus Z such
that an infinitesimal deviation from the position of the nucleus has to be
accompanied by such a decreasing of the density given by $\lim_{r\rightarrow
0}\frac{\mathrm{d}\ln\rho(r)}{\mathrm{d}r}=-2Z$. Figure 10 illustrates a
contour map of the electron density distribution in a plane containing the
nucleus for the n = 1 level of the H atom. The distance between adjacent
contours is 1 atomic unit (a.u.). The numbers on the left-hand side on each
contour correspond to the electron density in a.u. The numbers on the right-
hand side correspond to the proportion of the total electronic charge which
lies within a sphere of that radius. The figure therefore reflects the fact
that 99% of the single electronic charge of the H atom lies within a sphere of
radius 4 a.u.

Figure 10: Cusp behavior of $\rho$ near r=0

###  A.2 Virial Theorem

In the context of DFT the virial theorem serves as one the most important
exact constraints to be satisfied. Acting as a constraint, the virial theorem
has been used to calculate atomic energies within the Kohn-Sham DFT formalism
and it has also frequently utilized as an indicator of the quality of basis
sets in computational chemistry tasks RosasCrawford, Baskerville2019. In
addition to providing a means for researchers to assess the quality of
approximate calculations when the exact results are unknown a priori, the
virial theorem facilitates the optimization of the electronic wave–function in
a large space of nonlinear parameters.levy85, averil81, ghoshparr85