# Giant van Hove Density of States Singularities and Anomalies of Electron and
Magnetic Properties in Cubic Lattices

P.A. Igoshev and V.Yu. Irkhin  620108 M.N. Mikheev Institute of Metal Physics,
Ekaterinburg, Russia,
620002 Ural Federal University, Yekaterinburg, Russia

###### Abstract

Densities of states for simple (sc) and base-centered (bcc) cubic lattices
with account of nearest and next-nearest neighbour hopping integrals $t$ and
$t^{\prime}$ are investigated in detail. It is shown that at values of
$\tau\equiv t^{\prime}/t=\tau_{\ast}$, corresponding to the change of
isoenergetic surface topology, the formation of van Hove $\bf k$ lines takes
place. At small deviation from these special values, the weakly dispersive
spectrum in the vicinity of van Hove lines is replaced by a weak $\bf
k$-dependence in the vicinity of few van Hove points which possess huge masses
proportional to $|\tau-\tau_{\ast}|^{-1}$. The singular contributions to the
density of states originating from van Hove points and lines are considered,
as well as the change in the topology of isoenergetic surfaces in the $\bf
k$-space with the variation of $\tau$. Closed analytical expressions for
density of states as a function of energy and $\tau$ in terms of elliptic
integrals, and power-law asymptotics at $\tau=\tau_{\ast}$ are obtained.
Besides the case of sc lattice with small $\tau$ (maximum of density of states
corresponds to energy level of X $\bf k$-point), maximal value of the density
of states is always achieved at energies corresponding to inner $\bf k$-points
of the Brillouin zone positioned in high-symmetry directions, and not at zone
faces.

##  I Introduction

Figure 1: (a) The dependence of important energy points on $\tau$ for the
square lattice. Dashed (solid) line corresponds to local minimum (maximim),
dotted line corresponds to saddle point. (b) DOS of the square lattice at
different $\tau$. Dashed line shows the position of jump at
$\epsilon=w_{\Gamma}$ for $\tau=0.75$

The structure of the spectrum of non-interacting Bose and Fermi excitations is
important for many problems of solid state physics. These include, for
example, the electronic structure in the tight binding approximation and the
spectrum of spin waves for free magnons 1969:Jelitto ; 1972:Swendsen .
Subsequently, these results can be used for the investigations of magnetic and
electron properties within different microscopic models, e.g. the Heisenberg,
Kondo and Hubbard models with account of correlation effects. Besides that,
peculiar properties of materials with density of states (DOS) singularities
for electron and magnon excitations can be explained by the calculation of
their first-principle spectrum. It is well known that DOS peculiarities near
the Fermi level are crucial for ferromagnetic ordering, and large DOS values
favor another Fermi-liquid instabilities (superconducting, antiferromagnetic
etc.).

Figure 2: $\tau$ dependence of van Hove levels $w$ for a) sc lattice, b) bcc
lattice. The solid line corresponds to the local maximum of spectrum, the
dotted line to the local minimum, the short-dotted line to the saddle point.

Formation of narrow van Hove peaks of electron density of states results in
the localization of the states and in an increase of correlation effects role.
This fact was discussed in Ref. 1993:Trefilov  in the context of band
structure calculations focusing on anomalies of lattice and magnetic
properties. A geometric origin of electronic DOS peaks was investigated in
detail in Ref. 1990:Peschanskih . Simple examples are the bcc phase of Ca and
fcc phase of Sr where weakly dispersive one-dimension manifolds (their
prototype is a van Hove line) are positioned at the face of the Brillouin zone
near the Fermi level. In bcc Ca these manifolds correspond to parts of P–N and
N–H lines, whereas in fcc Sr to X–U, U–L, L–K, K–U, and K–W directions. An
analysis shows that $D$-line of the bcc lattice typically looks like van Hove
line in Li, V, Fe, Ba 1978:Moruzzy ; 1986:Papacostantopoulos .

For the paramagnetic phase spectrum of Ni, the L point of the Brillouin zone
possesses a large effective mass and actually provides strong magnetic
properties of Ni (“van Hove magnet”) and its temperature dependence of
magnetic susceptibility 2017:Katanin . The origin of weak ferromagnetism of
ZrZn2 is probably the contribution of a vicinity of the L point to the
electron DOS, which was revealed in the first-principle calculation 2001:Santi
. Large mass at the van Hove $\mathbf{k}$-point R provides weak ferromagnetism
(with Curie temperature 41 K) of Ni3Al 2011:Hamid , which apparently invokes
non-Fermi-liquid dependence of electron interaction contribution to
resistivity at low temperatures due to the scattering by spin fluctuations
2005:Niklowitz . Weak itinerant ferromagnet Y2Ni7 has low Curie temperature 58
K, small effective magnetic moment 0.44 $\mu_{\rm B}$ and the Rhodes-Wohlfarth
ratio about 6 Y2Ni7 . Band structure calculations predict the flat band
spectrum in the vicinity of the van Hove point $\Gamma$ and weak dispersion in
the P–Z direction of rhombohedral Brillouin zone (possible van Hove line or a
pair of large mass van Hove points) 2015:Singh .

Strong peculiarities of DOS originate from a vicinity of closely positioned
van Hove points (we call this van Hove structure). An effect of such structure
on electron and lattice properties, including many-body screening anomalies,
is considered in Refs. Katsnelson_Trefilov ; 1985:Katsnelson_Trefilov ;
Katsnelson_Trefilov2 ; 1992:Irkhin .

Analytical expression for DOS and lattice Green’s functions are obtained for a
few two- and three-dimensional lattices within the nearest-neighbour hopping
approximation which yields an even DOS function $\rho(\epsilon,\tau=0)$ for
bipartite lattices (square, sc and bcc) 1969:Jelitto ; 1971:Morita ;
1971:Katsura .

However, for a number of modern solid state problems the next-nearest
neighbour hopping has crucial value. Here belong the problem of electron
spectrum of high-temperature superconductivity in copper oxides ($t-J$ model)
1997:Izumov ; 2003:Damascelli  and metal-insulator transition in the
antiferromagnetic phase 2016:Timirgazin ; 2016:Timirgazinfcc ; 2019:Igoshev .

A conventional analysis of the density of electron states based on the band
calculations within the framework of tetrahedron method 1994:Andersen  is
unable to provide adequate description of van Hove singularities (which are
crucial for magnetic and superconducting instabilities) because of incorrect
account of contributions from small $\mathbf{v}(\mathbf{k})$ velocity points.
An alternative for such an approach is an analytical approximation of the
spectrum in the vicinity of high-symmetry points where the spectrum is almost
flat (dispersionless) 2017:Stepanenko .

Figure 3: DOS $t\rho(\epsilon)$ for sc (a) and bcc (b) lattice at different
$\tau$. Vertical dashed line is positioned at the giant van Hove singularity
energy levels.

The van Hove singularities of one-particle excitation spectrum are promising
for formation of novel or exotic instabilities. On the other hand, they are
important for the long-standing problem of connection of magnetic properties
of classical magnets like bcc-iron, cobalt etc. with their electron structure.
Therefore, a corresponding general treatment of van Hove singularities for
various three-dimensional lattices is useful; this is the aim of the present
work.

##  II Van Hove singularities for cubic lattices

General definition of the density of states for any one-particle excitation
spectrum $\varepsilon(\mathbf{k})$ reads

$\rho(\epsilon)=\frac{1}{N}\sum_{\mathbf{k}}\delta(\epsilon-\varepsilon(\mathbf{k})).$
(1) Table 1. Non-equivalent van Hove singularity $\mathbf{k}$-points for sc
lattice, see Fig. 2a.
---
Arrows denote the change of the type of van Hove singularity point as $\tau$
approaches $\tau^{\rm sc}_{\ast}$ from below.
$\mathbf{k}$ | $t(\mathbf{k})$ | inverse masses | signature
$\mathbf{k}_{\Gamma}$ | $w^{\rm sc}_{\Gamma}=-6+12\tau$ | $a^{\rm sc}_{1},a^{\rm sc}_{1},a^{\rm sc}_{1};a^{\rm sc}_{1}=2(1-4\tau)$ | min$\rightarrow$max
$\mathbf{k}_{\rm R}$ | $w^{\rm sc}_{\rm R}=+6+12\tau$ | $-a^{\rm sc}_{2},-a^{\rm sc}_{2},-a^{\rm sc}_{2};a^{\rm sc}_{2}=2(1+4\tau)$ | max
$\mathbf{k}_{\rm X}$ | $w^{\rm sc}_{\rm X}=-2-4\tau$ | $2,2,2(4\tau-1)$ |  $(++-)\rightarrow$min
$\mathbf{k}_{\rm M}$ | $w^{\rm sc}_{\rm M}=+2-4\tau$ | $-2,-2,2(4\tau+1);$ | $(+--)$
$(0,p^{\rm sc}_{1},p^{\rm sc}_{1})$ | $w^{\rm sc}_{\varSigma\ast}=-\tau^{-1}+2-4\tau$ | $-2\tau a^{\rm sc}_{3},-a^{\rm sc}_{3},a^{\rm sc}_{3};a^{\rm sc}_{3}=\tau^{-1}-4$ |  $\tau>1/4$,$(++-)$
$(p^{\rm sc}_{2},p^{\rm sc}_{2},p^{\rm sc}_{2})$ | $w^{\rm sc}_{\varLambda\ast}=-3\tau^{-1}/4$ | $2a^{\rm sc}_{4},-a^{\rm sc}_{4},-a^{\rm sc}_{4};a^{\rm sc}_{4}=(2\tau)^{-1}((4\tau)^{2}-1)$ |  $\tau>1/4$, $(+--)$

Peculiarities of $\rho(\epsilon)$ result in anomalies of electron and magnetic
properties, and corresponding phase transitions. There exists a general
relation between peculiarities of DOS for the spectrum
$\varepsilon(\mathbf{k})$ and topological properties of the spectrum
1953:vanHove . If excitation velocity
$\mathbf{v}(\mathbf{k})=\partial\varepsilon(\mathbf{k})/\partial\mathbf{k}$
vanishes at a $\mathbf{k}$ point, this is called the van Hove point. Energy
levels $\epsilon=\varepsilon(\mathbf{k})$ corresponding to van Hove
$\mathbf{k}$-points and only they provide peculiarities of DOS as a function
of $\epsilon$. The type of the singularity corresponding to an isolated van
Hove $\mathbf{k}$-point is determined by the signature of quadratic form with
the matrix $\partial^{2}\varepsilon(\mathbf{k})/\partial k_{i}\partial k_{j}$,
specifying the behavior of the spectrum in the vicinity of this point.

For a three-dimensional lattice, a local $\mathbf{k}$-minimum (maximum)
corresponds to one-sided square-root DOS growth (decrease) as $\epsilon$ moves
away from the level $\epsilon_{0}=\varepsilon(\mathbf{k})$:
$\rho(\epsilon)=\rho(\epsilon_{0})+A\sqrt{\theta(\pm(\epsilon-\epsilon_{0}))|\epsilon-\epsilon_{0}|}+O(\epsilon-\epsilon_{0})$
in the vicinity of $\epsilon_{0}$; the saddle $\mathbf{k}$-point of signature
$++-$ ($+--$) corresponds to one-side square-root decrease
$\rho(\epsilon)=\rho(\epsilon_{0})-A\sqrt{\theta(\mp(\epsilon-\epsilon_{0}))|\epsilon-\epsilon_{0}|}+O(\epsilon-\epsilon_{0})$,
$A>0$ being a constant proportional to square-root of product of masses (more
accurately, of eigenvalues of mass tensor).

If a spectrum, obtained within some first-principle calculations, possesses
weak dispersion and is non-monotonous on a whole segment in some high-symmetry
direction in the $\mathbf{k}$-space, there is one or a few $\mathbf{k}$-points
with huge masses.

We assume that there is a hopping within first (integral $t$) and second
(integral $t^{\prime}=\tau t$) coordination spheres:
$\varepsilon(\mathbf{k})=t(\mathbf{k};\tau)$. Here and below all energy scales
are taken in $t$ units. For bipartite lattices the symmetry

$\rho(-\epsilon;-\tau)=\rho(\epsilon;\tau)$ (2)

holds. Considering this property, we assume below $\tau\geq 0$.

Table 2. Non-equivalent van Hove singularity $\mathbf{k}$-points for bcc
lattice, see Fig. 2b.
---
An arrow denotes the change of the type of van Hove singularity point at
approaching $\tau$ from below to above $\tau^{\rm bcc}_{\ast}$
$\mathbf{k}$ | $t(\mathbf{k})$ | inverse masses | signature
$\mathbf{k}_{\Gamma}$ | $w^{\rm bcc}_{\Gamma}=-8+6\tau$ | $a^{\rm bcc}_{1},a^{\rm bcc}_{1},a^{\rm bcc}_{1};a^{\rm bcc}_{1}=8(1-\tau)$ | min$\rightarrow$max
$\mathbf{k}_{\rm H}$ | $w^{\rm bcc}_{\rm H}=+8+6\tau$ | $-a^{\rm bcc}_{2},-a^{\rm bcc}_{2},-a^{\rm bcc}_{2};a^{\rm bcc}_{2}=8(1+\tau)$ | max
$\mathbf{k}_{\rm P}$ | $w^{\rm bcc}_{\rm P}=-6\tau$ |  $a^{\rm bcc}_{3},a^{\rm bcc}_{3},a^{\rm bcc}_{3}$; $a^{\rm bcc}_{3}=8\tau$ | min
$\mathbf{k}_{\rm N}$ | $w^{\rm bcc}_{\rm N}=-2\tau$ | $-4\tau,4(1+\tau),4(-1+\tau)$ | $(+--)\rightarrow(++-)$
$(p^{\rm bcc}_{1},p^{\rm bcc}_{1},p^{\rm bcc}_{1})$ | $w^{\rm bcc}_{\varLambda\ast}=2\tau(2\tau^{2}-3)$ | $2a^{\rm bcc}_{4},2a^{\rm bcc}_{4},-a^{\rm bcc}_{4};a^{\rm bcc}_{4}=8\tau(1-\tau^{2})$ |  $\tau<1$, $(++-)$
$(0,0,p^{\rm bcc}_{2})$ | $w^{\rm bcc}_{\Delta\ast}=2\tau-4\tau^{-1}$ | $a^{\rm bcc}_{5},a^{\rm bcc}_{5},-a^{\rm bcc}_{5};a^{\rm bcc}_{5}=8(\tau^{-1}-\tau)$ |  $\tau>1$, $(+--)$

For the square lattice, the van Hove saddle $\mathbf{k}$-point always exists
by topological reasons and yields the logarithmic contribution to DOS. In the
nearest-neighbour approximation for bipartite lattices, the singularity
position at the center of the band coincides with the Fermi level for half-
filling case, so that the ferromagnetic instability is suppressed by
antiferromagnetic one (for which the free energy singular contribution is
renormalized by correlation effects 2019:Igoshev ) owing to perfect nesting of
the Fermi surface. Including the next-nearest hopping separates the
logarithmic DOS peak from the Fermi level corresponding to half-filling and
eventually results in the formation of the van Hove line $\Gamma$X
(X$=(0,\pi)$), $\varSigma$ direction, and in the change of the topology of the
spectrum at $\tau=1/2$ 1997:Oles . With further increase of $t^{\prime}$, the
local maximum of the spectrum is formed at the $\Gamma$ point, which yields a
jump in DOS doubling its value, whereas the van Hove saddle point (denote it
as $\varSigma^{\ast}$) migrates along the direction $\Gamma$M (M being
$(\pi,\pi)$) from the $\Gamma$ point (at $\tau<1/2$ the saddle point is
positioned at the X point) 2017:Igoshev , see DOS plots at different $\tau$ in
Fig 1b. Explicit expression for the square lattice DOS within the nearest and
next-nearest neighbour hopping approximation has the form

$\rho_{\rm
sq}(\epsilon;\tau)=\frac{2(1+\theta(\psi(\epsilon,\tau)))}{\pi^{2}}\\\
\times\begin{cases}F\left(-\zeta(\epsilon,\tau),\psi(\epsilon,\tau)\right),&w_{\rm
X}(\tau)<\epsilon<w_{\rm r}(\tau),\\\
F\left(\zeta(\epsilon,\tau),\varphi(\epsilon,\tau)\right),&w_{\rm
r}(\tau)<\epsilon<w_{\rm M}(\tau).\end{cases},$ (3)

where $\theta(x)$ is the Heaviside step function and, here and below, we
denote the energy level of $\mathbf{k}$ point A as $w_{\rm A}$,

$\displaystyle w_{\rm r}(\tau)$ $\displaystyle=$
$\displaystyle\begin{cases}w_{\Gamma}(\tau),&\tau\leq 1/2,\\\
w_{\varSigma^{\ast}}(\tau),&1/2<\tau.\end{cases}$ (4)
$\displaystyle\zeta(\epsilon,\tau)$ $\displaystyle=$ $\displaystyle
16(1+\tau\epsilon),$ (5) $\displaystyle\varphi(\epsilon,\tau)$
$\displaystyle=$ $\displaystyle(\epsilon+4\tau)^{2},$ (6)
$\displaystyle\psi(\epsilon,\tau)$ $\displaystyle=$
$\displaystyle(4\tau-\epsilon)^{2}-16,$ (7) $\displaystyle w_{\rm
X}(\tau)=-4\tau,$ (8) $\displaystyle w_{\varSigma\ast}(\tau)=-\tau^{-1},$ (9)
$\displaystyle w_{\Gamma,\rm M}(\tau)=\mp 4+4\tau,$ (10)

see $\tau$-dependence of energy levels in Fig. 1a. The function

$F(x,y)=\frac{1}{\sqrt{x}}\mathbb{K}(1-y/x)$ (11)

is symmetric, $F(y,x)=F(x,y)$, singular when $x$ or $y$ tend to zero, and
defined in the domain $x,y>0$, $\mathbb{K}$ being the full elliptic integral
of the first kind.

Thus, an especial important contribution to DOS originates from van Hove
points which possess huge masses, or from merging of van Hove
$\mathbf{k}$-points. The latter is typically connected with enhancement of one
of masses, as it will demonstrated below.

DOS can be presented as the sum of three contributions

$\rho(\epsilon;\tau)=\mathcal{R}_{\psi}(\epsilon,\tau)+\mathcal{R}_{\varphi^{\prime}}(\epsilon,\tau)+\mathcal{R}_{\varphi}(\epsilon,\tau).$
(12)

We obtained the explicit expressions for $\mathcal{R}_{i}$ for the sc lattice

1\. $\tau\leq 1/4$. $\mathcal{R}^{\rm sc}_{\psi}=\mathcal{R}^{\rm
sc}_{\varphi^{\prime}}=0$ and

$\mathcal{R}^{\rm sc}_{\varphi}=\begin{cases}\Phi_{\rm sc}(x^{\rm sc}_{\psi
1},+1),&w^{\rm sc}_{\Gamma}<\epsilon<w^{\rm sc}_{\rm X}(\tau),\\\ \Phi_{\rm
sc}(-1,+1),&w^{\rm sc}_{\rm X}(\tau)<\epsilon<w^{\rm sc}_{\rm M}(\tau),\\\
\Phi_{\rm sc}(-1,x^{\rm sc}_{\psi 2}),&w^{\rm sc}_{\rm
M}(\tau)<\epsilon<w^{\rm sc}_{\rm R}.\end{cases}$ (13)

2\. $1/4<\tau\leq 1/2$.

$\displaystyle\mathcal{R}^{\rm sc}_{\psi}$ $\displaystyle=$
$\displaystyle\begin{cases}2\Psi_{\rm sc}(x^{\rm sc}_{\varphi},+1),&w^{\rm
sc}_{\rm X}<\epsilon<w^{\rm sc}_{\varSigma\ast},\\\ 2\Psi_{\rm sc}(x^{\rm
sc}_{\zeta-},x^{\rm sc}_{\zeta+}),&w^{\rm
sc}_{\varSigma^{\ast}}<\epsilon<w^{\rm sc}_{\varLambda^{\ast}}.\end{cases};$
(14) $\displaystyle\mathcal{R}^{\rm sc}_{\varphi^{\prime}}$ $\displaystyle=$
$\displaystyle\begin{cases}2[\Phi_{\rm sc}(x^{\rm sc}_{\psi 1},x^{\rm
sc}_{\zeta-})+\\\ +\Phi_{\rm sc}(x^{\rm sc}_{\zeta+},+1)],w^{\rm
sc}_{\varSigma^{\ast}}<\epsilon<w^{\rm sc}_{\varLambda^{\ast}},\\\ 2\Phi_{\rm
sc}(x^{\rm sc}_{\psi 1},+1),w^{\rm sc}_{\varLambda^{\ast}}<\epsilon<w^{\rm
sc}_{\Gamma}.\end{cases};$ (15) $\displaystyle\mathcal{R}^{\rm sc}_{\varphi}$
$\displaystyle=$ $\displaystyle\begin{cases}\Phi_{\rm sc}(-1,x^{\rm sc}_{\psi
1}),&w^{\rm sc}_{\rm X}<\epsilon<w^{\rm sc}_{\Gamma},\\\ \Phi_{\rm
sc}(-1,+1),&w^{\rm sc}_{\Gamma}<\epsilon<w^{\rm sc}_{\rm M},\\\ \Phi_{\rm
sc}(-1,x^{\rm sc}_{\psi 2}),&w^{\rm sc}_{\rm M}<\epsilon<w^{\rm sc}_{\rm
R}.\end{cases}.$ (16)

3\. $\tau>1/2$. Kinks of functions $\mathcal{R}^{\rm
sc}_{\varphi},\mathcal{R}^{\rm sc}_{\varphi^{\prime}}$ at $\epsilon=w_{0}^{\rm
sc}(\tau)=4\tau-\tau^{-1}$ cancel each other.

$\displaystyle\mathcal{R}^{\rm sc}_{\psi}$ $\displaystyle=$
$\displaystyle\begin{cases}2\Psi_{\rm sc}\left(x^{\rm
sc}_{\varphi},+1\right),&w^{\rm sc}_{\rm X}<\epsilon<w^{\rm
sc}_{\varSigma^{\ast}},\\\ 2\Psi_{\rm sc}\left(x^{\rm sc}_{\zeta-},x^{\rm
sc}_{\zeta+}\right),&w^{\rm sc}_{\varSigma^{\ast}}<\epsilon<w^{\rm
sc}_{\varLambda^{\ast}}.\end{cases};$ (17) $\displaystyle\mathcal{R}^{\rm
sc}_{\varphi^{\prime}}$ $\displaystyle=$ $\displaystyle\begin{cases}2\Phi_{\rm
sc}\left(x^{\rm sc}_{\zeta+},{\rm min}[x^{\rm sc}_{\psi 2},+1]\right)\\\
+2\Phi_{\rm sc}\left(x^{\rm sc}_{\psi 1},x^{\rm sc}_{\zeta-}\right),w^{\rm
sc}_{\varSigma^{\ast}}<\epsilon<w^{\rm sc}_{\varLambda^{\ast}},\\\ 2\Phi_{\rm
sc}\left(x^{\rm sc}_{\psi 1},{\rm min}[x^{\rm sc}_{\psi 2},+1]\right),w^{\rm
sc}_{\varLambda^{\ast}}<\epsilon<w^{\rm sc}_{0}.\end{cases};$ (18)
$\displaystyle\mathcal{R}^{\rm sc}_{\varphi}$ $\displaystyle=$
$\displaystyle\begin{cases}\Phi_{\rm sc}\left(-1,x^{\rm sc}_{\psi
1}\right),&w^{\rm sc}_{\rm X}<\epsilon<w^{\rm sc}_{\rm M},\\\ \Phi_{\rm
sc}\left(-1,x^{\rm sc}_{\psi 1}\right)\\\ \hskip 14.22636pt+\Phi_{\rm
sc}\left(x^{\rm sc}_{\psi 2},+1\right),&w^{\rm sc}_{\rm M}<\epsilon<w^{\rm
sc}_{0},\\\ \Phi_{\rm sc}\left(-1,x^{\rm sc}_{\psi 2}\right)\\\ \hskip
14.22636pt+\Phi_{\rm sc}\left(x^{\rm sc}_{\psi 1},+1\right),&w^{\rm
sc}_{0}<\epsilon<w^{\rm sc}_{\Gamma},\\\ \Phi_{\rm sc}\left(-1,x^{\rm
sc}_{\psi 2}\right),&w^{\rm sc}_{\Gamma}<\epsilon<w^{\rm sc}_{\rm
R}.\end{cases}.$ (19)

Wihin these equations

$\Psi_{\rm
sc}(x_{1},x_{2};\epsilon,\tau)=\frac{2}{\pi^{3}}\int\limits_{x_{1}}^{x_{2}}\frac{dx\,F(-\zeta_{\rm
sc}(x;\epsilon,\tau),\psi_{\rm sc}(x;\epsilon,\tau))}{\sqrt{1-x^{2}}},$ (20)
$\Phi_{\rm
sc}(x_{1},x_{2};\epsilon,\tau)=\frac{2}{\pi^{3}}\int\limits_{x_{1}}^{x_{2}}\frac{dx\,F(\zeta_{\rm
sc}(x;\epsilon,\tau),\varphi_{\rm sc}(x;\epsilon,\tau))}{\sqrt{1-x^{2}}},$
(21)

where

$\displaystyle\zeta_{\rm sc}(x;\epsilon,\tau)$ $\displaystyle=$ $\displaystyle
16(\tau(\epsilon+2x)+(1-2\tau x)^{2}),$ (22) $\displaystyle\varphi_{\rm
sc}(x;\epsilon,\tau)$ $\displaystyle=$ $\displaystyle(\epsilon+2x+4\tau)^{2},$
(23) $\displaystyle\psi_{\rm sc}(x;\epsilon,\tau)$ $\displaystyle=$
$\displaystyle(\epsilon+2x-4\tau)^{2}-16(1-2\tau x)^{2},$ (24)

$x$ integration bounds have a form

$\displaystyle x^{\rm sc}_{\psi s}(\epsilon,\tau)$ $\displaystyle=$
$\displaystyle\frac{1}{2}\frac{4((-1)^{s}+\tau)-\epsilon}{1+4(-1)^{s}\tau},s=1,2$
(25) $\displaystyle x^{\rm sc}_{\varphi}(\epsilon,\tau)$ $\displaystyle=$
$\displaystyle-2\tau-\epsilon/2,$ (26) $\displaystyle x^{\rm
sc}_{\zeta\pm}(\epsilon,\tau)$ $\displaystyle=$
$\displaystyle\frac{+\frac{1}{2}\pm\sqrt{-\frac{3}{4}-\tau\epsilon}}{2\tau},\epsilon<w^{\rm
sc}_{\varLambda^{\ast}}(\tau).$ (27)

The topology of the spectrum is substantially changed with increasing $\tau$
above the value $\tau_{\ast}^{\rm sc}=1/4$ (at which the $\mathbf{k}$-points
form the whole $\varDelta$ line, the $\Gamma$ point changes its type from
minimum to maximum, whereas the X point changes its type from saddle point to
local minimum), see Fig. 2a and Table 1.

Figure 4: $\tau$-dependence of DOS, $W\rho(\epsilon;\tau)$, at van Hove
singularity energy levels for sc lattice (a), see Table 1 for the notations,
and bcc lattice(b), see Table 2, $W$ being bandwidth. For sc lattice, the
values at $\epsilon=w^{\rm sc}_{\varLambda^{\ast}}$ and $w^{\rm
sc}_{\varSigma^{\ast}}$ almost coincide, see the text.

At $\tau<\tau_{\ast}^{\rm sc}$ there exist only four van Hove points: minimum
at $\Gamma$, maximum at R and two saddle points X, M with opposite signatures
$(++-)$ and $(+--)$. Between corresponding pair of energy levels DOS plot
forms a wide plateau. At $\tau>\tau_{\ast}^{\rm sc}$, two additional van Hove
points $\varSigma^{\ast}$ (direction $\varSigma$) and $\varLambda^{\ast}$
(direction $\varLambda$) are split off from the $\Gamma$ point. Explicit
expressions for these points and levels are presented in Table 1. During
further increase of $\tau$, these points move away from the $\Gamma$ point and
for each of them both three masses diverge like $|\tau-\tau_{\ast}^{\rm
sc}|^{-1}$ as $\tau\rightarrow\tau_{\ast}^{\rm sc}$. Thus, at
$\tau\gtrsim\tau_{\ast}^{\rm sc}$ van Hove line decays into van Hove struture
(three closely positioned points $\Gamma,\varSigma^{\ast}$,
$\varLambda^{\ast}$). The plot of $\rho(\epsilon,\tau)$ at
$\tau>\tau_{\ast}^{\rm sc}$ forms a stable quasi-symmetric plateau between the
energy levels $w^{\rm sc}_{\varSigma^{\ast}},w^{\rm sc}_{\varDelta^{\ast}}$
(the signatures of van Hove points are analogous to ones of the pair X, M at
$\tau<\tau_{\ast}^{\rm sc}$) with the height $\propto(\tau-\tau_{\ast}^{\rm
sc})^{-1/2}$, width at the top $4(\tau-\tau_{\ast}^{\rm sc})^{2}/\tau$ and
universal dependence of DOS on the relative position of $\epsilon$ within the
plateau top (see Fig. 3a,4a). The change of the DOS within the plateau is
rather weak, which is related to simultaneous diverging of all three masses
for both the $\mathbf{k}$-points $\varSigma^{\ast}$ and $\varLambda^{\ast}$.
Maximal DOS value is achieved at $\epsilon=w^{\rm sc}_{\varSigma^{\ast}}$ or
$w^{\rm sc}_{\varLambda^{\ast}}$, corresponding to inner van Hove $\mathbf{k}$
points of the Brillouin zone (see Fig. 4a).

Figure 5: The spectrum for sc (a) and bcc (b) lattices in high symmetry
directions, the van Hove points being shown by vertical dashed line.

At $\tau=\tau_{\ast}^{\rm sc}$ the plateau contribution originating from above
discussed three-point van Hove structure fully vanishes and the main
contribution originates from the remaining background

$\rho_{\rm sc}(-3+\delta\epsilon,\tau_{\ast}^{\rm sc})=\frac{B_{\rm
sc}}{\sqrt[4]{\delta\epsilon}}+o(1),$ (28)

where $B_{\rm
sc}=\frac{\sqrt{2}}{\pi}\frac{\Gamma(5/4)}{\Gamma^{3}(3/4)}=0.222$. Thus, the
giant van Hove singularity is formed originating from valleys of global
minimum positioned on the $\varDelta$ line. An example of electron spectrum
calculation for different $\tau$ in high-symmetry directions of Brillouin zone
is shown in Fig. 5a.

For the bcc lattice we have the following explicit expressions

1\. $\tau\leq 1$.

$\displaystyle\mathcal{R}^{\rm bcc}_{\psi}$ $\displaystyle=$
$\displaystyle\begin{cases}2\Psi_{\rm bcc}(0,x^{\rm bcc}_{\varphi}),&w^{\rm
bcc}_{\rm P}<\epsilon<w^{\rm bcc}_{\varLambda^{\ast}},\\\ 2\Psi_{\rm
bcc}(0,x^{\rm bcc}_{\zeta}),&w^{\rm bcc}_{\varLambda^{\ast}}<\epsilon<w^{\rm
bcc}_{\rm N}.\end{cases},$ (29) $\displaystyle\mathcal{R}^{\rm
bcc}_{\varphi^{\prime}}$ $\displaystyle=$
$\displaystyle\begin{cases}2\Phi_{\rm bcc}(x^{\rm bcc}_{\zeta},x^{\rm
bcc}_{\psi 1}),&w^{\rm bcc}_{\varLambda^{\ast}}<\epsilon<w^{\rm bcc}_{\rm
N},\\\ 2\Phi_{\rm bcc}(0,x^{\rm bcc}_{\psi 1}),&w^{\rm bcc}_{\rm
N}<\epsilon<w^{\rm bcc}_{0}.\end{cases},$ (30) $\displaystyle\mathcal{R}^{\rm
bcc}_{\varphi}$ $\displaystyle=$ $\displaystyle\Phi_{\rm bcc}(x^{\rm
bcc}_{\psi 1},+1).$ (31)

2\. $\tau>1$.

$\displaystyle\mathcal{R}^{\rm bcc}_{\psi}$ $\displaystyle=$
$\displaystyle\begin{cases}2\Psi_{\rm bcc}(0,x^{\rm bcc}_{\varphi}),&w^{\rm
bcc}_{\rm P}<\epsilon<w^{\rm bcc}_{\rm N},\\\ 2\Psi_{\rm bcc}(x^{\rm
bcc}_{\zeta},+1),&w^{\rm bcc}_{\rm N}<\epsilon<w^{\rm
bcc}_{\varDelta^{\ast}}.\end{cases},$ (32) $\displaystyle\mathcal{R}^{\rm
bcc}_{\varphi^{\prime}}$ $\displaystyle=$
$\displaystyle\begin{cases}2\Phi_{\rm bcc}(0,x^{\rm bcc}_{\zeta}),&w^{\rm
bcc}_{\rm N}<\epsilon<w^{\rm bcc}_{\varDelta^{\ast}},\\\ 2\Phi_{\rm
bcc}(0,x^{\rm bcc}_{\psi 1})\\\ \hskip 14.22636pt+2\Phi_{\rm bcc}(x^{\rm
bcc}_{\psi 2},+1),&w^{\rm bcc}_{\varDelta^{\ast}}<\epsilon<w^{\rm
bcc}_{\Gamma}.\end{cases},$ (33) $\displaystyle\mathcal{R}^{\rm
bcc}_{\varphi}$ $\displaystyle=$ $\displaystyle\begin{cases}\Phi_{\rm
bcc}(x^{\rm bcc}_{\psi 1},x^{\rm bcc}_{\psi 2}),&w^{\rm
bcc}_{\varDelta^{\ast}}<\epsilon<w^{\rm bcc}_{\Gamma},\\\ \Phi_{\rm
bcc}(x^{\rm bcc}_{\psi 1},+1),&w^{\rm bcc}_{\Gamma}<\epsilon<w^{\rm bcc}_{\rm
H},\end{cases}$ (34)

where the arguments $\epsilon,\tau$ are omitted for brevity. The kinks of the
functions $\mathcal{R}^{\rm bcc}_{\varphi},\mathcal{R}^{\rm
bcc}_{\varphi^{\prime}}$ at $\epsilon=w_{0}^{\rm bcc}=2\tau$ cancel each
other.

$\Psi_{\rm
bcc}(x_{1},x_{2};\epsilon,\tau)=\frac{2}{\pi^{3}}\int\limits_{x_{1}}^{x_{2}}\frac{dxF(-\zeta_{\rm
bcc}(x;\epsilon,\tau),\psi_{\rm bcc}(x;\epsilon,\tau))}{\sqrt{x(1-x)}},$ (35)
$\Phi_{\rm
bcc}(x_{1},x_{2};\epsilon,\tau)=\frac{2}{\pi^{3}}\int\limits_{x_{1}}^{x_{2}}\frac{dxF(\zeta_{\rm
bcc}(x;\epsilon,\tau),\varphi_{\rm bcc}(x;\epsilon,\tau))}{\sqrt{x(1-x)}},$
(36)

where

$\displaystyle\zeta_{\rm bcc}(x;\epsilon,\tau)$ $\displaystyle=$
$\displaystyle 16(\tau(\epsilon+2\tau)+4(1-\tau^{2})x),$ (37)
$\displaystyle\varphi_{\rm bcc}(x;\epsilon,\tau)$ $\displaystyle=$
$\displaystyle(6\tau+\epsilon-4\tau x)^{2},$ (38) $\displaystyle\psi_{\rm
bcc}(x;\epsilon,\tau)$ $\displaystyle=$ $\displaystyle(\epsilon-2\tau-4\tau
x)^{2}-64x,$ (39)

and $x$ integration bounds reads

$\displaystyle x^{\rm bcc}_{\varphi}(\epsilon,\tau)$ $\displaystyle=$
$\displaystyle\frac{\epsilon+6\tau}{4\tau},w^{\rm bcc}_{\rm P}<\epsilon,$ (40)
$\displaystyle x^{\rm bcc}_{\zeta}(\epsilon,\tau)$ $\displaystyle=$
$\displaystyle\frac{\tau(\epsilon+2\tau)}{4(\tau^{2}-1)},(\epsilon-w^{\rm
bcc}_{\zeta})(\tau-1)>0,$ (41) $\displaystyle x^{\rm bcc}_{\psi
1,2}(\epsilon,\tau)$ $\displaystyle=$ $\displaystyle\frac{\left(\mp
2+\sqrt{\tau(\epsilon-2\tau)+4}\right)^{2}}{4\tau^{2}},\epsilon>w^{\rm
bcc}_{\varDelta^{\ast}}.$ (42)

The topology of the spectrum is changed at $\tau^{\rm bcc}_{\ast}=1$. At
$\tau<\tau^{\rm bcc}_{\ast}$ there are two van Hove $\mathbf{k}$-points of
local minimum, $\Gamma$ and P, the van Hove $\mathbf{k}$-point H of local
maximum and two saddle $\mathbf{k}$-points $\varLambda^{\ast}$ and N with
opposite signatures, see Table 2. At $\tau=0$ the point $\varLambda^{\ast}$
coincides with P, and the levels $w^{\rm bcc}_{\rm N},w^{\rm bcc}_{\rm P}$
merge, so that the van Hove line $D$ is formed which produces the DOS
singularity at the center of the band, $\rho_{\rm
bcc}(\epsilon,\tau=0)=(4/\pi^{3})[\ln^{2}(64/|\epsilon|)-\pi^{2}/16]+o(\epsilon)$
1971:Katsura . The cause of such energy dependence is linear vanishing of
inverse transverse mass at the $D$ line. At $0<\tau<\tau^{\rm bcc}_{\ast}$ the
van Hove saddle $\mathbf{k}$-points $\varLambda^{\ast}$ and N have opposite
signatures, energy levels $w^{\rm bcc}_{\rm N},w^{\rm
bcc}_{\varLambda^{\ast}}$ being boundaries of asymmetric plateau (for N point,
only one mass diverges as $\tau\rightarrow 0,1$, whereas for
$\varLambda^{\ast}$ three masses diverge and the right edge of plateau has a
much larger DOS value), see Fig. 3b. As $\tau$ increases van Hove point
$\varLambda^{\ast}$ migrates from P point to $\Gamma$ point along $\varLambda$
line. When $\tau$ approaches $\tau^{\rm bcc}_{\ast}$, the width of the plateau
vanishes and the energy levels $w^{\rm bcc}_{\rm N}$ and $w^{\rm
bcc}_{\Gamma}$ merge, i. e. the van Hove line $\varSigma$ is formed.

In the case $\tau>\tau^{\rm bcc}_{\ast}$ the N point signature changes (see
Table 2), and the role of $\varLambda^{\ast}$ point belongs to the
$\varDelta^{\ast}$ point which has the same signature, splits off from the
$\Gamma$ point at $\tau>\tau^{\rm bcc}_{\ast}$ and migrates along $\varDelta$
direction moving away from the point $\Gamma$. Similar to $\varLambda^{\ast}$
point at $\tau<\tau^{\rm bcc}_{\ast}$, for $\varDelta^{\ast}$ point all three
masses diverge at $\tau\rightarrow\tau^{\rm bcc}_{\ast}$. This implies that at
$\tau>\tau^{\rm bcc}_{\ast}$ an asymmetric plateau in $\rho_{\rm
bcc}(\epsilon,\tau)$ plot is formed between the energy levels $w^{\rm
bcc}_{\rm N},w^{\rm bcc}_{\varDelta^{\ast}}$ $\rho_{\rm bcc}(w^{\rm
bcc}_{\varDelta^{\ast}},\tau)$ being much larger than $\rho_{\rm bcc}(w^{\rm
bcc}_{\rm N},\tau)$. The energy level $w^{\rm bcc}_{\Gamma}$ is close to
$w^{\rm bcc}_{\varDelta^{\ast}}$, so that DOS value at $\epsilon=w^{\rm
bcc}_{\Gamma}$ competes with a value at $\epsilon=w^{\rm bcc}_{\rm N}$, see.
Fig. 4b. The maximal DOS value is achieved at $\epsilon=w^{\rm
bcc}_{\varLambda^{\ast}}$ for $\tau<\tau^{\rm bcc}_{\ast}$ and at
$\epsilon=w^{\rm bcc}_{\varDelta^{\ast}}$ for $\tau>\tau^{\rm bcc}_{\ast}$,
both corresponding to inner $\mathbf{k}$-points of the Brillouin zone.

One can see the formation of spectrum extremums when $\tau$ deviates from
values corresponding to van Hove lines formation ($\tau=0$ and $\tau^{\rm
bcc}_{\ast}$), see Fig. 5b. At $\tau=\tau_{\ast}^{\rm bcc}$ DOS has the
asymptotics

$\rho_{\rm
bcc}(\epsilon=-2+\delta\epsilon,1)=-\frac{5}{2\pi^{2}}+\begin{cases}\frac{B^{<}_{\rm
bcc}}{\sqrt[4]{-\delta\epsilon}},&\delta\epsilon<0,\\\ \frac{B^{>}_{\rm
bcc}}{\sqrt[4]{\delta\epsilon}},&\delta\epsilon>0.\end{cases}+o(1),$ (43)

where

$\displaystyle B^{>}_{\rm bcc}$ $\displaystyle=$
$\displaystyle\frac{3}{2\pi^{3}}\int\limits_{+1}^{+\infty}\frac{dt\mathbb{K}(1-t^{2})}{\sqrt{t}}=0.33260,$
(44) $\displaystyle B^{<}_{\rm bcc}$ $\displaystyle=$
$\displaystyle\frac{1}{\pi^{3}}\int\limits_{0}^{+\infty}\frac{dt}{\sqrt{t}}\mathbb{K}(-t^{2})=0.31358.$
(45)

##  III Impact on thermodynamic properties

As an example of van Hove singularity impact on thermodynamic properties, we
consider the temperature dependence of specific heat. For free electrons, this
quantity (per site and spin projection) reads

$C(T)=-T^{-1}\int d\epsilon\rho(\epsilon)(\epsilon-E_{\rm
F})^{2}f^{\prime}(\epsilon),$ (46)

$f(\epsilon)$ being the Fermi function. When the Fermi level is well away from
all peculiar DOS levels, at low temperature $C_{\rm reg}(T)\approx\gamma_{\rm
reg}T$, where $\gamma_{\rm reg}=\pi^{2}\rho(E_{\rm F})/6=\text{const}$.
However, due to the presence of van Hove singularity, $\gamma(T)=C(T)/T$ can
considerably deviate from constant.

We consider the case where the Fermi level $E_{\rm F}$ lies near the levels of
van Hove structures for the sc lattice at $\tau>\tau^{\rm sc}_{\ast}$ (Fig.
6a) and bcc lattice at $\tau>\tau^{\rm bcc}_{\ast}$ (Fig. 6b). We choose the
Fermi level position both at edges of the plateau and at its center.

Figure 6:  Temperature dependences of specific heat $C$ (left axis, solid
line) and $\gamma=C/T$ (right axis, dashed line) for (a) sc lattice,
$\tau=0.4$, the Fermi level being at quasi-symmetric $w_{\Sigma^{\ast}}$ –
$w_{\Lambda^{\ast}}$ plateau:
$E_{F}=w_{\Sigma^{\ast}},w_{\Lambda^{\ast}},w_{\rm
c}=(1/2)(w_{\Sigma^{\ast}}+w_{\Lambda^{\ast}})$; (b) bcc lattice, $\tau=1.1$,
the Fermi level being at non-symmetric $w_{\Delta^{\ast}}$ – $w_{\Gamma}$
structure: $w_{\Delta^{\ast}},w_{\Gamma},w_{\rm
c}=(1/2)(w_{\Delta^{\ast}}+w_{\Gamma})$. Lines of different colors correspond
to different positions of $E_{\rm F}$; the regular case $E_{\rm F}=0$ is shown
for comparison.

For the sc lattice, when $E_{\rm F}$ is on the top of the
$\varSigma^{\ast}-\varLambda^{\ast}$ plateau and $\tau$ is rather close to
$\tau_{\ast}^{\rm sc}$, the DOS value is large and practically insensitive to
the change of $E_{\rm F}$ within the plateau. The position of $E_{\rm F}$ at
the center of the plateau invokes the linear behavior of $C(T)$ at low
temperature below some scale of order a plateau width (of order of
$16|\tau-\tau_{\ast}^{\rm sc}|^{2}$), so that $\gamma_{\rm sc}(T)$ is large
and almost constant, but rapidly falls down when the temperature increases
above this scale. When $E_{\rm F}$ is at the left or right edge of the plateau
the temperature behaviour is almost the same for both the cases, which
manifests the symmetric form of the peak structure. At $\tau<\tau^{\rm
sc}_{\ast}$ there are no peculiarities in the whole energy range.

This implies that, unlike the case of weak ferromagnetism owing to the
vicinity of large-mass $\mathbf{k}$-point, the ferromagnetism produced by the
plateau is much more stable, possesses large Curie temperature and is expected
to reproduce classical local moments picture of ferromagnetic ordering Moriya
, see also 2015:Igoshev .

As the energy comes through the plateau boundaries, DOS decreases according a
square-root law, so that the coefficient $A$ at the square root (see
discussion above) is proportional to $(\tau-\tau_{\ast}^{\rm sc})^{-3/2}$.

For the bcc lattice at $\tau>\tau^{\rm bcc}_{\ast}$, the van Hove structure
peak is formed by the close levels $\epsilon=w^{\rm bcc}_{\varDelta^{\ast}}$
and $w^{\rm bcc}_{\Gamma^{\ast}}$ corresponding to saddle ($\varDelta^{\ast}$)
and maximum ($\Gamma$) $\mathbf{k}$-points. In contrast to the sc lattice,
this peak is typically more narrow and asymmetric (the value of DOS at
$\epsilon=w^{\rm bcc}_{\varDelta^{\ast}}$ prevails). The narrowness of the
peak causes merging $C(T)$ plots at different position of $E_{\rm F}$ as $T$
overcomes the energy scale of the order of peak width (of order of
$4|\tau-\tau_{\ast}^{\rm bcc}|^{2}$). For $E_{\rm F}=w^{\rm
bcc}_{\varDelta^{\ast}}$, $\gamma_{\rm bcc}$ rapidly falls down with
increasing $T$ as temperature is below this energy scale. As the temperature
overcomes the scale, the falling of $\gamma_{\rm bcc}(T)$ retains, but becomes
weaker. As $E_{\rm F}$ deviates from $w^{\rm bcc}_{\varDelta^{\ast}}$, the
dependence $\gamma_{\rm bcc}(T)$ becomes non-monotonous at low temperatures.
At $\tau<\tau^{\rm bcc}_{\ast}$ the level $\epsilon=w^{\rm
bcc}_{\varLambda^{\ast}}$ plays the role of $w^{\rm bcc}_{\varDelta^{\ast}}$
at $\tau>\tau^{\rm bcc}_{\ast}$.

We see that the temperature behaviour of the contributions from the van Hove
singularities to thermodynamic properties is somewhat different for the sc and
bcc lattices.

The strong asymmetry of DOS with respect to $E_{F}$ can yield giant values of
thermoelectric power when the Fermi level lies near the plateau boundaries,
but beyond it.

##  IV Conclusions

To conclude, we performed a systematic investigation of the effect of van Hove
singularities on the spectrum of one-particle excitation and density of states
for bipartite cubic lattices. We also considered in detail the dependence of
spectrum topology and DOS behaviour on the next-nearest neighbour hopping
integral.

For non-bipartite fcc lattice an analytic investigation of van Hove energy
levels and numerical analysis of the DOS within the nearest and the next-
nearest neighbour hopping approximation were performed only in a limited
interval $\tau\in(-1,0)$ 1972:Swendsen  which did not allow to find van Hove
singularity lines. However, at special value $\tau^{\rm fcc}_{\ast}=1/2$ a
giant van Hove line occurs at the bottom of the band $w^{\rm fcc}_{\rm b}$
producing DOS singularity of the type $1/\sqrt{\epsilon-w^{\rm fcc}_{\rm b}}$
1998:Ulmke . The corresponding analysis of stability of this singularity and
its influence on thermodynamic properties will be considered elsewhere.

We are grateful to M.I. Katsnelson and A.O. Anokhin for useful discussion. The
research was carried out within the state assignment of Ministry of Science
and Education of Russia (theme ¡¡Quantum¿¿ No. AAAA-A18-118020190095-4) and
with the support by Program 211 of the Government of the Russian Federation
(Agreement 02.A03.21.0006).

## References

  * (1) R.J. Jelitto. The Density of States of Some Simple Excitations in Solids, J. Phys. Chem. Solids 30, 609 (1969).
  * (2) R.H. Swendsen and H. Callen. Green’s Functions of the Face-Centered-Cubic Heisenberg Ferromagnet with Second-Neighbor Interactions, Phys. Rev. B 6, 2860 (1972).
  * (3) S. V. Vonsovskii, M. I. Katsnelson, and A. V. Trefilov, Localized and Itinerant Behavior of Electrons in Metals, Phys. Met. Metallogr. 76, 247 (1993).
  * (4) M.I. Katsnel’son, G.V. Peschanskikh, A.V. Trefilov. The Singularities in the Density of Electron States and Their Effect on the Elasticity Moduli in Alkaline Earth Metals, Sov. Phys. Solid State 32, 272 (1990).
  * (5) V.L. Moruzzy, J.P. Janak, A.R. Williams. Calculated Electronic Properties of Metals, N.Y.: Plenum Press (1978).
  * (6) D.A. Papacostantopoulos. Handbook of Band Structure of Elemental Solids, N.Y.: Plenum Press (1986).
  * (7) A. Hausoel, M. Karolak, E. Sasioglu, A. Lichtenstein, K. Held, A. Katanin, A. Toschi, G. Sangiovanni. Local magnetic moments in iron and nickel at ambient and Earth’s core conditions, Nature Communications 8, 16062 (2017).
  * (8) G. Santi, S. B. Dugdale, and T. Jarlborg. Longitudinal Spin Fluctuations and Superconductivity in Ferromagnetic ZrZn2 from Ab Initio Calculations, Phys. Rev. Lett. 87, 247004 (2001)
  * (9) A. S. Hamid, A. Uedono, Zs. Major, T. D. Haynes, J. Laverock, M. A. Alam, S. B. Dugdale, and D. Fort. Electronic structure and Fermi surface of the weak ferromagnet Ni3Al, Phys. Rev. B 84, 235107 (2011).
  * (10) P. G. Niklowitz, F. Beckers, G. G. Lonzarich, G. Knebel, B. Salce, J. Thomasson, N. Bernhoeft, D. Braithwaite, and J. Flouquet. Spin-fluctuation-dominated electrical transport of Ni3Al at high pressure, Phys. Rev. B 72, 024424 (2005).
  * (11) J. Inoue. Electronic structure and magnetism of Y–M (M=Mn, Fe, Co and Ni), Physica B 149, 376 (1988); Y. Nishihara and S. Ogawa. Itinerant Electron Ferromagnetism in Y2Ni7, J. Phys. Soc. Jpn. 60, 300 (1991).
  * (12) D.J. Singh. Electronic structure and weak itinerant magnetism in metallic Y2Ni7, Phys. Rev. B 92, 174403 (2015).
  * (13) M. I. Katsnel’son, A. V. Trefilov. Electronic phase transitions due to correlation effects, JETP Letters 40, 1092 (1984).
  * (14) M.I. Katsnelson, A.V. Trefilov. Anomalies in properties of metals and alloys due to electron correlations, Phys. Lett. 109A, 109 (1985).
  * (15) M.I. Katsnelson, A.V. Trefilov. Anomalies of electronic and lattice properties of metals and alloys, caused by screening anomalies, Physica B 163, 182 (1990); Anomalies caused in phonon spectra by charge density fluctuations, JETP Letters 42, 393 (1985).
  * (16) V.Yu. Irkhin, M.I. Katsnelson, and A.V. Trefilov. On the magnetic contribution to lattice properties of itinerant ferromagnets: the role of van Hove electron spectrum singularities, J. Magn. Mag. Mater. 117, 210 (1992).
  * (17) T. Morita and T. Horiguchi. Calculation of the Lattice Green’s Function for the bcc, fcc, and Rectangular Lattices, J. Math. Phys. 12, 986 (1971).
  * (18) S. Katsura and T. Horiguchi. Lattice Green’s Function for the Body-Centered Cubic Lattice, J. Math. Phys. 12, 230 (1971).
  * (19) Yu. A. Izyumov. Strongly correlated electrons: the $t-J$ model, Phys. Usp. 40, 445 (1997).
  * (20) A. Damascelli, Z. Hussain, Z.-X. Shen. Angle-resolved photoemission studies of the cuprate superconductors, Rev. Mod. Phys. 75, 473 (2003).
  * (21) M. A. Timirgazin, P. A. Igoshev, A. K. Arzhnikov, V. Yu. Irkhin. Metal–Insulator Transition in the Hubbard Model: Correlations and Spiral Magnetic Structures, J. Low. Temp. Phys. 185, 651 (2016).
  * (22) M. A. Timirgazin, P. A. Igoshev, A. K. Arzhnikov, V. Yu. Irkhin. Magnetic States, Correlation Effects and Metal-Insulator Transition in fcc Lattice, J. Phys.: Condens. Matter 28, 505601 (2016).
  * (23) P.A. Igoshev and V.Yu. Irkhin. Metal–Insulator Transition in the Presence of van Hove Singularities for Bipartite Lattices, J. Exp. Theor. Phys. 128, 909 (2019).
  * (24) P.E. Blöchl, O. Jepsen, and O.K. Andersen. Improved tetrahedron method for Brillouin-zone integrations, Phys. Rev. B 49, 16223 (1994).
  * (25) A.A. Stepanenko, D.O. Volkova, P.A. Igoshev, A.A. Katanin. Kohn anomalies in momentum dependence of magnetic susceptibility of some three-dimensional systems, J. Exp. Theor. Phys. 125, 879 (2017).
  * (26) L. van Hove. The Occurrence of Singularities in the Elastic Frequency Distribution of a Crystal, Phys. Rev. 89, 1189 (1953).
  * (27) M. Fleck, A.M. Oleś, and L. Hedin. Magnetic phases near the van Hove singularity in $s$\- and $d$-band Hubbard models, Phys. Rev. B 56, 3159 (1997).
  * (28) P.A. Igoshev, E.E. Kokorina, and I.A. Nekrasov. Investigation of the Magnetocaloric Effect Correlated Metallic Systems with van Hove Singularities in the Electron Spectrum, Phys. Met. and Metallogr. 118, 207 (2017).
  * (29) T. Moriya. Spin Fluctuations in Itinerant Electron Magnetism. Springer-Verlag, Berlin-Heidelberg, 1985.
  * (30) P.A. Igoshev, A.V. Efremov, and A. A. Katanin. Magnetic exchange in $\alpha$-iron from ab initio calculations in the paramagnetic phase, Phys. Rev. B 91, 195123 (2015).
  * (31) M. Ulmke. Ferromagnetism in the Hubbard model on fcc-type lattices, Eur. Phys. J. B 1, 301 (1998).


# Towards Leveraging AutoML for Sustainable Deep Learning: A Multi-Objective
HPO Approach on Deep Shift Neural Networks

Leona Hennig, Tanja Tornede, Marius Lindauer
Institute of Artificial Intelligence, Leibniz University Hannover, Germany
{l.hennig, t.tornede<EMAIL_ADDRESS>
###### Abstract

Deep Learning (DL) has advanced various fields by extracting complex patterns
from large datasets. However, the computational demands of DL models pose
environmental and resource challenges. Deep shift neural networks (DSNNs)
offer a solution by leveraging shift operations to reduce computational
complexity at inference. Following the insights from standard DNNs, we are
interested in leveraging the full potential of DSNNs by means of AutoML
techniques. We study the impact of hyperparameter optimization (HPO) to
maximize DSNN performance while minimizing resource consumption. Since this
combines multi-objective (MO) optimization with accuracy and energy
consumption as potentially complementary objectives, we propose to combine
state-of-the-art multi-fidelity (MF) HPO with multi-objective optimization.
Experimental results demonstrate the effectiveness of our approach, resulting
in models with over 80% in accuracy and low computational cost. Overall, our
method accelerates efficient model development while enabling sustainable AI
applications.

##  1 Introduction

Deep Learning (DL) is considered one of the most promising approaches to
extracting information from large data collections with complex structures.
This includes performing computations in IoT environments and on edge devices
(Li et al., 2018; Zhou et al., 2019). With the ever-increasing size and
performance of such models due to the progress in science and industry, a
computational cost is associated with them (Sze et al., 2017). Minimizing this
cost directly affects a model’s environmental impact (Schwartz et al., 2020).
Thus, resources are freed up and can be used for other tasks, e.g., those with
resource limitations (Howard et al., 2017). With our approach, we contribute
to DL in such low-resource settings.

Deep shift neural networks (DSNNs) offer great potential in reducing power
consumption compared to traditional deep learning models (Elhoushi et al.,
2021). Instead of floating point arithmetic, they leverage shift operations —
specifically, bit shifting — as the computational unit, which boosts
efficiency by replacing costly multiplication operations in convolutional
networks. We suspect that the configuration of DSNNs has a huge impact on both
performance and computational efficiency.

One of the key challenges with DSNNs is determining the appropriate level of
precision for the shift operations to minimize quantization errors without
excessively increasing the computational load. In this study, we leverage
DSNNs in combination with automated machine learning (AutoML) to find the
optimal configuration of DSNNs in the spirit of Green AutoML (Tornede et al.,
2023). This is achieved by hyperparameter optimization (HPO) using the
framework SMAC3 proposed by Lindauer et al. (2022), which automates the search
for optimal model configurations. Integrating multi-fidelity (MF) and multi-
objective optimization (MO) techniques (Belakaria et al., 2020) facilitates an
optimal exploration of the hyperparameter space that jointly prioritizes
performance and energy consumption (Deb, 2014). SMAC’s MO implementation
effectively balances the trade-off between achieving high predictive accuracy
and minimizing energy consumption. The MF aspect allows for the efficient use
of computational resources by evaluating configurations at varying levels of
detail. Employing tools like CodeCarbon (Lacoste et al., 2019; Lottick et al.,
2019) during the training and evaluation phases provides real-time insights
into the energy consumption and carbon emissions associated with each model
configuration.

Overall, we make the following contributions:

  1. 1.

A configuration space specific for DSNNs,

  2. 2.

a Green AutoML approach for efficiency-driven model building,

  3. 3.

insights into the design decisions for obtaining optimal tradeoffs between
accuracy and energy efficiency, and

  4. 4.

combining MO and MF approaches in SMAC to improve optimization performance and
computational resource utilization.

##  2 Background

The following chapter introduces foundational concepts for our approach.

###  2.1 Deep Shift Neural Networks

Deep shift neural networks (DSNNs) are a novel approach to reducing the
computational and energy demands of deep learning (Elhoushi et al., 2021).
They achieve a considerable reduction in latency time by simplifying the
network architecture such that they replace the traditional multiplication
operations in neural networks by bit-wise shift operations and sign flipping,
making DSNNs suitable for computing devices with limited resources. There are
two methods for training DSNNs (Elhoushi et al., 2021): DeepShift-Q
(Quantization) and DeepShift-PS (Powers of two and Sign). DeepShift-Q involves
training regular weights constrained to powers of 2 by quantizing weights to
their nearest power of two during both forward and backward passes. DeepShift-
PS directly includes the values of the shifts and sign flips as trainable
parameters.

The DeepShift-Q approach obtains the sign matrix $S$ from the trained weight
matrix $W$ as $S=\mathit{sign}(W)$. The power matrix $P$ is the base-2
logarithm of $W$’s absolute values, i.e., $P=\log_{2}(|W|)$. After rounding
$P$ to the nearest power of two, $P_{\mathit{r}}=\mathit{round}(P)$, the
quantized weights $\tilde{W}_{q}$ are calculated by applying the sign from
$S$, shown as $\tilde{W}_{q}=\mathit{flip}(2^{P_{\mathit{r}}},S)$. The
DeepShift-PS approach optimizes neural network weights by directly adapting
the shift ($\tilde{P}$) and sign ($\tilde{S}$) values. The shift matrix
$\tilde{P}$ is obtained by rounding the base-2 logarithm of the weight values,
$\tilde{P}=\mathit{round}(P)$, and the sign flip $\tilde{S}$ is computed as
$\tilde{S}=\mathit{sign}(\mathit{round}(S))$. Weights are calculated as
$\tilde{W}_{\mathit{ps}}=\mathit{flip}(2^{\tilde{P}},\tilde{S})$, where the
sign flip operation $\tilde{S}$ assigns values of $-1$, $0$, or $+1$ based on
$s$.

###  2.2 Hyperparameter Optimization

The increasing complexity of deep learning algorithms enhances the need for
automated hyperparameter optimization (HPO) to increase model performance
(Bischl et al., 2023). Consider a dataset
$\mathcal{D}=\\{(x_{i},y_{i})\\}_{i=1}^{N}\in\mathbb{D}\subset\mathcal{X}\times\mathcal{Y}$,
where $\mathcal{X}$ is the instance space and $\mathcal{Y}$ is the target
space, and a hyperparameter configuration space
$\Lambda=\\{\lambda_{1},\ldots,\lambda_{L}\\}$, $L\in\mathbb{N}$. In our
study, $\mathcal{M}$ denotes the space of possible DSNN models. An algorithm
$\mathcal{A}:\mathbb{D}\times\Lambda\rightarrow\mathcal{M}$ trains a model
$M\in\mathcal{M}$, instantiated with a configuration of $L$ hyperparameters
sampled from $\Lambda$, on a training subset of $\mathcal{D}$. The dataset
$\mathcal{D}$ is split into training, validation, and test sets:
$\mathcal{D}_{\textit{train}},\mathcal{D}_{\textit{val}},$ and
$\mathcal{D}_{\textit{test}}$ respectively. The performance of the algorithm
is assessed via an expensive-to-evaluate loss function
$\mathcal{L}:\mathcal{M}\times\mathbb{D}\rightarrow\mathbb{R}$. The
optimization objective of HPO is to find the configuration
$\lambda^{*}\in\Lambda$ with minimal validation loss $\mathcal{L}$, such that:

$\lambda^{*}\in\operatorname*{arg\,min}_{\lambda\in\Lambda}\mathcal{L}\big{(}\mathcal{A}(\mathcal{D}_{\textit{train}},\lambda),\mathcal{D}_{\textit{val}}\big{)}.$
(2.1)

This process tunes hyperparameters based on validation set performance. Models
are trained on $\mathcal{D}_{\textit{train}}$ and optimized using
$\mathcal{D}_{\textit{val}}$. Finally, the model’s performance is assessed on
$\mathcal{D}_{\textit{test}}$.

###  2.3 Bayesian Optimization

Bayesian Optimization (BO) is a strategy for global optimization of black-box
loss functions
$\mathcal{L}:\mathcal{M}\times\mathbb{D}\xrightarrow{}\mathbb{R}$ that are
expensive to evaluate (Jones et al., 1998). BO uses a surrogate model
$\mathcal{S}$, a probabilistic model to approximate the loss function,
commonly given by a Gaussian Process or a Random Forest (Rasmussen & Williams,
2006; Hutter et al., 2011). An acquisition function
$\mathcal{C}:\Lambda\xrightarrow{}\mathbb{R}$ guides the search for the next
optimal evaluation points by balancing the exploration-exploitation trade-off,
based on the set of previously queried points
$\\{(\lambda_{1},\mathcal{L}_{1}),...,(\lambda_{m-1},\mathcal{L}_{m-1})\\}$ at
time $m$. $\mathcal{L}$ is only evaluated at certain points, with which the
surrogate model is updated.

###  2.4 Multi-Fidelity Optimization

Since it is not feasible to fully train multiple configurations of DSNNs for
comparison due to computational efficiency, we employ a multi-fidelity (MF)
approach (Li et al., 2017), which is a common strategy in AutoML to navigate
the trade-off between performance and approximation error (Hutter et al.,
2019). MF approaches train cheap-to-evaluate proxies of black-box functions
following different heuristics, e.g., allocating a small number of epochs to
many configurations in the beginning and training the best-performing ones on
an increasing number of epochs. Formally, we define a space of fidelities
$\mathcal{F}$ and aim to minimize a high-fidelity function $F\in\mathcal{F}$
(Kandasamy et al., 2019):

$\min_{\lambda\in\Lambda}F(\lambda)\,.$ (2.2)

We aim to approximate $F\in\mathcal{F}$, using a series of lower-fidelity,
less expensive approximations $\\{f(\lambda)_{1},\ldots,f(\lambda)_{j}\\}$,
where $j$ denotes the total number of fidelity levels. The allocated resources
for evaluating a model’s performance at various fidelities are referred to as
a budget. MF assumes that the highest fidelities approximate the black-box
function best.

###  2.5 Multi-Objective Optimization

Multi-Objective Optimization (MO) addresses problems involving multiple, often
competing objectives. This approach is used in scenarios where trade-offs
between two or more conflicting objectives must be navigated, such as, in the
context of DSNNs, enhancing accuracy alongside reducing energy consumption. MO
aims to identify Pareto-optimal solutions (Deb, 2014). To ensure effective
approximation of the Pareto front, new points are added based on the current
observation dataset
$\mathcal{D}_{\mathit{obs}}=\\{(\lambda_{1},\mathcal{L}(\lambda_{1})),\ldots,(\lambda_{n},\mathcal{L}(\lambda_{n}))\\}$
at time $n$. These points augment the surface formed by the non-dominated
solution set $D^{\star}_{n}$, which satisfies the condition for $d$ objective
variables and a loss function
$\mathcal{L}=(\mathcal{L}_{1},\ldots,\mathcal{L}_{d})$ (Wada & Hino, 2019):
For all
$\lambda,(\lambda,\mathcal{L}(\lambda))\in\mathcal{D}^{\star}_{n}\subset\mathcal{D}_{n}$
and
$\lambda^{\prime},(\lambda^{\prime},\mathcal{L}(\lambda^{\prime}))\in\mathcal{D}_{n}$
exists $k\in\\{1,\ldots,d\\}$, such that
$\mathcal{L}_{k}(\lambda)\leq\mathcal{L}_{k}(\lambda^{\prime})$.

##  3 Approach

To computationally enhance Deep shift neural networks (DSNN) via AutoML, we
employ multi-fidelity optimization (MF). We introduce different fidelities to
the training process by increasing the number of shift layers in the model.
The initially trained models will include fewer shift layers, which will
increase during the optimization process. We assume this will guide the search
toward the highest-performing shift networks since a low number of shift
layers introduces the least numerical inaccuracy and mathematical uncertainty
to the model. We aim to investigate whether picking the best-performing models
under low model-intrinsic uncertainty continues to show improved performance
when combined with more shift layers.

In Section 2, we explained the MF approach. One algorithm that formalizes its
heuristics is successive halving (Jamieson & Talwalkar, 2016), where $n_{c}$
configurations are trained on an initial small budget $b_{I}$. After that, the
$\nu/(\nu+1)$ best-performing configurations are trained on a budget of $\nu
b_{I}/(\nu+1)$ until the best configuration is determined. We address the
trade-off between $b_{I}$ and $n_{c}$, or between approximation error and
exploration inherent in successive halving, using the HyperBand algorithm for
MF. HyperBand (Li et al., 2017) runs successive halving in multiple brackets,
where each bracket provides a combination of $n_{c}$ and a fraction of the
total budget per configuration so that they sum up to the total budget.

We extend this to multi-fidelity multi-objective optimization (MFMO). We
simultaneously address the accuracy of the model as well as its energy
consumption. The goal is to optimize the performance of the DSNNs, ensuring
they achieve high accuracy and robustness in their predictive capabilities;
secondly, to minimize energy consumption, a critical factor given the
environmental implications of computational efficiency. To this end, we
implement a two-dimensional objective function
$f_{MO}:\Lambda\xrightarrow{}\mathbb{R}^{2},\quad
f_{M0}(\lambda)=\big{(}f_{loss}(\lambda),f_{emission}(\lambda)\big{)}$, where,
given a configuration $\lambda\in\Lambda$, $f_{loss}(\lambda)$ aims to
minimize the loss, enhancing the model’s accuracy, and $f_{emission}(\lambda)$
seeks to minimize the energy consumption during training and inference,
promoting environmental sustainability. We aim to solve the following
optimization problem:

$\operatorname*{arg\,min}_{\lambda\in\Lambda}f_{M0}(\lambda)\,.$ (3.1)

SMAC3 uses a mean aggregation strategy by calculating the objective’s
arithmetic mean to aggregate multiple objectives into a single scalar value
for MF optimization. We use the ParEGO algorithm (Knowles, 2006) to compute
Pareto optimal objectives. It transforms the multi-objective problem into a
series of single-objective problems by introducing varying weight parameters
for the objectives in each iteration of HyperBand. Thus optimizing a different
scalarization per evaluation to approximate the Pareto front. The resulting
single-objective optimization function can now be evaluated in an MF setting.

##  4 Experiments

In the following section, we detail the setup and methodology used to evaluate
our approach discussed in Section 3, focusing on optimizing deep shift neural
networks (DSNNs) through multi-fidelity, multi-objective optimization (MFMO)
as well as multi-fidelity optimization (MF). We discuss how our approach
successfully navigates the model performance and environmental impact trade-
offs.

###  4.1 Evaluation Setup

We train and evaluate our models on the Cifar10 dataset (Krizhevsky et al.,
2009), using NVIDIA A100 GPUs. For hyperparameter optimization (HPO), we rely
on using SMAC3 (Lindauer et al., 2022). To incorporate the environmental
impact into our HPO workflow, we use the CodeCarbon emissions tracker (Lacoste
et al. (2019), Lottick et al. (2019)) to track carbon emissions from
computational processes by monitoring energy use and regional energy mix in
$g\mathit{CO}_{2}\mathit{eq}$, grams of $\mathit{CO}_{2}$ equivalent (Tornede
et al., 2023). We chose a pre-trained ResNet20 (He et al., 2016).

###  4.2 Results

Figure 1: Test accuracy of the default DSNN and a configured DSNN

Figure 1 displays the Top-1 test accuracy of a DSNN trained with the default
configuration from Table 1 as defined by Elhoushi et al. (2021), as well as
the test accuracy from the configured DSNN according to SMAC3 with MF using
HyperBand, with the number of shift layers inducing the fidelities. The
Pareto-optimal Solution No. 1 configuration from Table 1 is used. The MF
algorithm, trained on the same seed as MFMO, identifies it as the optimal
model configuration out of fifty evaluated configurations. While volatile to a
certain degree, the plot clearly shows that the model’s performance with the
optimal configuration surpasses that of the model instantiated with the
default configuration by at least three percent. This verifies our approach
that MF can successfully be executed with DSNN-specific budgets such as the
shift depth.

Furthermore, it confirms our assumption from Section 3 that the models can be
evaluated under low model-intrinsic uncertainty and continue to perform well
when instantiated with more shift layers.

Table 1 shows two Pareto optimal solutions out of 33 evaluated configurations
with our MFMO approach. Evaluating solution No. 1 results in a Top-1 accuracy
of 83.50% and 0.1661 $g\mathit{CO}_{2}\mathit{eq}$. Solution No. 2 yields a
Top-1 accuracy of 84,67%, producing 0.1673 $g\mathit{CO}_{2}\mathit{eq}$. Note
that they were trained for fewer epochs due to computational limitations. Both
configurations yield good performance results while keeping emissions low.

The DSNN-specific model hyperparameters include activation integer bits,
activation fraction bits, weight bits, and shift depth. Solution No.1 includes
fewer shift layers with fewer bits representing the weights activation
function values. Activation bits and weight bits seem to affect emissions and
model performance similarly. Fewer shift layers increase the computational
demand of a model. However, the reduced number of bits used to replace
floating-point operations (FLOPs) seems to compensate for this since solution
No. 1 produces approximately the same amount of emissions as solution No. 2.
This one contains more than three times the number of shift layers, using only
four convolutional layers with FLOPs, but uses substantially more
representation bits. There seems to be a trade-off between the number of shift
layers and the number of representation bits that, when navigated efficiently,
yields model configurations with satisfying performance and energy
consumption. This supports our claim that our MFMO approach is promising for
building high-performance, energy-efficient DSNNs.

Parameter |  Search Space |  Default |  POS 1 |  POS 2
---|---|---|---|---
Batch Size |  [32, 128] |  128 |  97 |  102
Optimizer |  SGD, Adam, Adagrad, Adadelta, RMSProp, RAdam, Ranger |  SGD |  Ranger |  SGD
Learning Rate |  [0.001, 0.1] |  0.1 |  0.0847 |  0.0798
Momentum |  [0.0, 0.9] |  0.9 |  0.5016 |  0.6738
Epochs |  [5, 100] |  200 |  67 |  91
Weight Bits |  [2, 8] |  5 |  4 |  7
Activation Integer Bits |  [2, 32] |  16 |  4 |  29
Activation Fraction Bits |  [2, 32] |  16 |  4 |  10
Shift Depth |  [0, 20] |  20 |  5 |  16
Shift Type |  Q, PS |  PS |  Q |  Q
Rounding |  deterministic, stochastic |  deterministic |  stochastic |  deterministic
Weight Decay |  [1e-6, 1e-2] |  0.0001 |  0.0026 |  0.00012
Accuracy (in %) |  |  |  83.50 |  84.67
Emissions (in $g\mathit{CO}_{2}\mathit{eq}$) |  |  |  0.1661 |  0.1673
Table 1: MFMO findings, incl. two Pareto optimal solutions (POS 1 & 2)

##  5 Conclusion

In this work, we present our Green AutoML approach towards the sustainable
optimization of DSNNs through a multi-fidelity, multi-objective (MFMO) HPO
framework. We address the critical intersection between advancing the
capabilities of Deep Learning and environmental sustainability. By leveraging
AutoML tools and integrating the environmental impact as an objective, we
navigate the trade-off between model performance and efficient resource
utilization.

Our experimental results highlight the potential of our approach. We
successfully optimized a DSNN to achieve high accuracy while minimizing energy
consumption. We have delivered a thorough configuration space for DSNNs,
introduced a Green AutoML approach for efficiency-driven model development,
and provided valuable insights into the design decisions. Future work includes
extending our approach to multiple benchmarks and neural network architectures
to sufficiently verify our approach on a broad spectrum of model designs and
applications. This way, we hope to gain more insight into critical design
decisions for DSNNs and other models built for energy-efficient computations.
Revisiting our MFMO implementation to find a more efficient way for ParEGO and
Hyperband to intertwine, e.g. by finding a more effective way to assign
budgets and weights to configurations, is subject to further work to lighten
the computational load when computing the MFMO Pareto fronts. Furthermore, we
will investigate more DSNN-specific fidelity types and MO algorithms to
achieve further reductions in model emissions.

#### Acknowledgments

This work was supported by the German Federal Ministry of the Environment,
Nature Conservation, Nuclear Safety and Consumer Protection (GreenAutoML4FAS
project no. 67KI32007A).

## References

  * Belakaria et al. (2020) S. Belakaria, A. Deshwal, and J. Doppa.  Multi-Fidelity Multi-Objective bayesian optimization: An output space entropy search approach.  _AAAI_ , 34(06):10035–10043, 2020.
  * Bischl et al. (2023) B. Bischl, M. Binder, M. Lang, T. Pielok, J. Richter, S. Coors, J. Thomas, T. Ullmann, M. Becker, A. Boulesteix, D. Deng, and M. Lindauer.  Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges.  _WIREs Data. Mining. Knowl. Discov._ , 13(2), 2023.
  * Deb (2014) K. Deb.  _Multi-objective Optimization_ , pp. 403–449.  Springer US, 2014.
  * Elhoushi et al. (2021) M. Elhoushi, Z. Chen, F. Shafiq, Y. Henry Tian, and J. Yiwei Li.  DeepShift: Towards multiplication-less neural networks.  In _IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2021, virtual, June 19-25, 2021_ , pp. 2359–2368, 2021.
  * He et al. (2016) K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In _2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016_ , pp. 770–778, 2016.
  * Howard et al. (2017) A. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam.  MobileNets: Efficient convolutional neural networks for mobile vision applications.  _arXiv:1704.04861_ , 2017.
  * Hutter et al. (2011) F. Hutter, H. Hoos, and K. Leyton-Brown.  Sequential model-based optimization for general algorithm configuration.  In C. Coello Coello (ed.), _Learning and Intelligent Optimization - 5th International Conference, LION 5, Rome, Italy, January 17-21, 2011. Selected Papers_ , volume 6683, pp. 507–523, 2011.
  * Hutter et al. (2019) F. Hutter, L. Kotthoff, and J. Vanschoren (eds.).  _Automated Machine Learning - Methods, Systems, Challenges_.  Springer Publishing Company, Incorporated, 2019.
  * Jamieson & Talwalkar (2016) K. Jamieson and A. Talwalkar.  Non-stochastic best arm identification and Hyperparameter Optimization.  In A. Gretton and C. Robert (eds.), _Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics (AISTATS’16)_ , volume 51. Proceedings of Machine Learning Research, 2016.
  * Jones et al. (1998) D. Jones, M. Schonlau, and W. Welch.  Efficient global optimization of expensive black-box functions.  _J. Glob. Optim._ , 13(4):455–492, 1998.
  * Kandasamy et al. (2019) K. Kandasamy, G. Dasarathy, J. Oliva, J. Schneider, and B. Póczos.  Multi-fidelity gaussian process bandit optimisation.  _J. Artif. Intell. Res._ , 66:151–196, 2019.
  * Knowles (2006) J. Knowles.  ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems.  _IEEE Trans. Evol. Comput._ , 10(1):50–66, 2006.
  * Krizhevsky et al. (2009) A. Krizhevsky, G. Hinton, and o.  Learning multiple layers of features from tiny images.  _University of Toronto_ , 2009.
  * Lacoste et al. (2019) A. Lacoste, A. Luccioni, V. Schmidt, and T. Dandres.  Quantifying the carbon emissions of machine learning.  _arXiv:1910.09700_ , 2019.
  * Li et al. (2018) H. Li, K. Ota, and M. Dong.  Learning IoT in edge: Deep learning for the internet of things with edge computing.  _IEEE Netw._ , 32(1):96–101, 2018.
  * Li et al. (2017) L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar.  Hyperband: A novel bandit-based approach to hyperparameter optimization.  _J. Mach. Learn. Res._ , 18:185:1–185:52, 2017.
  * Lindauer et al. (2022) M. Lindauer, K. Eggensperger, M. Feurer, A. Biedenkapp, D. Deng, C. Benjamins, T. Ruhkopf, R. Sass, and F. Hutter.  SMAC3: A versatile bayesian optimization package for hyperparameter optimization.  _J. Mach. Learn. Res._ , 23:54:1–54:9, 2022.
  * Lottick et al. (2019) K. Lottick, S. Susai, S. Friedler, and J. Wilson.  Energy usage reports: Environmental awareness as part of algorithmic accountability.  _arXiv:1911.08354_ , 2019.
  * Rasmussen & Williams (2006) C. Edward Rasmussen and C. Williams.  _Gaussian Processes for Machine Learning_.  The MIT Press, 2006.
  * Schwartz et al. (2020) R. Schwartz, J. Dodge, N. Smith, and O. Etzioni.  Green AI.  _Commun. ACM_ , 63(12):54–63, 2020.
  * Sze et al. (2017) V. Sze, Y. Chen, T. Yang, and J. Emer.  Efficient processing of deep neural networks: A tutorial and survey.  _Proc. IEEE_ , 105(12):2295–2329, 2017.
  * Tornede et al. (2023) T. Tornede, A. Tornede, J. Hanselle, F. Mohr, M. Wever, and E. Hüllermeier.  Towards green automated machine learning: Status quo and future directions.  _J. Artif. Intell. Res._ , 77:427–457, 2023.
  * Wada & Hino (2019) T. Wada and H. Hino.  Bayesian optimization for multi-objective optimization and multi-point search.  _arXiv:1905.02370_ , 2019.
  * Zhou et al. (2019) Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang.  Edge intelligence: Paving the last mile of artificial intelligence with edge computing.  _Proc. IEEE_ , 107(8):1738–1762, 2019.