††thanks: C. X. and H. Y. contributed equally to this manuscript.††thanks: C.
X. and H. Y. contributed equally to this manuscript.

# Three-nodal surface phonons in solid-state materials: Theory and material
realization

Chengwu Xie  School of Physical Science and Technology, Southwest University,
Chongqing 400715, China;  Hongkuan Yuan  School of Physical Science and
Technology, Southwest University, Chongqing 400715, China;  Ying Liu  ying
<EMAIL_ADDRESS>(Y. L.); School of Materials Science and Engineering, Hebei
University of Technology, Tianjin 300130, China;  Xiaotian Wang
<EMAIL_ADDRESS>(X. W.); School of Physical Science and Technology,
Southwest University, Chongqing 400715, China;  Gang Zhang
<EMAIL_ADDRESS>(G. Z.); Institute of High Performance Computing,
Agency for Science, Technology and Research (A*STAR), 138632, Singapore

###### Abstract

This year, Liu et al. [Phys. Rev. B 104, L041405 (2021)] proposed a new class
of topological phonons (TPs; i.e., one-nodal surface (NS) phonons), which
provides an effective route for realizing one-NSs in phonon systems. In this
work, based on first-principles calculations and symmetry analysis, we
extended the types of NS phonons from one- to three-NS phonons. The existence
of three-NS phonons (with NS states on the $k_{i}$ = $\pi$ ($i$ = $x$, $y$,
$z$) planes in the three-dimensional Brillouin zone (BZ)) is enforced by the
combination of two-fold screw symmetry and time reversal symmetry. We screened
all 230 space groups (SGs) and found nine candidate groups (with the SG
numbers (Nos.) 19, 61, 62, 92, 96, 198, 205, 212, and 213) hosting three-NS
phonons. Interestingly, with the help of first-principles calculations, we
identified $P2_{1}$2121-type YCuS2 (SG No. 19), $Pbca$-type NiAs2 (SG No. 61),
$Pnma$-type SrZrO2 (SG No. 62), $P4_{1}$212-type LiAlO2 (SG No. 92),
$P4_{3}$212-type ZnP2 (SG No. 96), $P2_{1}$3-type NiSbSe (SG No. 198),
$Pa\bar{3}$-type As2Pt (SG No. 205), $P4_{3}$32-type BaSi2 (SG No. 212), and
$P4_{1}$32-type CsBe2F5 (SG No. 213) as realistic materials hosting three-NS
phonons. The results of our presented study enrich the class of NS states in
phonon systems and provide concrete guidance for searching for three-NS
phonons and singular Weyl point phonons in realistic materials.

##  I Introduction

Topological quantum states of matter add1 ; add2 ; add3  are an important
topic in the field of modern condensed-matter physics. Over the past 15 years,
we have witnessed the emergence of many types of topological electronic
materials, such as topological insulators add4 ; add5 ; add6 ; add7 ,
topological crystalline insulators add8 ; add9 ; add10 , topological Kondo
insulators add11 ; add12 ; add13 , higher-order topological insulators add14 ;
add15 ; add16 ; add17 , topological semimetals add18 ; add19 ; add20 , and
higher-order topological semimetals add21 ; add22 ; add23 ; add24 . In
particular, the types and numbers of topological semimetals add20  are rapidly
increasing. In contrast to Dirac, Weyl, and Majorana fermions, which are
allowed in high-energy physics, the types of quasiparticles in topological
semimetals add25  are more diverse owing to fewer constraints imposed by the
space group (SG) symmetries of the crystal. Based on the dimensionality of the
band-crossings in the momentum space, the topological semimetals can be
classified into nodal point add26 ; add27 ; add28 ; add29 ; add30 , nodal line
add31 ; add32 ; add33 ; add34 ; add35 , and nodal surface (NS) add36 ; add37 ;
add38 ; add39 ; add40  semimetals with zero-, one-, and two-dimensional band-
crossings, respectively.

Three-dimensional topological semimetals with two-dimensional band-crossings
can host NS states in the Brillouin zone (BZ). Each point on the NS should be
a two-fold degenerate point with linear band dispersion along the surface
normal direction. Researchers hope that NS semimetals exhibit exotic physical
properties, such as stronger quantum oscillations and peculiar plasmon
excitations. Wu et al. add36  summarized an essential NS state dictated by
nonsymmorphic symmetry without spin-orbit coupling (SOC). The existence of a
series of NS semimetals in realistic electronic systems has been predicted,
including BaVS3 add40 , ZrSiS add41 ; add42 , K6YO4 add36 , FeB4 add43 , Ti3Al
add37 , and X(MoS)3 (X = K, Rb, and Cs) add44 . However, in general, SOC in
electronic materials cannot be ignored; thus, the proposed two-dimensional
nonsymmorphic symmetry-enforced NS states in electronic systems will usually
be destroyed or reduced to one-dimensional nodal lines when SOC is considered
add42 . Moreover, the NS states in some materials are far from the Fermi level
and exhibit large energy variations, which hinder their experimental
detection.
The proposed topological phonons (TPs) add45 ; add46  have renewed the
interest in topological quantum states; TPs are a basic kind of boson-type
quasiparticles; they are not affected by the Pauli exclusion principle and
SOC. Therefore, TPs can normally be observed in spinless phononic systems in
all frequency ranges. In addition to the proposed nodal point phonons add47 ;
add48 ; add49 ; add50 ; add51 ; add52 ; add53 ; add54 ; add55  and nodal line
phonons add56 ; add57 ; add58 ; add59 ; add60 ; add61 ; add62 ; add63 ; add64
, one-NS phonons add65  have been presented by Liu et al. based on symmetry
analysis and first-principles calculations. The researchers provided a
complete list of the one-NS phonons in the 230 SGs and discovered that RbTeAu
family materials with SG number (No.) 51 may contain one-NS states (on the
$k_{x}$ = $\pi$ plane). The occurrence of one-NS states is ensured by screw
rotation symmetry along the $i$ axis ($i$ = $x$, $y$, or $z$) and time-
reversal symmetry $\mathcal{T}$. Fig. 1(a) presents a schematic diagram of
one-NS phonons. Moreover, two more types of NS phonons should exist: two- and
three-NS phonons, as illustrated in Fig. 1(b) and Fig. 1(c), respectively.
In this study, we extended the class of NS phonons from one- to three-NS
phonons. For the three-NS phonons, the NS states are localized on the $k_{i}$
= $\pi$ ($i$ = $x$, $y$, $z$) planes in the three-dimensional BZ. We screened
all 230 SGs; the SGs with Nos. 19, 61, 62, 92, 96, 198, 205, 212, and 213 are
candidate groups that can obtain three-NS phonons. Because the three-NS
phonons in these SGs are symmetry-enforced, one can easily achieve three-NS
phonons in realistic materials with the previously presented SGs. For example,
in this work, we identified $P2_{1}$2121-type YCuS2 (SG No. 19), $Pbca$-type
NiAs2 (SG No. 61), $Pnma$-type SrZrO2 (SG No. 62), $P4_{1}$212-type LiAlO2 (SG
No. 92), $P4_{3}$212-type ZnP2 (SG No. 96), $P2_{1}$3-type NiSbSe (SG No.
198), $Pa\bar{3}$-type As2Pt (SG No. 205), $P4_{3}$32-type BaSi2 (SG No. 212),
and $P4_{1}$32-type CsBe2F5 (SG No. 213) as realistic materials that can host
three-NS phonons.

Figure 1: Schematic diagrams of (a) one-NS, (b) two-NS, (c) and three-NS
phonons, respectively.

##  II Symmetry Analysis of Three-NS Phonons.

In this part, we searched all essential NSs, which are only dictated by
symmetries, in spinless systems add36 . Such an NS is protected by the
combination of time-reversal symmetry ($\mathcal{T}$) and two-fold screw
rotation symmetry ($S_{2i}$).

Without loss of generalization, we take two-fold screw rotation along the
z-direction as an example: $S_{2z}$:$(x,y,z)\to(-x,-y,z+\frac{1}{2})$ with a
half translation in the lattice constant along its rotation axis. It also
affects the momentum space:
$S_{2z}$:$(k_{x},k_{y},k_{z})\to(-k_{x},-k_{y},k_{z})$, thereby only
preserving the momentum along $k_{z}$. Without SOC,
$S_{2z}^{2}$=$T_{100}$=$e^{-ik_{z}}$, where $T_{100}$ is the translation along
the z-direction. For time-reversal symmetry, in spinless systems,
$\mathcal{T}^{2}$ = 1, which is antiunitary and inverses the momentum k.
Consequently, their combination $\mathcal{T}S_{2z}$ is also antiunitary.
Remarkably, on planes where $k_{z}$ = $\pm\pi$,
$(\mathcal{T}S_{2z})^{2}$=$e^{-ik_{z}}|_{k_{z}=\pm\pi}$ = $-1$, which suggests
Kramer-like degeneracy on these planes. Thereby, it leads to Kramer-like
degeneracy. Hence, the phonon bands on the $k_{i}$ = $\pi$ ($i$ = $x$, $y$,
$z$) planes must become two-fold degenerate, thereby forming three-NS phonons.
Furthermore, the presence of three two-fold rotation symmetries (i.e.,
$S_{2x}$, $S_{2y}$, and $S_{2z}$) leads to three NSs on the planes $k_{i}$ =
$\pm\pi$ ($i$ = $x$, $y$, $z$). In this study, we proposed all the three-NS
phonons by searching all 230 SGs in phonon systems. According to the results,
the SGs with Nos. 19, 61, 62, 92, 96, 198, 205, 212, and 213 (see Table 1) can
host three-NS phonons.

##  III Computational Details

First-principles calculations based on density functional theory were
performed to study the ground states of $P2_{1}$2121-type YCuS2, $Pbca$-type
NiAs2, $Pnma$-type SrZrO2, $P4_{1}$212-type LiAlO2, $P4_{3}$212-type ZnP2,
$P2_{1}$3-type NiSbSe, $Pa\bar{3}$-type As2Pt, $P4_{3}$32-type BaSi2, and
$P4_{1}$32-type CsBe2F5 materials, as implemented in the Vienna Ab Initio
Simulation Package. The projector augmented wave method and generalized
gradient approximation add66  with Perdew–Burke–Ernzerhof functions were used
for the ionic potential and exchange-correlation interaction. In addition, a
plane wave cutoff energy of 500 eV was used for the structural relaxation. The
following $k$-mesh samples were used for YCuS2, NiAs2, SrZrO2, LiAlO2, ZnP2,
NiSbSe, As2Pt, BaSi2, and CsBe2F5: $9\times 7\times 5$, $5\times 5\times 3$,
$5\times 5\times 5$, $7\times 7\times 7$, $7\times 7\times 3$, $7\times
7\times 7$, $7\times 7\times 7$, $9\times 9\times 9$, and $5\times 5\times 5$,
respectively. All these materials are experimentally synthesized materials.
The phononic dispersions of the $2\times 2\times 1$ YCuS2, $2\times 2\times 1$
NiAs2, $2\times 1\times 2$ SrZrO2, $2\times 2\times 2$ LiAlO2, $2\times
2\times 1$ ZnP2, $2\times 2\times 2$ NiSbSe, $2\times 2\times 2$ As2Pt,
$2\times 2\times 2$ BaSi2, and $1\times 1\times 1$ CsBe2F5 cells were examined
with density functional perturbation theory and PHONOPY codes add67 .

Table 1: A complete list of three-NS phonons in 230 SGs. The first and second columns present the SG numbers and SG symbols, the third column lists the three-NSs along the symmetry paths, and the fourth column presents the corresponding realistic materials. Space group | Space group | Three-nodal | Realistic
---|---|---|---
numbers | symbols | surfaces | materials
19 |  $P2_{1}$2121 |  NSTYS, NSSXU, and NSUZT |  YCuS2
61 | $Pbca$ |  NSTYS, NSSXU, and NSUZT |  NiAs2
62 | $Pnma$ |  NSTYS, NSSXU, and NSUZT |  SrZrO2
92 |  $P4_{1}$212 |  NSZRA, and NSMXR |  LiAlO2
96 |  $P4_{3}$212 |  NSZRA, and NSMXR |  ZnP2
198 |  $P2_{1}$3 |  NSRXM | NiSbSe
205 | $Pa\bar{3}$ |  NSRXM |  As2Pt
212 |  $P4_{3}$32 |  NSRXM |  BaSi2
213 |  $P4_{1}$32 |  NSRXM |  CsBe2F5

##  IV Materials with Three-NS phonons

For phonon systems with SG Nos. 19, 61, 62, the three-NSs (i.e., NSTYS, NSSXU,
and NSUZT) appear on the planes $k_{y}$ = $\pi$, $k_{x}$ = $\pi$, and $k_{z}$
= $\pi$, respectively. Some realistic materials were selected as examples to
demonstrate that they host three-NSs in their phonon dispersions:
$P2_{1}$2121-type YCuS2 (SG No. 19) can be prepared add68  by fusing the high-
purity elements in evacuated quartz ampoules; Murray and Heyding add69
prepared $Pbca$-type NiAs2 (SG No. 61) by heating the elements in sealed
evacuated Vycor tubes; $Pnma$-type SrZrO2 powders (SG No. 62) were prepared
with the polymeric precursor method by Cavalcante et al. add70 . The crystal
structures of these three materials are shown in Fig. 2. They are completely
relaxed; their theoretically determined lattice constants and the previously
published experimentally determined data are listed in Table 2.

Figure 2: Crystal structures of $P2_{1}$2121-type YCuS2 (SG No. 19), $Pbca$-type NiAs2 (SG No. 61), and $Pnma$-type SrZrO2 (SG No. 62), respectively.  Table 2: Theoretically and experimentally determined lattice constants of YCuS2, NiAs2, and SrZrO2. Materials | Theoretical lattice | Experimental lattice
---|---|---
| constants |  constants add68 ; add69 ; add70
YCuS2 | a = 3.96 Å, b = 6.26 Å, | a = 3.97 Å, b = 6.27 Å,
| c = 13.48 Å | c = 13.38 Å
NiAs2 | a = 5.80 Å, b = 5.89 Å, | a = 5.77 Å, b = 5.83 Å,
| c = 11.50 Å | c = 11.41 Å
SrZrO2 | a = 5.91 Å, b = 8.29 Å, | a = 5.81 Å, b = 8.19 Å,
| c = 5.84 Å | c = 5.79 Å
Figure 3: (a) Three-dimensional BZ and symmetry points. Three-NS states (red,
green, and yellow) are localized on the $k_{i}$ = $\pi$ ($i$ = $x$, $y$, $z$)
planes in three-dimensional BZ. (b)–(d) Calculated phonon dispersions of
YCuS2, NiAs2, and SrZrO2, respectively. Three NS regions (i.e., NSTYS, NSSXU,
and NSUZT) are highlighted in green, red, and yellow, respectively.  Figure 4:
(a), (c), and (e) Enlarged phonon dispersions of three regions (see Fig.
3(b)–(d)) of YCuS2, NiAs2, and SrZrO2, respectively. (b), (d), and (f) phonon
dispersions along $a$ (0.70, 0.25, 0.00) -$P$ (0.50, 0.25, 0.00) -$a^{\prime}$
(0.30, 0.25, 0.00), $b$ (0.00, 0.25, 0.7) -$Q$ (0.00, 0.25, 0.5) -$b^{\prime}$
(0.00, 0.25, 0.3), and $c$ (0.00, 0.7, 0.25) -$N$ (0.00, 0.5, 0.25)
-$c^{\prime}$ (0.00, 0.3, 0.25), respectively. All points at $P$, $Q$, and $N$
points are two-fold degenerate with linear phonon band dispersions.

The phonon dispersions of YCuS2, NiAs2, and SrZrO2 along the $\Gamma-X-
S-Y-\Gamma-Z-S-X-U-Z-T-Y-S-R$ paths (see Fig. 3(a)) are shown in Fig.
3(b)–(d), respectively. Three regions (highlighted in red, yellow, and green)
are of interests in this study. The enlarged figures of the phonon dispersions
of YCuS2, NiAs2, and SrZrO2 in the three regions are shown in Fig. 4(a), (c)
and (e), respectively. All the phonon bands along the $S$-$X$-$U$,
$U$-$Z$-$T$, and $T$-$Y$-$S$ planes have two-fold degeneracy. To explain this
in more detail, some symmetry lines (i.e., $a$-$P$-$a^{\prime}$,
$b$-$Q$-$b^{\prime}$, and $c$-$N$-$c^{\prime}$; see Fig. 3(a)) are selected,
they are perpendicular to the $S$-$X$, $T$-$Z$, and $Y$-$T$ symmetry lines,
respectively. Subsequently, we calculate the phonon dispersions along the
$a$-$P$-$a^{\prime}$, $b$-$Q$-$b^{\prime}$, and $c$-$N$-$c^{\prime}$ paths;
the results are presented in Fig. 4(b), (d), (f), respectively. Evidently, the
points (highlighted by red circles) at the $P$, $Q$, and $N$ symmetry points
are two-fold degenerate and have linear band dispersions. In addition, two-
fold Kramer-like degeneracy occurs at every point on the $S$-$X$-$U$,
$U$-$Z$-$T$, and $T$-$Y$-$S$ planes, thereby forming three-NSs on the $k_{i}$
= $\pi$ ($i$ = $x$, $y$, $z$) planes. These density functional theory results
agree well with the argument (Section II) that the antiunitary symmetry
$\mathcal{T}S_{2i}$ ensures the existence of three-NS phonons on the $k_{i}$ =
$\pm\pi$ ($i$ = $x$, $y$, $z$) planes.

Figure 5: (a) and (b) Crystal structures of $P4_{1}$212-type LiAlO2 (SG No.
92) and $P4_{3}$212-type ZnP2 (SG No. 96), respectively; (c) Three-dimensional
BZ and symmetry points. Three-NS states (highlighted in yellow and green) are
localized on $k_{i}$ = $\pi$ ($i$ = $x$, $y$, $z$) planes in three-dimensional
BZ. (b)–(d) Calculated phonon dispersions of LiAlO2 and ZnP2, respectively. NS
regions (i.e., NSMXR, and NSZRA) are highlighted in green and yellow,
respectively.

In the next step, some realistic materials with SG Nos. 92 and 96 and three-NS
phonons are presented. The first example is $P4_{1}$212-type LiAlO2 (SG No.
92); Remeika and Ballman add71  prepared these single crystals from a flux.
The second example is $P4_{3}$212-type ZnP2 (SG No. 96). Researchers add72
have reported that ZnP2 crystals can exist in an enantiomorphic form with SG
$P4_{3}$212 = $D_{4}^{8}$. The theoretically determined and previously
published experimental lattice constants are shown in Table 3. The crystal
structures of the two materials are shown in Fig. 5(a) and (b). The phonon
dispersions of the two materials along the $\Gamma-X-M-\Gamma-Z-R-A-M-X-R$
paths (see Fig. 5(c)) are presented in Fig. 5(d) and (e). To examine the
three-NS phonons in these two materials, we only focused on two paths:
$Z$-$R$-$A$ and $M$-$X$-$R$, which are highlighted in yellow and green in Fig.
5(d) and (e), respectively. The enlarged phonon dispersions of these two
regions for LiAlO2 and ZnP2 are shown in Fig. 6(a) and (c), respectively. All
the phonon bands along the $Z$-$R$-$A$ and $M$-$X$-$R$ paths are two-fold
degenerate. To present examples, we select two symmetry points $N$ and $Q$ on
the $k_{y}$ = $\pi$ and $k_{z}$ = $\pi$ planes, respectively. We construct the
two paths $b$-$Q$-$b^{\prime}$ and $c$-$N$-$c^{\prime}$, which vertically pass
through the $k_{z}$ = $\pi$ and $k_{y}$ = $\pi$ planes, respectively. The
obtained phonon dispersions along the $b$-$Q$-$b^{\prime}$ and
$c$-$N$-$c^{\prime}$ paths for LiAlO2 and ZnP2 are shown in Fig. 6(b) and (d),
respectively. There are two two-fold degenerate points at $Q$ and $N$, which
represent the NS states on the $k_{z}$ = $\pi$ and $k_{y}=\pi$ planes. Because
LiAlO2 and ZnP2 with SG Nos. 92 and 96 host four-fold screw rotation, $S_{4z}$
= $\\{C_{4z}|00\frac{1}{2}\\}$, there should be NSs on the $k_{x}$ = $\pm\pi$
planes.

Table 3: Theoretically and experimentally determined lattice constants of LiAlO2 and ZnP2. Materials | Theoretical lattice | Experimental lattice
---|---|---
| constants |  constants add71 ; add72
LiAlO2 | a = b = 5.21 Å, | a = b = 5.17 Å,
| c = 6.30 Å | c = 6.59 Å
ZnP2 | a = b = 5.06 Å, | a = b = 5.10 Å,
| c = 18.53 Å | c = 18.62 Å
Figure 6: (a), (c) Enlarged phonon dispersions of two regions (see Fig. 5(d)
and (e)) of LiAlO2 and ZnP2, respectively. (b), (d) Phonon dispersions along
$b$ (0.00, 0.25, 0.7) -$N$ (0.00, 0.25, 0.5) -$b^{\prime}$ (0.00, 0.25, 0.3),
and $c$ (0.00, 0.7, 0.25) -$Q$ (0.00, 0.5, 0.25) -$c^{\prime}$ (0.00, 0.3,
0.25), respectively. All the points at the $Q$ and $N$ points are two-fold
degenerate points with linear phonon band dispersions.

Finally, some realistic materials with SG Nos. 198, 205, 212, and 213 are
presented, which host three-NS phonons, i.e., NSRXM. The first example is
$P2_{1}$3-type NiSbSe (SG No. 198). It was prepared by letting powders of
binary nickel chalcogenides react with the respective pnictogen component in
evacuated sealed silica tubes add73 . The second example is $Pa\bar{3}$-type
As2Pt (SG No. 205). Ramsdell add74  produced artificial PtAs2, which is
identical to natural sperrylite. The third example is $P4_{3}$32-type BaSi2
(SG No. 212). This compound is an interesting material add75  that can host
three types of polymorphs (orthorhombic, trigonal, and cubic crystal classes
with $Pnma$, $P\bar{3}m1$, and $P4_{3}32$ SGs) at up to 40 kbar and 1000
${}^{\circ}C$. $P4_{3}$32-type BaSi2 represents one kind of the high-pressure
phases. The fourth example is $P4_{1}$32-type CsBe2F5 (SG No. 213). Le Fur and
Al$\rm{\acute{e}}$onard add76  dissolved Cs2CO2 carbonate in a hydrofluoric
solution containing excess BeF2. A single CsBe2F5 crystal can be obtained via
evaporation at 55 ${}^{\circ}C$. The crystal structures of these materials are
shown in Fig. 7. We determined their lattice constants with structural-
relaxation calculations (see Fig. 7 and Table 4).

Figure 7: (a)–(d) Crystal structures of $P2_{1}$3-type NiSbSe (SG No. 198), $Pa\bar{3}$-type As2Pt (SG No. 205), $P4_{3}$32-type BaSi2 (SG No. 212), and $P4_{1}$32-type CsBe2F5 (SG No. 213), respectively.  Table 4: Theoretically and experimentally determined lattice constants of NiSbSe, As2Pt, BaSi2, and CsBe2F5 Materials | Theoretical lattice | Experimental lattice
---|---|---
| constants |  constants add73 ; add74 ; add75 ; add76
NiSbSe | a = b = c = 6.13 Å | a = b = c = 6.08 Å
As2Pt | a = b = c = 6.06 Å | a = b = c = 5.92 Å
BaSi2 | a = b = c = 6.77 Å | a = b = c = 6.71 Å
CsBe2F5 | a = b = c = 8.06 Å | a = b = c = 7.93 Å
Figure 8: (a) Three-dimensional BZ and symmetry points. Three-NS states (green
color) are localized on $k_{i}$ = $\pi$ ($i$ = $x$, $y$, $z$) planes in three-
dimensional BZ. (b)–(e) Calculated phonon dispersions of NiSbSe, As2Pt, BaSi2,
and CsBe2F5 materials, respectively. NS regions (i.e., NSRXM) are highlighted
in green.

We calculated the phonon dispersions of NiSbSe, As2Pt, BaSi2, and CsBe2F5
along the symmetry paths $\Gamma-X-M-\Gamma-R-X-M$ (see Fig. 8(a)); the
results are shown in Fig. 8(b)–(d). Let us focus on the two-fold degenerate
phonon bands along the $R$-$X$-$M$ paths (see Fig. 9(a), (c), (e), and (g)).
To prove that these bands are degenerated, we chose the path
$a$-$P$-$a^{\prime}$ that vertically passes through the $k_{y}$ = $\pi$ plane.
The obtained phonon dispersions along the $a$-$P$-$a^{\prime}$ path for these
materials are shown in Fig. 9(b), (d), (f), and (h), respectively. There are
two evident two-fold degenerate points at $P$ with linear band dispersions. We
can conclude that an NS phonon exists on the $k_{y}$ = $\pi$ plane on which
the two low-energy phonon bands cross linearly. Owing to $C_{3,111}$ symmetry,
equivalent NS phonons can be found on the $k_{x}$ = $\pi$ and $k_{y}$ = $\pi$
planes. We would like to point out that although symmetry requires the
occurrence of three-NS phonons and limits the possible positions on the
$k_{i}$ = $\pi$ ($i$ = $x$, $y$, $z$) planes, it does not limit the
frequencies and dispersions of three-NS phonons.

Figure 9: (a), (c), (e), (g) Enlarged phonon dispersions of $R$-$X$-$M$ path
(see Fig. 8(b)–(e)) of As2Pt, BaSi2, and CsBe2F5 materials, respectively. (b),
(d), (f), (h) Phonon dispersions along $a$ (0.00, 0.70, 0.25) -$P$ (0.00,
0.50, 0.25) -$a^{\prime}$ (0.00, 0.30, 0.25) path. All points at the $P$
symmetry point are two-fold degenerate points with linear phonon band
dispersions (indicated by red circles in (b), (d), (f), and (h)).

##  V Summary and remarks

In conclusion, according to the symmetry analysis results, there are three-NS
phonons in the SGs with SG Nos. 19, 61, 62, 92, 96, 198, 205, 212, and 213 of
the 230 SGs. More interestingly, by performing first-principles calculations,
we discovered that the realistic materials $P2_{1}$2121-type YCuS2 (SG No.
19), $Pbca$-type NiAs2 (SG No. 61), $Pnma$-type SrZrO2 (SG No. 62),
$P4_{1}$212-type LiAlO2 (SG No. 92), $P4_{3}$212-type ZnP2 (SG No. 96),
$P2_{1}$3-type NiSbSe (SG No. 198), $Pa\bar{3}$-type As2Pt (SG No. 205),
$P4_{3}$32-type BaSi2 (SG No. 212), and $P4_{1}$32-type CsBe2F5 (SG No. 213)
include three-NS phonons in their phonon dispersions.

We present the following remarks: (i) Because phonons obey Bose–Einstein
statistics and are not limited by the Fermi energy, three-NS in the phonon
system may be more common in realistic materials; (ii) Unlike fermions in
electronic systems with heavy elements, SOC can be neglected for TPs in phonon
systems. Hence, three-NS phonons in phonon systems can be considered real NS
states without SOC-induced gaps; (iii) Although three-NS phonons in SGs 19,
61, 62, 92, 96, 198, 205, 212, and 213 can be determined by the combination of
two-fold screw symmetry and time reversal symmetry, the frequencies and
dispersions of three-NS phonons are not limited; (iv) One may ask what is the
difference between three-NS and one-/two-NS states? As we know, constrained by
the no-go theorem, among all the Weyl semimetals add77  discovered in
experiments before 2019, Weyl points always occur in pairs in the momentum
space, without exception. Interestingly, in 2019, as demonstrated by Yu et al.
add78 , the three-NS state is a good platform for realizing a singular Weyl
point by circumventing the no-go theorem. However, for two- and one-NS states,
although Weyl points and NS states can coexist, there must be more than one
Weyl point in the BZ. Fortunately, in this year, Ma et al. add77  observed a
singular Weyl point surrounded by three-NSs in PtGa with SG No. 198 in an
experiment.

Acknowledgments X.T.W. thanks Prof. Zhi-Ming Yu for his help regarding to this
manuscript. Y.L. is grateful for the support from the Nature Science
Foundation of Hebei Province (No. A2021202002). X.T.W. is grateful for the
support from the National Natural Science Foundation of China (No. 51801163)
and the Natural Science Foundation of Chongqing (No. cstc2018jcyjA0765).

## References

  * (1) F. D. M. Haldane,?Rev. Mod. Phys. 89, 040502 (2017).
  * (2) C.-K. Chiu, J. C. Y. Teo, A. P. Schnyder, and S. Ryu, Rev. Mod. Phys. 88, 035005 (2016).
  * (3) N. Goldman, J. C. Budich, and P. Zoller, Nat. Phys. 12, 639 (2016).
  * (4) M. Z. Hasan and C. L. Kane, Rev. Mod. Phys. 82, 3045 (2010).
  * (5) X.-L. Qi and S.-C. Zhang, Rev. Mod. Phys. 83, 1057 (2011).
  * (6) L. Fu, C. L. Kane, and E. J. Mele, Phys. Rev. Lett. 98, 106803 (2007).
  * (7) Y. Tokura, K. Yasuda, and A. Tsukazaki, Nat. Rev. Phys. 1, 126 (2019).
  * (8) L. Fu, Phys. Rev. Lett. 106, 106802 (2011).
  * (9) Y. Ando and L. Fu, Annu. Rev. Condens. Matter Phys. 6, 361 (2015).
  * (10) T. H. Hsieh, H. Lin, J. Liu, W. Duan, A. Bansil, and L. Fu, Nat. Commun. 3, 982 (2012).
  * (11) M. Dzero, K. Sun, V. Galitski, and P. Coleman, Phys. Rev. Lett. 104, 106408 (2010).
  * (12) M. Dzero, J. Xia, V. Galitski, and P. Coleman, Annu. Rev. Condens. Matter Phys. 7, 249 (2016).
  * (13) M. Dzero, K. Sun, P. Coleman, and V. Galitski, Phys. Rev. B 85, 045130 (2012).
  * (14) F. Schindler, A. M. Cook, M. G. Vergniory, Z. Wang, S. S. P. Parkin, B. A. Bernevig, and T. Neupert, Sci. Adv. 4, eaat0346 (2018).
  * (15) E. Khalaf, Phys. Rev. B 97, 205136 (2018).
  * (16) M. Ezawa, Phys. Rev. Lett. 120, 026801 (2018).
  * (17) R. Chen, C.-Z. Chen, J.-H. Gao, B. Zhou, and D.-H. Xu, Phys. Rev. Lett. 124, 036803 (2020).
  * (18) A. A. Burkov, Nat. Mater. 15, 1145 (2016).
  * (19) H. Gao, J. W. F. Venderbos, Y. Kim, and A. M. Rappe, Annu. Rev. Mater. Res. 49, 153 (2019).
  * (20) B. Q. Lv, T. Qian, and H. Ding, Rev. Mod. Phys. 93, 025002 (2021).
  * (21) D. C$\rm{\breve{a}lug\breve{a}ru,V.Juri\check{c}i\acute{c}}$, and B. Roy, Phys. Rev. B 99, 041301(R) (2019).
  * (22) Z. Zhang, Z.-M. Yu, and S. A. Yang, Phys. Rev. B 103, 115112 (2021).
  * (23) W. Wu, Z.-M. Yu, X. Zhou, Y. X. Zhao, and S. A. Yang, Phys. Rev. B 101, 205314 (2020).
  * (24) Z.-M. Yu, W. K. Wu, X.-L. Sheng, Y. X. Zhao, and S. Y. A. Yang, Phys. Rev. B 99, 121106(R) (2019).
  * (25) Z.-M. Yu, Z. Zhang, G.-B. Liu, W. Wu, X.-P. Li, R.-W. Zhang, S. A. Yang, and Y. Yao, arXiv:2102.01517 (2021).
  * (26) N. P. Armitage, E. J. Mele, and A. Vishwanath, Rev. Mod. Phys. 90, 015001 (2018).
  * (27) Q. D. Gibson, L. M. Schoop, L. Muechler, L. S. Xie, M. Hirschberger, N. P. Ong, R. Car, and R. J. Cava, Phys. Rev. B 91, 205128 (2015).
  * (28) B.-J. Yang and N. Nagaosa, Nat. Commun. 5, 4898 (2014).
  * (29) B. Yan and C. Felser, Annu. Rev. Condens. Matter Phys. 8, 337 (2017).
  * (30) A. A. Soluyanov, D. Gresch, Z. Wang, Q. Wu, M. Troyer, X. Dai, and B.?A. Bernevig, Nature (London) 527, 495 (2015).
  * (31) C. Fang, Y. Chen, H.-Y. Kee, and L. Fu, Phys. Rev. B 92, 081201 (2015).
  * (32) Q. Xu, R. Yu, Z. Fang, X. Dai, and H. Weng, Phys. Rev. B 95, 045136 (2017).
  * (33) T. He, X. Zhang, Y. Liu, X. Dai, G. Liu, Z.-M. Yu, and Y. Yao, Phys. Rev. B 102, 075133 (2020).
  * (34) L. Jin, X. M. Zhang, Y. Liu, X. F. Dai, X. N. Shen, L. Y. Wang, and G. D. Liu, Phys. Rev. B 102, 125118 (2020).
  * (35) H. Zhang, X. Zhang, T. He, X. Dai, Y. Liu, G. Liu, L. Wang, and Y. Zhang, Phys. Rev. B 102, 155116 (2020).
  * (36) W. Wu, Y. Liu, S. Li, C. Zhong, Z.-M. Yu, X.-L. Sheng, Y. X. Zhao, and S. A. Yang, Phys. Rev. B 97, 115125 (2018).
  * (37) X. M. Zhang, Z.-M. Yu, Z. M. Zhu, W. K. Wu, S.-S. Wang, X.-L. Sheng, and S. A. Yang, Phys. Rev. B 97, 235150 (2018).
  * (38) B.-B. Fu, C.-J. Yi, T.-T. Zhang, M. Caputo, J.-Z. Ma, X. Gao, B. Q. Lv, L.-Y. Kong, Y.-B. Huang, P. Richard, M. Shi, V. N. Strocov, C. Fang, H.-M. Weng, Y.-G. Shi, T. Qian, and H. Ding, Sci. Adv. 5, eaau6459 (2019).
  * (39) S. Z. Chen, S. Li, Y. Chen, and W. Duan, Nano Lett. 20, 5400 (2020).
  * (40) Q.-F. Liang, J. Zhou, R. Yu, Z. Wang, and H. Weng, Phys. Rev. B 93, 085427 (2016).
  * (41) A. Topp, R. Queiroz, A. Grneis, L. M$\rm{\ddot{u}}$chler, A. W. Rost, A. Varykhalov, D. Marchenko, M. Krivenkov, F. Rodolakis, J. L. McChesney, B. V. Lotsch, L. M. Schoop, and C. R. Ast, Phys. Rev. X 7, 041073 (2017).
  * (42) C. Chen, X. Xu, J. Jiang, S.-C. Wu, Y. P. Qi, L. X. Yang, M. X. Wang, Y. Sun, N. B. M. Schr$\rm{\ddot{o}}$ter, H. F. Yang, L. M. Schoop, Y. Y. Lv, J. Zhou, Y. B. Chen, S. H. Yao, M. H. Lu, Y. F. Chen, C. Felser, B. H. Yan, K. Liu, and Y. L. Chen, Phys. Rev. B 95, 125126 (2017).
  * (43) F. Zhou, Y. Liu, J. H. Wang, M. Q. Kuang, T. Yang, H. Chen, X. T. Wang, and Z. X. Cheng, Phys. Rev. Materials 5, 074201 (2021).
  * (44) T. Yang and X. Zhang, J. Mater. Chem. C 8, 9046 (2020).
  * (45) O. Stenull, C. L. Kane, and T. C. Lubensky, Phys. Rev. Lett. 117, 068001 (2016).
  * (46) Y. Liu, X. Chen, and Y. Xu, Adv. Funct. Mater. 30, 1904784 (2019).
  * (47) T. T. Zhang, Z. D. Song, A. Alexandradinata, H. M. Weng, C. Fang, L. Lu, and Z. Fang, Phys. Rev. Lett. 120, 016401 (2018).
  * (48) J. X. Li, Q. Xie, S. Ullah, R. H. Li, H. Ma, D. Z. Li, Y. Y. Li, and X.-Q. Chen, Phys. Rev. B 97, 054305 (2018).
  * (49) H. Miao, T. T. Zhang, L. Wang, D. Meyers, A. H. Said, Y. L. Wang, Y. G. Shi, H. M. Weng, Z. Fang, and M. P. M. Dean, Phys. Rev. Lett. 121, 035302 (2018).
  * (50) B. W. Xia, R. Wang, Z. J. Chen, Y. J. Zhao, and H. Xu, Phys. Rev. Lett. 123, 065501 (2019).
  * (51) Q.-B. Liu, Y. Qian, H.-H. Fu, and Z. Wang, Npj Comput. Mater. 6, 95 (2020).
  * (52) Q.-B. Liu, Z. Wang, and H.-H. Fu, Phys. Rev. B 103, L161303 (2021).
  * (53) J. Liu, W. Hou, E. Wang, S. Zhang, J.-T. Sun, and S. Meng, Phys. Rev. B 100, 081204(R) (2019).
  * (54) Y. J. Jin, Z. J. Chen, X. L. Xiao, and H. Xu, Phys. Rev. B 103, 104101 (2021).
  * (55) J.-Y. You, X.-L. Sheng, and G. Su, Phys. Rev. B 103, 165143 (2021).
  * (56) G. Liu, Y. J. Jin, Z. G. Chen, and H. Xu, Phys. Rev. B 104, 024304 (2021).
  * (57) B. Zheng, B. Xia, R. Wang, Z. Chen, J. Zhao, Y. Zhao, and H. Xu, Phys. Rev. B 101, 100303(R) (2020).
  * (58) Y. J. Jin, Z. J. Chen, B. W. Xia, Y. J. Zhao, R. Wang, and H. Xu, Phys. Rev. B 98, 220103(R) (2018).
  * (59) R. Y. Wang, Z. J. Chen, Z. Q. Huang, B. W. Xia, and H. Xu, Phys. Rev. Materials 5, 084202 (2021).
  * (60) B. B. Zheng, F. Y. Zhan, X. Z. Wu, R. Wang, and J. Fan, Phys. Rev. B 104, L060301 (2021).
  * (61) J. J. Zhu, W. K. Wu, J. Z. Zhao, Hao. Chen, L. F. Zhang, S. Y. A. Yang, arXiv:2104.14816 (2021).
  * (62) Q.-B. Liu, H.-H. Fu, and R. Q. Wu, Phys. Rev. B 104, 045409 (2021).
  * (63) J. Li, J. Liu, S. A. Baronett, M. Liu, L. Wang, and R. Li, Y. Chen, D. Li, Q. Zhu, and X.-Q. Chen, Nat. Commun. 12, 1204 (2021).
  * (64) T. T. Zhang, H. Miao, Q. Wang, J. Q. Lin, Y. Cao, G. Fabbris, A. H. Said, X. Liu, H. C. Lei, Z. Fang, H. M. Weng, and M. P. M. Dean, Phys. Rev. Lett. 123, 245302 (2019).
  * (65) Q.-B. Liu, Z.-Q. Wang, and H.-H. Fu, Phys. Rev. B 104, L041405 (2021).
  * (66) J. P. Perdew, K. Burke, and M. Ernzerhof, Phys. Rev. Lett. 77, 3865 (1996).
  * (67) A. Togo and I. Tanaka, Scr. Mater. 108, 1 (2015).
  * (68) L. D. Gulay, V. Ya. Shemet, and I. D. Olekseyuk, J. Alloys Compd. 388, 59 (2005).
  * (69) J. J. Murray and R. D. Heyding, Can. J. Chem. 45, 2675 (1967).
  * (70) L. S. Cavalcante, A. Z. Simoes, E. Longo, J. C. Sczancoski, R. Erlo, M. T. Escote, V. M. Longo, and J. A. Varela, Solid State Sci. 9, 1020 (2007).
  * (71) J. P. Remeika and A. A. Ballman, Appl. Phys. Lett. 5, 180 (1964).
  * (72) K. B. Aleinikova, A. I. Kozlov, S. G. Kozlova, and V. V. Sobolev, Phys. Solid State, 44, 1257 (2002).
  * (73) A. J. Foecker and W. Jeitschko, J. Sol. State. Chem. 162, 69 (2001).
  * (74) L. S. Ramsdell, Am. Mineral. 10, 281 (1925).
  * (75) J. Evers, J. Solid State Chem. 32, 77 (1980).
  * (76) Y. L. Fur and S. Al$\rm{\acute{e}}$onard, Acta Cryst. 28, 2115 (1972).
  * (77) J.-Z. Ma, Q.-S. Wu, M. Song, S.-N. Zhang, E. B. Guedes, S. A. Ekahana, M. Krivenkov, M. Y. Yao, S.-Y. Gao, W.-H. Fan, T. Qian, H. Ding, N. C. Plumb, M. Radovic, J. H. Dil, Y.-M. Xiong, K. Manna, C. Felser, O. V. Yazyev, and M. Shi, Nat. Commun. 12, 1 (2021).
  * (78) Z.-M. Yu, W. Wu, Y. X. Zhao, and S. A. Yang, Phys. Rev. B 100, 041118(R) (2019).


# Exemplar-based Pattern Synthesis with Implicit Periodic Field Network

Haiwei Chen1,2 Jiayi Liu1,2 Weikai Chen3 Shichen Liu1,2 Yajie Zhao1,2
1University of Southern California
2USC Institute for Creative Technologies
3Tencent Game AI Research Center
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Synthesis of ergodic, stationary visual patterns is widely applicable in
texturing, shape modeling, and digital content creation. The wide
applicability of this technique thus requires the pattern synthesis approaches
to be scalable, diverse, and authentic. In this paper, we propose an exemplar-
based visual pattern synthesis framework that aims to model the inner
statistics of visual patterns and generate new, versatile patterns that meet
the aforementioned requirements. To this end, we propose an implicit network
based on generative adversarial network (GAN) and periodic encoding, thus
calling our network the Implicit Periodic Field Network (IPFN). The design of
IPFN ensures scalability: the implicit formulation directly maps the input
coordinates to features, which enables synthesis of arbitrary size and is
computationally efficient for 3D shape synthesis. Learning with a periodic
encoding scheme encourages diversity: the network is constrained to model the
inner statistics of the exemplar based on spatial latent codes in a periodic
field. Coupled with continuously designed GAN training procedures, IPFN is
shown to synthesize tileable patterns with smooth transitions and local
variations. Last but not least, thanks to both the adversarial training
technique and the encoded Fourier features, IPFN learns high-frequency
functions that produce authentic, high-quality results. To validate our
approach, we present novel experimental results on various applications in 2D
texture synthesis and 3D shape synthesis.

##  1 Introduction

The synthesis of visual patterns, may that be a wooden texture for painting,
or a simulation of natural cave systems, is a technique that is applied
ubiquitously in computer-aided design and digital content creation. Visual
patterns can be understood as arts, shapes, or natural textures following
certain geometric structures. In an application context, let us start by
defining several characteristics that are desirable for an algorithm that
generates visual patterns:

  * •

Authenticity. Probably the most prioritized quality of synthesized visual
patterns is its visual quality. When patterns are synthesized from an
exemplar, the quality is determined by whether they faithfully recreate the
source pattern.

  * •

Diversity. It would be undesirable for a synthesizer to only copy patterns
from the source. Diversity is thus an equally important measurement that
evaluates whether the synthesized patterns vary from the source and each
other. We strive to achieve two different levels of diversity: the patterns
should be diversified both within a generated sample and across samples.

  * •

Scalability. As patterns are usually demanded at different and potentially
large scales for many practical applications, we want a scalable synthesizer
to be able to efficiently generate patterns of arbitrary size. Scalability is
particularly valuable when it comes to the synthesis of 3D models, as the
extra dimension translates to a much larger amount of computations.

A scalable design choice leads us to formulate the synthesis problem as
generating patterns from a continuous, real coordinate space. This is
generally known as the implicit formulation, where a nonlinear function maps
points defined in $\mathbb{R}^{2}$ or $\mathbb{R}^{3}$ to features that
represent the synthesized subjects. In particular, the implicit function has
been shown to be an efficient representation for synthesizing 3D volumes
[henzler2020learning, park2019deepsdf, mildenhall2020nerf].

Patterns that scale well to an infinitely large space, in general, possess a
stationary property - a shift-invariant structure that can be expanded by
tiling or stacking blocks of elements. We therefore develop our method by
pivoting on the fact that many types of natural and artistic patterns can be
analyzed and recreated in a stationary framework. The goal of synthesizing an
authentic and diverse stationary pattern from an exemplar, however, requires
careful modeling that is compatible with the underlying structure of the
pattern.

Generative adversarial networks (GAN) [goodfellow2014generative] is one of the
most promising techniques so far to model data distribution in an unsupervised
manner and has been frequently adapted to convolutional models that synthesize
visually authentic images [shaham2019singan, jetchev2016texture,
bergmann2017learning, zhou2018non, xian2018texturegan, shocher2019ingan]. How
would a GAN generator be leveraged to modeling stationary pattern? As all
stationary patterns contain a repeating structure with local variations that
“vivifies” its appearance, in an ideal situation, a stationary pattern can be
modeled by a discrete random field, where each random variable is associated
with a patch of the basic element. Thus a natural GAN formulation models image
patches with a spatially defined latent field [jetchev2016texture,
bergmann2017learning]. In a convolutional framework, however, problems arise
when fake samples generated from a discrete noise image are discriminated from
randomly sampled patches from a real image. The first problem is that the
sampled patch does not necessarily agree with the scale of the repeating
structure. The second problem is that the sampled patch can be arbitrarily
shifted from the center of a stationary element. A typical deconvolutional
network [dong2015image, zeiler2010deconvolutional] that upsamples from an
evenly-spaced noise image may not sufficiently address the previously
mentioned problems. To study their effects we designed a convolutional network
following the DCGAN architecture [radford2015unsupervised] to synthesize a
honeycomb pattern from a $2\times 2$ noise map, which is trained against
random patches sampled from a source image. The comparison between its result
and that synthesized by our generator network that is trained with an
identical discriminator is shown in Figure. 1. We found that the noise map
does not capture well the honeycomb structure as seams and intercepting
elements are visible from the synthesized image.

Though various techniques have been proposed in the past to address the
aforementioned issues (e.g. [jetchev2016texture, bergmann2017learning]), in
this paper, we consider a more natural way to model stationary patterns with
an implicit periodic generator. The core of the formulation is to match the
repeatable structure of a stationary pattern to the period of a learnable
continuous periodic function. Instead of modeling the pattern with a discrete
noise tensor, we define latent variables in a continuous space, where the
extent of each latent factor is learned to match with the extent of the
repeating elements. The benefits of this design align well with the desirable
characteristics for visual pattern synthesis: 1) learned periodicity of the
implicit field encourages latent factors to model the stationary variations
observed from the exemplar pattern; 2) a continuous representation provides
flexibility during training to learn a distribution from randomly shifted
patches cropped from the exemplar; 3) a Fourier encoding scheme learns high-
frequency details from the exemplar. This allows our model to synthesize
visually authentic results, and 4) multilayer perceptron (MLP) that implicitly
takes coordinates as input scales well to the generation of 3D shapes when
compared to 3D convolution. Based on these design choices, we term our network
the Implicit Periodic Field Network (IPFN).

We validate our proposed design by showing various applications in texture
image synthesis and 3D volume synthesis in Section. 4. Specifically, besides
synthesizing stationary patterns, we design a conditional formulation of our
model to tackle the synthesis of directional patterns and to provide
application in controllable shape generation. An ablation study is also
conducted to verify the effectiveness of our design choices.

Figure 1: Comparison between synthesized honeycomb from a DCGAN convolution
generator and the periodic MLP generator. Seams and intercepting patterns are
visible in the former result due to difficulty for the convolution generator
to capture the repeating structure.

##  2 Related Work

#### Pattern Synthesis

2D visual patterns are generally referred to as “texture” due to their
prevalent applications in computer-aided design. Well-known early attempts to
synthesize texture derive patterns from smoothly interpolated noise
[perlin1985image, perlin2002improving, worley1996cellular] and create
aesthetically pleasing materials that display a high level of randomness.
[henzler2020learning, portenier2020gramgan] are two recent works related to us
that utilize randomness from a continuously-defined Perlin noise
[perlin1985image] to synthesize exemplar-based textures in an implicit field.
Their works demonstrate the advantage of smooth noise field and implicit
formulation in efficiently and diversely generating a 3D texture field.
However, just like their procedural precedence, these methods have been shown
to be limited in synthesizing patterns that are more complicated in structures
(e.g. bricks, rocky surface) [portenier2020gramgan].

Later works in traditional image synthesis are generally divided into pixel-
based method (e.g. [efros1999texture, wei2003texture]), patch-based method
(e.g. [kopf2007solid, efros2001image, kwatra2003graphcut]) and optimization-
based method (e.g. [kopf2007solid, kwatra2005texture]) and have shared with us
important considerations for recreating new patterns from an exemplar. For
instance, synthesis on a patch level encourages preservation of fine local
details, and the efforts are focused on ”quilting” the discontinuity between
patches [efros2001image, kwatra2003graphcut] and encouraging global similarity
[kopf2007solid]. Early statistical model [efros1999texture,
portilla2000parametric] utilizes a random field representation that captures
variations in stationary patterns and synthesizes a variety of new patterns
under the Julesz’ conjecture. Our work is inspired by key ideas from the
traditional modeling of stationary textures, while we strive to unify
authenticity, diversity, and scalability with a neural representation to
overcome limitations that existed in the traditional approaches.

Compared to earlier parametric models, the artificial neural network is
powerful in its generalizability in pattern recognition [hansen1990neural].
Thus neural synthesis of texture images has marked the advances in recent
endeavors. How does a neural network learn the stylish representation of a
texture without simply reconstructing it? A milestone that unifies both
synthesized quality and sampling power adversarially trains a generator
network and a discriminator network to learn the mapping from a latent space
to the texture distribution [goodfellow2014generative]. We are particularly
interested in the generative adversarial networks (GANs) that models the inner
statistics of the exemplars. This is marked by a patch-based approach that
represents an image as a collection of smaller images: [li2016precomputed,
jetchev2016texture] formalizes patch-based synthesis in the GAN setting with
the concept of Markovian and spatial GAN. [bergmann2017learning] motivated us
with its periodic design in latent codes, which effectively captures
stationary patterns from an input exemplar. [zhou2018non] can be seen as a
complement to our design by focusing on addressing non-stationary texture
synthesis by learning to expand an image patch. In addition,
[shaham2019singan, shocher2019ingan] present multi-scale, global-focused
approaches to effectively recreate natural images. While the aforementioned
approaches all utilize convolutional designs, our work extends texture
synthesis to the continuous domain with an implicit representation of
textures, as we argue that such representation provides a more natural and
efficient way to synthesize stationary patterns.

The synthesis of 3D shapes is of particular interest in computer graphics and
thus has a rich history. To name a few, this includes volumetric field design
[zhang2006vector, palacios2016tensor, chen2016synthesis], procedural
generation [ijiri2008example, merrell2007example] and 3D texturing
[zhou2006mesh, bhat2004geometric]. However, very few works have considered the
synthesis of 3D patterns with neural networks, with the exception of
[henzler2020learning, portenier2020gramgan], which explores the generation of
3D solid textures.

#### Implicit Network

Implicit network refers to multilayer perceptron that learns a continuous
function in real coordinate space. In particular, the implicit network is
mainly utilized in the reconstruction of 3D shapes [park2019deepsdf,
saito2019pifu, liu2019learning, sitzmann2020implicit], where shapes are
represented by a distance function, or a radiance field [mildenhall2020nerf,
yu2021pixelnerf]. We are motivated by the signed distance representation of
shapes and the Fourier encoding explored in [mildenhall2020nerf,
tancik2020fourier] in our design of the implicit networks, and our work adopts
these features to a generative setting where novel 3D patterns are
synthesized.

##  3 Method

Our method is best introduced by expanding from the Wasserstein GAN value
function [gulrajani2017improved] constructed as the Earth-Mover distance
between the real data distribution $\mathbb{P}_{data}$ and the generator
distribution $\mathbb{P}_{g}$:

$\underset{G}{\min}\>\underset{D}{\max}\>\mathbb{E}_{\bm{x}\sim\mathbb{P}_{data}}[\log(D(\bm{x}))]-\mathbb{E}_{\bm{z}\sim\mathbb{P}_{Z}}[\log(D(G(\bm{z}))].$
(1)

Our first change of the above objective is to draw real-valued coordinates
$\bm{c}\in\mathbb{R}^{k}$ ($k=2$ for image and $k=3$ for volume) from a
distribution $\\{s\bm{c}\>|\>\bm{c}\sim\mathbb{P}_{\bm{c}}\\}$ as input to the
generator, where $s$ is a constant scalar. This underlies the implicit nature
of the generative network, as $G$ learns a mapping from the real coordinate
space in a stochastic process. Instead of being sampled from a prior
distribution $\mathbb{P}_{Z}$, latent variables are drawn from a random field
$f_{z}(\bm{c})$ defined on the real coordinate space. $G$ is therefore a
function of both the coordinates $\bm{c}\in\mathbb{R}^{k}$ and the latent
variables $f_{z}(\bm{c})\in\mathbb{R}^{d}$. This updates Eq.1 to

$\begin{split}\underset{G}{\min}\>\underset{D}{\max}\>&\mathbb{E}_{\bm{x}\sim\mathbb{P}_{data}}[\log(D(\bm{x}))]-\\\
&\mathbb{E}_{\bm{c}\sim\mathbb{P}_{c},\bm{z}\sim\mathbb{P}_{f_{z}(\bm{c})}}[\log(D(G(\bm{z},\bm{c}))].\end{split}$
(2)

In the implementation, our goal is to synthesize color, distance function,
normal vectors, etc. that are defined in a grid-like structure
$X:\mathbb{R}^{H\times W\times C}$ or $X:\mathbb{R}^{H\times W\times D\times
C}$ and thus we also sample grid coordinate input $C:\mathbb{R}^{H\times
W\times 2}$ or $C:\mathbb{R}^{H\times W\times D\times 3}$ whose center is
drawn from a uniform distribution $U(-s,s)$. The randomness to the grid center
position is critical in encouraging smooth and seamless transition between
blocks of patterns as it models the distribution of randomly sampled patches
from the input pattern.

In the following sections, key modules are discussed in detail: 1) a
deformable periodic encoding of the coordinates to model stationary patterns;
2) implementation of the latent random field; 3) a conditional variance of our
objective for the synthesis of directional exemplar and controllability of the
synthesized patterns, and 4) the overall network structure that completes our
pattern synthesis framework.

###  3.1 Periodic Encoding

As discussed in the introduction, it is critical to represent stationary
patterns from the input by a repeated structure that avoids simply
reconstructing the original exemplar. Simply mapping spatial coordinates to
the visual pattern does not satisfy this requirement: since each coordinate is
unique in the real-value space, the network would learn to overfit the
coordinates to their associated positions in the exemplar and therefore fail
to capture a repeatable structure.

The benefits of a periodic encoding to the coordinates are two-fold: Firstly,
it disentangles patch-level appearance from their specific position in the
exemplar, which allows the pattern synthesized by the generator to be shift-
invariant. Secondly, recent advances in implicit network [mildenhall2020nerf,
tancik2020fourier, sitzmann2020implicit] have found a Fourier feature mapping
with a set of sinusoids effective in learning high-frequency signals by
addressing the “spectral bias”[basri2020frequency] inherent to MLPs. In our
work, we use the following periodic mapping for the input coordinates:

$\begin{split}\gamma(\bm{c})=[\cos(2^{0}\pi a\bm{c}),\sin(2^{0}\pi
a\bm{c}),&\\\ \cdot\cdot\cdot,\cos(2^{i}\pi a\bm{c}),\sin(2^{i}\pi
a\bm{c})],\end{split}$ (3)

where the learnable parameter $a\in\mathbb{R}^{k}$ determines the period of
the encoding. This design allows the network to learn to match the period of
the encoding to the repeatable structure of the exemplar. It thus provides
robustness to the scale of the patches sampled in training as such scale no
longer dictates the period of the synthesized pattern.

###  3.2 Latent Random Field

In a generative framework, latent noise sampled from a prior distribution
model the variation in observations. A noise function that is smoothly defined
in the real coordinate space encourages smooth transition between the
synthesized patches. In a 2D example, we start with a discrete random field
that maps a uniform grid of coordinates to random variables
$\\{f_{z}(\bm{c})\>|\>\bm{c}:\mathbb{R}^{H\times W\times 2}\\}$. Then the
discrete random field is smoothly interpolated to form a smooth latent field
(see the visualization in Figure 2). In our implementation, we used an
exponential interpolation:

$f(\bm{x})=\sum_{i=1}^{K}w_{i}(\bm{x})f_{z}(\bm{c}_{i}),\>w_{i}(\bm{x})=\frac{\mathrm{e}^{\frac{||\bm{x}-\bm{c}_{i}||_{2}}{\sigma}}}{\sum_{j=1}^{K}\mathrm{e}^{\frac{||\bm{x}-\bm{c}_{j}||_{2}}{\sigma}}},$
(4)

where the latent code at spatial position $x$ is interpolated from $K=4$
latent vectors defined at the grid corners. In implementation, the discrete
grid used to define the random field has a spacing of 1. To match the extent
of a latent factor with the learned period $a$, we simply scale the uniform
grid of the discrete random field accordingly.

###  3.3 Conditional IPFN

Extending our GAN objective to be conditional enables many practical
applications. Assume each input patch is paired with a guidance factor
$\bm{g}$, the conditional objective is simply an extension:

$\begin{split}\underset{G}{\min}\>\underset{D}{\max}\>&\mathbb{E}_{\bm{x}\sim\mathbb{P}_{data}}[\log(D(\bm{x}|\bm{g}))]-\\\
&\mathbb{E}_{\bm{c}\sim\mathbb{P}_{c},\bm{z}\sim\mathbb{P}_{f_{z}(\bm{c})}}[\log(D(G(\bm{z},\bm{c}|\bm{g}))].\end{split}$
(5)

Here we outline two applications in pattern synthesis using the conditional
formulation:

  * •

Synthesis of directional pattern: Many natural patterns have a directional
distribution that is oftentimes considered non-stationary. A typical example
is a leaf texture - a midrib defines the major direction that separates the
blade regions by half (see Figure 3). A conditional extension of our model is
able to model patch distribution along a specified direction. For simplicity,
we present a 2D formulation that can be easily extended to 3D. With a user-
defined implicit 2D line equation $ax+by+c=0$, the guidance factor is defined
as $\bm{g}(x,y)=ax+by+c$. Pixel coordinates $(p_{x},p_{y})$ from an input
texture image with width $w$ and height $h$ are transformed as
$(x,y)=(\frac{2p_{x}}{w}-1,\frac{2p_{y}}{h}-1)$ to be normalized to the value
range of $[-1,1]$. In our experiments we have found it sufficient to condition
our model on the horizontal ($y=0$) and vertical direction ($x=0$) for the
evaluated exemplars.

  * •

Controlling synthesis of 3D shapes: In the modeling of geometric patterns, it
is often desirable for the synthesis algorithm to provide explicit control of
certain geometric properties such as density and orientation. These geometric
properties can be calculated from the exemplar shape. Let $g(x)$ be a shape
operator that defines the geometric property of interest, our conditional
model trained with the sample pair $(x,g(x))$ then learns a probabilistic
mapping from a guidance vector field to the target 3D shape. An intuitive
example of this application can be found in Section 4.5.

###  3.4 Network Structure

Figure 2: Overview of our network architecture discussed in Section 3.4.

The overall structure of IPFN is visualized in Figure 2. The Generator Network
$G$ is a 10-layer MLP with ReLU activations between layers and a sigmoid
function in the end. A grid of coordinates is sampled based on a randomly
shifted center. The coordinates are then passed to two separate branches: 1)
the periodic encoder, and 2) a projection on the latent field to obtain a
5-dim latent vector for each coordinate. The latent codes and the periodically
encoded coordinates are then concatenated as input to the generator mlp, which
outputs the fake sample. The discriminator $D$ discriminates between the
generated samples and randomly cropped patches from the real input. We
implement $D$ by following the DCGAN architecture [radford2015unsupervised]
with a stride of 2. For discriminating 3D volumes, 2D convolution layers are
replaced by 3D convolution.

Figure 3: Main results for 2D texture synthesis with comparisons to Henzler et
al. [henzler2020learning], Bergmann et al. [bergmann2017learning], and Zhou et
al. [zhou2018non] on synthesizing two stationary patterns (top four rows) and
two directional patterns (bottom four rows).

##  4 Experiments

We hypothesize that our approach is most suitable for synthesizing texture
patterns and 3D shapes with repeating structures and local variations in
appearance. To this end, we demonstrate our main results by applying IPFN to
the synthesis of 2D texture images and 3D structured shapes. In addition, IPFN
is adapted to two applications in 3D texturing and shape manipulation. To
evaluate the effectiveness of the proposed techniques, we have also conducted
an ablation study where several key designs are altered.

Evaluation metric While the quality for pattern synthesis is not easily
quantifiable, human eyes usually provide a reasonable qualitative assessment
for whether the synthesized patterns capture the aesthetics and structure of
the exemplar. In our evaluation, we present comparisons of visual results that
are self-manifesting, since the synthesized patterns bear obvious
characteristics of the underlying designs of the synthesizer. In addition, we
have provided quantifiable metrics in terms of Single Image Frechét Inception
Distance and inference time and memory.

Implementation details For all of our experiments, the network is optimized
under WGAN loss with gradient penalty [gulrajani2017improved]. Adam optimizer
[kingma2014adam] is used with a learning rate of $1e-4$ for both the
discriminator D and the generator G. In each iteration, both D and G are
updated for 5 steps sequentially. Input images and volumes are randomly
cropped to a smaller-scale patch. For positional encoding, we choose a
bandwidth $i=5$ as a wider kernel tends to produce sinusoidal artifacts,
whereas a narrower kernel produces blurry results. The input coordinates are
randomly shifted by an offset in the range $[-4,4]$ to accommodate for the
chance that the network may learn an increased period for the periodic
encoding. Accordingly, noises are interpolated from a $5\times 5$ grid
($5^{3}$ for 3D volume) discrete random field, where the point locations are
ranged between $[-5,5]$. A single-exemplar experiment is typically trained for
12,500 iterations with a batch size of 8 and runs on a single Nvidia GTX 1080
GPU, which takes about 6-9 hours to complete. Inference Time: IPFN only
requires 24 milliseconds to generate a $1024\times 1024$ image. 3D volumes are
generated iteratively and a large-scale $512^{3}$ volume takes only 22.9
seconds to be generated. Source code will be made publicly available upon
acceptance.

###  4.1 Texture Pattern Synthesis

Image sources selected from the Oxford Describable Textures Dataset (DTD)
[cimpoi2014describing] are shown in Figure 3. Specifically, we selected two
exemplars with stationary patterns (top 4 rows in Figure 3) and two exemplars
with directional patterns (bottom 4 rows in Figure 3) to demonstrate that IPFN
synthesizes visually similar patterns in both cases. During training, images
were randomly cropped into patches of size $128\times 128$. During inference,
the synthesized images were scaled up four times to a size of $512\times 512$.
Our results are compared to the three most relevant baseline generative
methods that synthesize texture patterns from a single image:

  * •

Henzler et al. [henzler2020learning]: A method that similarly utilizes
implicit network and smooth noise field for texture synthesis. The synthesized
results were obtained from running the officially released code.

  * •

Bergmann et al. [bergmann2017learning]: A convolutional method that combines
noise with periodic signals to synthesize stationary pattern. The synthesized
results were obtained from running the officially released code.

  * •

Zhou et al. [zhou2018non]: A convolutional image expansion approach targeted
for non-stationary texture synthesis. Since [zhou2018non] expands from an
image input deterministically and does not utilize latent code, only one
synthesized result is shown per row. The synthesized results were obtained
from the authors.

Visual inspection is sufficient to show that IPFN provides promising results.
When compared to [henzler2020learning], IPFN synthesized results with obvious
structures as noise is not directly mapped to the output. While
[bergmann2017learning] synthesizes periodic samples that display diversity
across samples and similarity to the stationary exemplars, their synthesized
patterns lack variation within the image. In comparison, our synthesized
patterns show a higher level of local variations and adapt well to the
directional cases. [zhou2018non] has provided the most visually authentic
results among the baselines. However, in the stationary cases, radial
distortion is noticeable near the boundaries of its synthesized images.
Moreover, without requiring image input, IPFN provides a more direct approach
to synthesizing diversified samples from random noise.

| honey | crosshatch | rock | leaf
---|---|---|---|---
Henzler [henzler2020learning] | 332.66 | 310.49 | 351.23 | 225.11
Bergmann [bergmann2017learning] | 62.75 | 177.88 | 120.64 | 164.37
Zhou [zhou2018non] | 14.54 | 154.63 | 118.29 | 38.13
Ours | 10.15 | 130.83 | 113.81 | 103.6
Table 1: SIFID scores between the exemplars and the generated patterns from
ours and different baselines.

Single Image Frechét Inception Distance (SIFID) SIFID introduced in
[shaham2019singan] is a metric commonly used to assess the realism of
generated images. For the computation of SIFID, we have used a patch size of
$128\times 128$ in all experiments, where the synthesized patterns have the
same resolution as the original exemplars. Table 1 shows the SIFID comparisons
between ours and the baselines in various categories of exemplars. For Zhou et
al. [zhou2018non], only the generated (expanded) portion of the images were
used. The results show that our method can generate results that better
resemble the distribution of the real texture in the stationary categories
(honey, crosshatch, rock) as our generated patterns receive lower SIFID
scores. For the leaf category, a typical directional pattern, Zhou et al.
[zhou2018non] achieves the best performance as its method specifically targets
non-stationary expansions, while our method still performs better than other
baselines that have not taken into consideration the synthesis of non-
stationary patterns.

Time (ms) / memory (GB) | $128^{2}$ | $256^{2}$ | $512^{2}$ | $1024^{2}$
---|---|---|---|---
Henzler [henzler2020learning] | 218/1.38 | 278/1.62 | 328/2.72 | 458/6.45
Bergmann [bergmann2017learning] |  7/2.37 | 13/5.79 | 42/19.68 | 115/31.88
Zhou [zhou2018non] | 356/1.20 | 349/1.34 | 510/2.00 | 612/4.66
Ours |  8/0.76 |  11/0.85 |  15/1.23 |  24/2.81
Table 2: Comparisons of inference time and inference memory consumption,
measured in milliseconds (ms) / gigabytes (GB), when patterns of increasing
size (top row) are generated.

Inference Time and Memory Table 2 measures the inference time and memory
consumption of our network compared to the baselines when generating image at
different sizes. Our implicit formulation is shown to be significantly more
efficient in both time and space without the needs to rely on computation of
pseudo-random noise ([henzler2020learning]) or convolutional operations
([bergmann2017learning, zhou2018non]). This validates our claim on the
scalability of our design.

###  4.2 Ablation Study

Figure 4: Synthesized honeycomb textures for the ablation study. The blue
boxes represent the learned scale of periodic encoding in ours, where in w/o
deformation, the period is default to 1, which does not match with the
repeating structure of the honeycomb pattern and results in visual artifact.

To validate our design choices, we have conducted an ablation study by
removing two designs that we consider critical for our network to be
effective. The comparison results are shown in Figure 4. w/o deformation is a
network model that encodes input coordinates without the learnable parameters
$a$ described in Section 3.1. w/o shift is a model that is trained without
randomly shifting the input coordinates. The resulted patterns are indicative
of the effects of these designs: when coordinates are encoded at a fixed
scale, the w/o deformation model generates hexagons that are seemingly glued
together as the presumed scale does not match with the actual period of the
repeating structure. The w/o shift model synthesized distorted patterns as we
speculate that, without the random sampling of the input coordinates, the
network faces difficulty in matching the patch-based priors of the image
patches.

###  4.3 Volumetric Shape Synthesis

Figure 5: Main results for 3D volume synthesis. a. Exemplar porous structure.
b. Synthesized structure models interior tunnels. c. Global views of
synthesized porous structures. c. Exemplar foam structure. e. Two scales of
noise fields for the foam structure synthesis. f. Synthesized foam structures.
Larger scale of the noise field leads to more isotropic foam structures.
Figure 6: IPFN learns multi-channel textures that are applicable to seamless
3D texturing. The original 3D texture in this example is not symmetric and
therefore visible seams can be found on the texture-mapped surface and in the
closeup view (A in figure). As synthesized patterns learnt from this exemplar
can be tiled in any direction, the mapped surface (B in surface) is seamless.

For the evaluation on volumetric shape synthesis, We have obtained two 3D
models from turbosquid.com: a porous structure (Figure 5.a) and a foam
structure ((Figure 5.d). The 3D meshes are preprocessed into signed distance
fields by uniformly sampling points in a volumetric grid. For the porous
structure, we have sampled $256\times 256\times 256$ points and extracted
$64\times 64\times 64$ patches during training. For the foam structure, we
have sampled $200\times 200\times 128$ points and extracted $32\times 32\times
32$ patches during training. During inference, porous structures are
synthesized at their original resolution, while we scale the synthesized foam
structures to be twice as large as the original shape in the XY direction.
Figure 5.c and Figure 5.f show the synthesized shapes. For the porous
structures, both outer structures and interior structures are learned (see
Figure 5.b for zoom-in interior views) and the structures are diversified both
across and within samples. For the foam structures, we have shown different
results by varying the extent of the latent random field (see Figure 5.e). A
larger-scale random field encourages the synthesizer to generate globally
varied structures, whereas a smaller scale produces locally anisotropic
structures.

###  4.4 Application: Seamless 3D Texturing

Due to the periodic nature of the synthesized patterns, noise manipulation
allows IPFN to create textures that are mirror-symmetric. This property
provides an immediate application to seamless 3D texture mapping: in Figure 6,
the original 9-channel texture, composed of color, normal, and bump maps, is
tiled and directly mapped to a planar surface. Due to discrepancies on the
edges, as the textures are wrapped to create the tiled patterns, the mapped
surfaces show visible seams. We recreate this texture through our network
under a constant latent vector (Figure 6 B). When repeatedly mapped to the
surface, the symmetric texture is seamless while faithfully reflecting the
appearance and structure of the original texture.

Figure 7: Synthesized foam structures with controllable density. a. The grey
scale bar controls the synthesized structure from the highest density (white)
to the lowest (black). b. Smooth interpolation of the guidance factor allows
us to synthesize a foam structure with smoothly changing densities.

###  4.5 Application: 3D Foam with controllable density

The original foam shape used in our experiment contains holes of various
sizes, which corresponds to the density of the foam structure. This geometric
property $g$ can be easily approximated with an unsigned distance field
representation. For a patch $X^{\prime}$ of size $H^{\prime}\times
W^{\prime}\times D^{\prime}$, we estimate its density by:

$g(X^{\prime})=\frac{1}{m}\sum_{i=1}^{H^{\prime}}\sum_{j=1}^{W^{\prime}}\sum_{k=1}^{D^{\prime}}|sdf(X_{ijk})|$
(6)

, where m is a normalization factor. Figure 7 shows the synthesized foam
structures by gradually increasing the density factor (Figure 7.a) and by a
linearly interpolated density map ((Figure 7.b).

##  5 Limitations and Discussions

Figure 8: The examples demonstrating limitations of our network

The main limitation of our method is its emphasis on modeling stationary
patterns. While this is based on our observation that a broad range of natural
patterns is stationary or directional, our method does not provide a natural
way to address a class of patterns that are radial, which are exemplified by
web structures and spiral patterns (see the third column in Figure 8).

While our conditional formulation is in theory compatible with the synthesis
of landscape images, experiments found that the quality of the synthesized
landscapes are subpar - while the synthesized landscape appears globally
similar to the exemplar, some local regions contain ”fading-out” elements that
are blended with the background (see the first two columns in Figure 8). We
speculate that this phenomenon is due to an under-representation of these
elements in the exemplar.

The above limitations have inspired us to consider many potential improvements
of our methods in future works. A multi-scale synthesis approach marked in
[shaham2019singan, shocher2019ingan] strikes a good balance between learning
the distribution of global structure and local, high-frequency details of an
image. Different geometric encoding schemes may also extend our framework to
synthesize beyond stationary patterns. We believe there are still ample
opportunities for the extension of our methods to a broader range of 3D
applications.

## Acknowledgements

This research was sponsored by the Army Research Office and was accomplished
under Cooperative Agreement Number W911NF-20-2-0053, and by the U.S. Army
Research Laboratory (ARL) under contract number W911NF-14-D-0005. The views
and conclusions contained in this document are those of the authors and should
not be interpreted as representing the official policies, either expressed or
implied, of the Army Research Office or the U.S. Government. The U.S.
Government is authorized to reproduce and distribute reprints for Government
purposes notwithstanding any copyright notation.