# Electronic, Magnetic and Vibrational Properties of Single Layer Aluminum
Oxide

A. Kutay Ozyurt  Department of Physics, Izmir Institute of Technology, 35430,
Izmir, Turkey  Deniz Molavali  Department of Physics, Izmir Institute of
Technology, 35430, Izmir, Turkey  Hasan Sahin  Department of Photonics, Izmir
Institute of Technology, 35430, Izmir, Turkey

###### Abstract

The structural, magnetic, vibrational and electronic properties of single
layer aluminum oxide (AlO2) are investigated by performing state-of-the-art
first-principles calculations. Total energy optimization and phonon
calculations reveal that aluminum oxide forms a distorted octahedral structure
(1T′-AlO2) in its single layer limit. It is also shown that surfaces of
1T′-AlO2 display magnetic behavior originating from the O atoms. While the
ferromagnetic (FM) state is the most favorable magnetic order for 1T′-AlO2,
transformation to a dynamically stable antiferromagnetic (AFM) state upon a
slight distortion in the crystal structure is also possible. It is also shown
that Raman activities (350-400 cm-1) obtained from the vibrational spectrum
can be utilized to distinguish the possible magnetic phases of the crystal
structure. Electronically, both FM and the AFM phases are semiconductors with
an indirect band gap and they can form a type-III vdW heterojunction with
graphene-like ultra-thin materials. Moreover, it is predicted that presence of
oxygen defects that inevitably occur during synthesis and production do not
alter the magnetic state, even at high vacancy density. Apparently, ultra-thin
1T′-AlO2 with its stable crystal structure, semiconducting nature and robust
magnetic state is a quite promising material for nanoscale device
applications.

##  I Introduction

Compounds formed by the combination of aluminum and oxygen atoms have found
their use in many areas of technology. Among the possible compounds, Alumina
(Al2O3) with its high melting point,[1] self-healing crystal structure,[2]
high dielectric constant[3] and insulating properties[4, 5] have been widely
used in electronic device applications. While bulk Al2O3 is a suitable
material for high-throughput anti-reflective polymer lenses[6], its thin forms
have been shown to improve the spin-polarized electron tunneling in magnetic
tunnel junctions.[7] In addition, successful synthesis of its nanowire,[8]
quantum dot[9] and thin two-dimensional[10, 11] forms have also been reported
so far.

Aluminum oxides can also be stabilized in the form of aluminates
(Al${}_{\text{x}}$O${}_{\text{y}}$). It was demonstrated that thin layers of
Lanthanum-based aluminates can serve as a source of two-dimensional electron
liquid exhibiting electric-field-tunable superconductivity.[12] It was also
shown that using ultrathin Lithium aluminate nanoflakes as a polar additive,
having an enormous active surface area to immobilize the lithium polysulfides,
leads to extremely stable cycling performance in batteries.[13] Furthermore,
one-dimensional forms of strontium-europium-based aluminates are luminescent
nanoribbons that can function as both light generators and waveguides, and
therefore they are suitable materials for miniaturized photonic circuitry.[14]
In addition to experimental studies, stabilization of single layer
$\alpha$-AlO displaying indirect semiconducting behavior was also predicted
theoretically.[15]

At the point where nanotechnology has reached, it is necessary to further
miniaturize materials, and nowadays, there is also a great need for atomically
thin building blocks of all kinds. Following the first successful synthesis of
graphene,[16] its unusual properties such as high thermal conductivity,[17]
semi-metallic behavior,[18] and ultra-high mechanical strength[19] have been
demonstrated in a short period of time. Studies in the following years have
revealed that ultra-thin stable phases of other graphene-like materials may
also exist. Among them especially h-BN with its insulating properties,[20]
MoS2 with its thickness-dependent electronic and optical properties,[21, 22]
ReS2 for its in-plane anisotropic and thickness-independent properties[23] and
silicene with its ultra-high carrier mobility[24] have been the most prominent
ones.

Although many graphene-like materials have been successfully synthesized to
date, very few of them show magnetic properties. Recently, it has been found
that the itinerant ferromagnetism persists in 2D Fe3GeTe2 down to the
monolayer with an out-of-plane magnetocrystalline anisotropy.[25] In addition,
intrinsic ferromagnetic half-metallic behavior of monolayer square CrBr2 and
the strain-dependent phase transitions were predicted using first-principles
calculations.[26] Very recently, growth of iron chloride (FeCl2) films on
Au(111) and graphite with atomic thickness by molecular-beam epitaxy (MBE)
technique has also been reported.[27] Besides, we showed that the magnetic
ground state of single layer FeCl2 crystal can be modulated by the point
defects and structural stability was maintained despite the heavy defect
doping.[28]

In this work, we investigated the ultra-thin graphene-like crystal structure
of aluminum oxide (AlO2) by means of ab-initio calculations. The rest of the
paper is organized as follows; computational methodology is given in Sec. II.
Possible magnetic phases and their vibrational characteristics are
investigated in Sec. III.1. In addition, electronic properties of AFM and FM
phases and their possible vdW type heterostructures are investigated in
Sec.III.2. Formation of single O vacancy and resulting modifications in
electronic properties are given in Sec.III.3. Finally, the conclusions are
presented in Sec.IV.

Table 1: For the different magnetic phases of 1T′-AlO2: lattice constants for rectangular unit cell (a and b), thickness ($h$), the amount of donated (received) electron ($\rho{}_{\text{Al}}$ and $\rho{}_{\text{O}}$), the work function ($\phi$), calculated cohesive energy for per atom (ECoh), the band gap value (EGap), total magnetic moment for per the formula, ($\mu$), in-plane stiffness along zigzag direction, ($C_{zz}$), in-plane stiffness along armchair direction, ($C_{arm}$), Poisson ratio along zigzag direction, ($\upsilon_{zz}$), Poisson ratio along armchair direction, ($\upsilon_{arm}$), calculated hole effective mass (m${}_{h}^{*}$), calculated electron effective mass (m${}_{e}^{*}$) where m0 is the free electron mass. |  | a | b | h | $\rho{}_{\text{Al}}$ | $\rho{}_{\text{O}}$ | $\phi$ |  ECoh |  EGap | $\mu$ | $C_{zz}$ | $C_{arm}$ | $\upsilon_{zz}$ | $\upsilon_{arm}$ |  m${}_{h}^{*}$ |  m${}_{e}^{*}$ |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
|  | (Å) | (Å) | (Å) |  (e${}^{\text{-}}$) |  (e${}^{\text{-}}$) | (eV) | (eV/atom) | (eV) |  ($\mu_{B}$) | (N/m) | (N/m) | - | - |  (m0) |  (m0) |
| FM | 3.05 | 4.84 | 1.77 | 0.54 | 7.23 | 8.01 | 5.57 | 0.63 | 1 | 126 | 138 | 0.40 | 0.36 | 2.29 | 1.33 |
| AFM | 3.06 | 4.82 | 1.78 | 0.54 | 7.23 | 7.96 | 5.56 | 1.18 | 0 | 108 | 123 | 0.33 | 0.37 | 1.20/18.34 | 8.52 |
Figure 1: (a) Energy versus lattice constant (a) of 2D 1T′-AlO2. Top views of
the geometric structures of (b) FM and (c) AFM state where blue and red atoms
stand for Al and O, respectively. For 3D plot of magnetization charge
densities minority and majority states are shown by blue and yellow color.

##  II Computational Methodology

For determination of optimized crystal structures and resulting electronic and
magnetic characteristics of AlO2 first-principles calculations were performed
by using Vienna ab initio Simulation Package (VASP).[29, 30, 31] The exchange-
correlation potential was treated using the Perdew-Burke-Ernzerhof Generalized
Gradient Approximation (PBE-GGA).[32] The projector augmented wave (PAW)
potentials were used as pseudo-potentials.[33] Van der Waals (vdW) correction
within the form of DFT-D2 method of Grimme is used for approximation of
dispersion forces.[29] Bader charge technique was used for determination of
charge for each atom in the crystal structure.[34]

The kinetic energy cutoff and the convergence criteria for the total energy
were set to 500 eV and 10-5 eV, respectively. At least 18.2 Å vacuum spacing
was inserted in order to avoid adjacent layer-layer interactions along the
vertical direction. For the structural optimization of the primitive unit
cell, k-space was sampled by using 9$\times$9$\times$1 k-point mesh, and
doubled to 18$\times$18$\times$1 in density of states calculations. Structural
optimizations were carried on until the pressure is in between $\pm$1 kB along
the each lattice direction. The cohesive energy per atom was calculated using
the equation $E_{Coh}=\frac{1}{n_{tot}}[n_{Al}E_{Al}+n_{O}E_{O}-E_{SL}]$ where
$n_{Al}$ ($n_{O}$) is the number of Al (O) atoms in the system, while $E_{Al}$
and $E_{O}$ refer to individual energies of Al and O atoms, respectively.
$E_{SL}$ is the total energy of the corresponding single layer and $n_{tot}$
is the total number of atoms per unit cell.

Figure 2: Phonon band structure of 1T′-AlO2 calculated for (a) the FM and (b)
the AFM states. (c) The Raman activity spectrum of the zone-centered phonon
modes. (d) The representative images of Raman active phonon vibrations.

The vibrational properties were investigated via PHONOPY code, which uses the
small displacement method to generate the force constant matrix.[35] In
addition, the zone-centered phonon frequencies and the corresponding off-
resonant Raman activities were calculated using the finite-difference method.
Basically, in a Raman experiment instantly scattered photons, whose dispersion
with respect to a shift in frequency gives the Raman spectrum, are collected.
The treatment of Raman activities of each vibration is based on Placzek’s
classical theory of polarizability in which the activity of a Raman active
mode is proportional to the square of the term, $|\hat{e}_{s}R\hat{e}_{i}|$
where $\hat{e}_{s}$ and $\hat{e}_{i}$ stand for the polarization vectors of
the scattered and the incident light, respectively. In addition, $R$
represents the Raman tensor whose elements are the derivatives of the
polarizability of the material with respect to the phonon normal modes.
Notably, for a Raman active phonon mode some of the elements of Raman tensor
are non-zero.

##  III Results

###  III.1 Structural, Magnetic and Vibrational Properties

Characteristic vibrational, electronic and magnetic fingerprints of any
material are directly determined by its crystal symmetry. Therefore, a
reliable prediction of the ground state crystal structure is of importance.
Total energy optimization calculations show that (see Fig.1(a)) the AlO2
crystal has a six-atom rectangular unit cell with the lattice parameter of a
(b) is 3.05 (4.84) Å. Here the optimized structure of AlO2 displays slightly
distorted 1T hexagonal lattice (namely 1T′-AlO2), where the central Al
subplane is surrounded by an octahedral type arrangement of O atoms. It worths
mentioning that 1T′-AlO2 displays ferromagnetic (FM) ground state with 1
$\mu_{B}$ per formula. Magnetic charge density distribution
($\rho_{\uparrow}$-$\rho_{\downarrow}$) shown in Fig. 1(b) reveals that the
ferromagnetic behavior stems from pz orbitals of O atoms. However, there is
also a possible antiferromagnetic (AFM) phase of the crystal structure which
is $\sim$40 meV less preferable than the FM one. Here cohesive energy of FM
and AFM phase is calculated to be 5.57 and 5.56 $eV/atom$, respectively. In
the AFM phase, that emerges as a result of the neighboring oxygens approaching
each other, ferromagnetically ordered AlO2 chains are formed and adjacent
chains prefer to be antiferromagnetically ordered with each other. Compared to
the FM phase, as a result of closer packing of AlO2 chains in the AFM phase,
there is a narrowing in the b lattice parameter and a slight increase in the
layer thickness (defined as the distance between the neighboring subplanes of
O atoms).

Bader charge analysis shows that crystal lattice is constructed with Al-O
bonds having ionic character, where the charge donation from Al to each O is
1.23 $e^{\text{-}}$ for both FM and AFM states. Additionally, the work
functions of the FM and AFM phases are calculated to be 8.01 and 7.96 eV,
respectively, which are significantly higher than that of single layer
structures of MoS2 (6.11 eV), WS2 (5.89 eV), MoSe2 (5.49 eV) and h-BN (4.7
eV).[36, 37] Apparently, such materials having high work function are
desirable for their use as charge extraction and injection layers in
electronic and optical devices in order to improve device capability.[38, 39]

For a discussion on resistance of the 2D crystal structure distortion under
mechanical load, determination of in-plane stiffness ($C$) and the Poisson
ratio ($\upsilon$) is of importance. Due to its anisotropic nature, the 1T′
phase of AlO2 possesses anisotropy by means of its elastic constants. Here,
the two elastic constants along with the zigzag ($zz$) and armchair ($arm$)
directions are calculated by using the equations[40]
$C_{zz}=\frac{1}{C_{22}}[C_{11}C_{22}-C_{12}^{2}]$ and
$C_{arm}=\frac{1}{C_{11}}[C_{11}C_{22}-C_{12}^{2}]$. Accordingly, in-plane
stiffness values of FM and AFM phases are found to be 126(138) and 108(123)
N/m for FM and AFM phases along the $zz$($arm$) directions, respectively.
Compared with the single layer Al2O3 (48 N/m)[11], AlO2 seems to be a much
stiffer material. On the other hand, as compared to single layer of AlN(116
N/m)[41], AlO2 phases are found to possess similar mechanical features in
terms of the in-plane stiffness.

The Poisson ratio, which is defined as the ratio of the transverse contraction
strain to the longitudinal extension, describes the mechanical response of a
material to the external load. The Poisson ratio through the $zz$ and $arm$
directions are calculated by using the equations[40]
$\upsilon_{zz}=\frac{C_{12}}{C_{22}}$ and
$\upsilon_{arm}=\frac{C_{12}}{C_{11}}$. It is known that for a stable linear
elastic material, the Poisson ratio varies between -1.0 and 0.5.[42] Here, the
Poisson ratios of FM (AFM) phases of 1T′-AlO2 are calculated to be 0.40 (0.33)
along $zz$ direction and 0.36 (0.37) along $arm$ direction. These values
indicate the soft nature of AlO2 when compared with those of the well-known 2D
structures, such as graphene and MoS2, for which the Poisson ratios are
reported to be 0.19 and 0.26, respectively.[19, 43]

The dynamical stability of 1T′-AlO2 in FM and AFM states is also examined
through their phonon band dispersions, which are obtained by utilizing the
small displacement method. Phonon dispersions depicted in Fig.2 show that all
the phonon branches have real eigenfrequencies through the whole Brillouin
zone and therefore both FM and AFM phases of 1T′-AlO2 crystal are dynamically
stable. It is seen that distorted crystal structure of 6-atom primitive unit
cell of the crystal structure leads to presence of 18 phonon branches. The
highest frequency optical phonon branch is found at 622 (625) cm-1 for the FM
(AFM) order which is lower than those single layer AlN ($\sim$ 900cm-1), Al2O3
(1500cm-1), and $\alpha$-AlO ($\sim$ 650cm-1).[41, 11, 15]

For further discussion of vibrational characteristics, zone-centered first-
order off-resonant Raman spectra of FM and AFM phases are also calculated. As
shown in Fig. 2(c-d), The Raman active modes labeled by numbers I-II, which
have the frequencies of 374.8 (401) cm-1 for FM (AFM) states, are attributed
to the shear-like vibration of upper and lower O atoms towards the Al atoms.
In addition, the phonon modes III-IV are calculated to be at the frequencies
477 (482) cm-1 for FM (AFM) states and are shown to arise from the shear-like
vibration of O atoms parallel to the Al atoms. Moreover, the Raman active
modes, V-VI having frequencies 582 (583) cm-1 for FM (AFM), represent the out-
of-plane vibration of upper and lower O atoms with respect to each other.
While the two magnetic phases are distinguishable in terms of the frequencies
of the Raman active modes, the calculated activities reveal their distinct
features. The Raman activities of the most prominent modes (I-II) are found to
be the same while the activities of III and V in the FM phase are found to be
lower as compared to that of IV and VI in the AFM state.

Figure 3: Electronic band structure of 1T′-AlO2 for (a) FM and (b) AFM phase.
Dashed and solid lines show the majority and minority spin states which are
degenerate in AFM state. (c) Comparative band alignments of 1T′-AlO2 phases
with MoS2, WSe2, InO and AlN single layers.

###  III.2 Electronic Properties

Electronic characteristics of FM and AFM phases of single layer 1T′-AlO2 are
investigated through their electronic band dispersions and the band decomposed
charge densities. As shown in Fig. 3(a), FM phase of AlO2 is a ferromagnetic
semiconductor with an indirect bandgap of 0.63 eV. As a consequence of the
large splitting in majority and minority states valence band maximum (VBM) and
conduction band minimum (CBM) of the FM phase is composed of the same spin
type. While the edge of the VBM reside at the Y-$\Gamma$ point, the CBM is
located at the Brillouin zone center. It is also seen that both VBM and CBM
band edge states have the same tilted O-$p_{z}$ character. As shown in Table
I, the hole effective mass at the vicinity of VBM is found to be 2.29 m0,
while the electron effective mass at the CBM is calculated to be 1.33 m0
(where m0 is electron mass).

In addition, AFM phase displays indirect semiconducting behavior with 1.18 eV
bandgap where the VBM and CBM are located at the X and the $\Gamma$ of the BZ,
respectively. It appears that VBM states stem from orbitals perpendicularly
oriented to the crystal surface leads to the formation of quite flat bands
around the X point and therefore relatively large hole effective mass for the
AFM state. Moreover, the hole effective mass is found to be anisotropic and
two different values are calculated along with the $\Gamma$-X and X-S
directions (1.20 and 18.34 m0, respectively) while the electron effective mass
at the vicinity of the CBM is found to be 8.52 m0. Apparently, lattice
contraction originating from the formation of AlO2 chains leads to the much
heavier charge carriers in the AFM phase.

Considering a nano device in which a 2D ultra-thin material is used as the
building block, it is clear that within the device architecture the material
will form interfaces with other materials that can be thought as
heterostructures. Here, we also investigate the vertical vdW heterostructures
of 1T′-AlO2 phases constructed with the 2D crystals such as MoS2, WSe2, InO,
and AlN. As presented in Fig. 3(c), as a consequence of their high
photoelectric threshold, energies of VBM and the CBM of both phases of 1T′AlO2
are much lower than that of the other materials considered. Therefore, it can
deduced that either FM- or AFM-phase of 1T′-AlO2 form type-III heterojunction
with single layers of MoS2, WSe2, InO, and AlN. Obviously, such type-III vdW
heterostructures might exhibit a negative differential resistance phenomenon
due to the existence of band-to-band tunneling as the electrons have a
tendency to tunnel from VBM of these crystal structures to the CBM of
1T′AlO2.[44, 45, 46]

Figure 4: The atomic structure, magnetization charge density and corresponding
electronic density of states of single oxygen defected 1T′-AlO2 in (a-b) FM
and (c-d) AFM phase. The missing atom is shown by dashed red circle.

###  III.3 Effect of Oxygen Vacancies

Whether the synthesis technique is a top-down or bottom-up approach, the
formation of point defects in two-dimensional crystals is inevitable. In order
to investigate the effect of atomic vacancies on the electronic properties of
single layer 1T′-AlO2, the formation of a single O vacancy, VO, is considered
and the defected crystal structure is fully optimized. In order to simulate
the single O vacancy, a supercell structure containing 96 atoms is considered
and an O atom is removed from the layer. After full optimization of the
O-vacant single layer of AlO2, it is shown that structural deformations around
the defect site are compensated by the local distortion of three Al atoms
which tend to form Al-dimers as the hole gets larger via O vacancy. While the
Al-Al distance is 2.84 Å in the host lattice, that of between the Al atoms
around the defect site is 2.75 Å as a result of the structural distortion. On
the other hand, by the formation of the vacancy, O atoms around the defect
site move towards the Al atoms surrounding the defect site and thus, the Al-O
bond lengths get smaller. It is also found that the formation of VO shrinks
the lattice by 0.4% along the $x$-direction while it locally enlarges the host
lattice by 0.3%.

The defect formation energy is calculated using the relation;
$E_{f}=[E_{tot}\text{(def)}+E_{\text{O}}]-E_{tot}\text{(pristine)}$ where
Etot(def) and Etot represents the total energies of the defected and the
pristine single layers, respectively while EO stands for the single O energy.
Here, the calculated defect formation energy of VO is 6.24 eV which indicates
that it is very difficult to subsequently create an O vacancy in the single
layer 1T′-AlO2.

Moreover, magnetic ground state calculations reveal that FM interaction is
still energetically more favorable in defected 1T′-AlO2. It is found that the
formation of O vacancy leads to a decrease in the total magnetic moment of the
structure (2$\mu_{B}$). Apparently, the creation of a single O vacancy induces
local magnetic moments which are altered by the deviations from the nominal
stoichiometry. The effect of single O vacancy on the electronic properties of
1T′-AlO2 is investigated in terms of the total electronic density of states.
Apparently, O vacancy induces midgap states that result in a transition to the
metallic state in the oxygen vacant domains of the 1T′-AlO2 crystal. As shown
in Fig. 4(c), similar to the FM phase, the presence of the oxygen vacancy does
not cause a remarkable change in the magnetic ground state when the crystal
structure is in the AFM phase. It is also seen that while the magnetic order
of the AFM remains unchanged, a magnetic moment of 2$\mu_{B}$ per oxygen
occurs because the balance between the sublattice atoms is disturbed due to
the missing oxygen atom. Additionally, as shown Fig. 4(d) formation of oxygen
vacancies also leads to a semiconductor-to-metal transition at the vicinity of
defect domains.

##  IV Conclusions

In conclusion, the first-principles calculations were performed on single
layer AlO2 and its structural, magnetic, vibrational and the electronic
properties were investigated. Total energy optimizations revealed that single
layer 1T′-AlO2 possesses two distinct magnetic phases, namely FM and AFM.
While the FM phase of 1T′-AlO2 was found to be energetically more favorable
over the AFM one, both phases are dynamically stable and they are
distinguishable in terms of their Raman spectra. Electronically, both FM- and
AFM-1T′-AlO2 structures were shown to be semiconductors with an indirect band
gap. It is also seen that 1T′-AlO2 crystals form a broken-gap type-III
heterojunction when combined with other graphene-like ultra-thin materials.
Moreover, we predicted that presence of oxygen vacancies yields semiconductor-
to-metal transition without altering the magnetic order in the crystal
structure. The ultra-thin 1T′-AlO2 with its semiconducting nature and robust
magnetic ground state unaffected by the presence of structural defects is a
promising material for nanoscale device applications.

##  V Acknowledgments

Computational resources were provided by TUBITAK ULAKBIM, High Performance and
Grid Computing Center (TR-Grid e-Infrastructure).

## References

  * Mavrič et al. [2018] Mavrič, A.; Fanetti, M.; Mali, G.; Valant, M. High-temperature stabilization of bulk amorphous Al2O3. _J. Non-Cryst. Solids_ 2018, _499_ , 363–370.
  * Yang et al. [2018] Yang, Y.; Kushima, A.; Han, W.; Xin, H.; Li, J. Liquid-like, self-healing aluminum oxide during deformation at room temperature. _Nano Lett._ 2018, _18_ , 2492–2497.
  * Peacock and Robertson [2002] Peacock, P.; Robertson, J. Band offsets and Schottky barrier heights of high dielectric constant oxides. _J. Appl. Phys._ 2002, _92_ , 4712–4721.
  * Vishvakarma et al. [2021] Vishvakarma, S.; Kannan, R.; Vashishtha, N.; Pal, A. S. Evaluation of hybrid sol-gel alumina coating for electrical resistance applications. _Surf. Topogr._ 2021, _9_ , 045030.
  * Kim et al. [2016] Kim, J.; Ha, J.-H.; Lee, J.; Song, I.-H. Optimization for permeability and electrical resistance of porous alumina-based ceramics. _J. Korean Ceram. Soc._ 2016, _53_ , 548–556.
  * Yanagishita et al. [2009] Yanagishita, T.; Nishio, K.; Masuda, H. Anti-reflection structures on lenses by nanoimprinting using ordered anodic porous alumina. _Appl. Phys. Express_ 2009, _2_ , 022001.
  * Wilt et al. [2018] Wilt, J.; Goul, R.; Acharya, J.; Sakidja, R.; Wu, J. Z. In situ atomic layer deposition and electron tunneling characterization of monolayer Al2O3 on Fe for magnetic tunnel junctions. _AIP Adv._ 2018, _8_ , 125218\.
  * Meng et al. [2014] Meng, F. C.; Liu, C.; Zhang, X. L.; Ren, H.; Jia, T. Synthesis and Characterization of $\gamma$-Al2O3 Nanowires by a Surfactant Assisted Precipitation Method. _Key Eng. Mater._ 2014, _602-603_ , 93 – 96.
  * Li et al. [2017] Li, Z.; Kong, L.; Huang, S.; Li, L. Highly luminescent and ultrastable CsPbBr3 perovskite quantum dots incorporated into a silica/alumina monolith. _Angew. Chem_ 2017, _129_ , 8246–8250.
  * Napari et al. [2021] Napari, M.; Huq, T.; Meeth, D.; Heikkilä, M.; Niang, K.; Wang, H.; Iivonen, T.; Wang, H.; Leskelä, M.; Ritala, M.; Flewitt, A.; Hoye, R.; MacManus-Driscoll, J. Role of ALD Al 2 O 3 Surface Passivation on the Performance of p-Type Cu2O Thin Film Transistors. _ACS Appl. Mater. Interfaces_ 2021, _13_ , 4156–4164.
  * Song et al. [2016] Song, T.; Yang, M.; Chai, J.; Callsen, M.; Zhou, J.; Yang, T.; Zhang, Z.; PAN, J.; Chi, D.; Feng, Y. P.; Wang, S. The stability of aluminium oxide monolayer and its interface with two-dimensional materials. _Sci. Rep._ 2016, _6_ , 29221.
  * Cheng et al. [2015] Cheng, G.; Tomczyk, M.; Lu, S.; Veazey, J.; Huang, M.; Irvin, P.; Ryu, S.; Lee, H.; Eom, C.-B.; Hellberg, C.; Levy, J. Electron pairing without superconductivity. _Nature (London)_ 2015, _521_ , 196–9.
  * Ghosh et al. [2020] Ghosh, A.; Kumar, A.; Roy, A.; Nguyen, C.; Ahuja, A.; Adil, M.; Chatti, M.; Kar, M.; MacFarlane, D. R.; Mitra, S. Ultrathin Lithium Aluminate Nanoflake-Inlaid Sulfur as a Cathode Material for Lithium–Sulfur Batteries with High Areal Capacity. _ACS Appl. Energy Mater._ 2020, _3_ , 5637–5645.
  * Li et al. [2015] Li, X.; Budai, J. D.; Liu, F.; Chen, Y.-S.; Howe, J. Y.; Sun, C.; Tischler, J. Z.; Meltzer, R. S.; Pan, Z. Crystal structures and optical properties of new quaternary strontium europium aluminate luminescent nanoribbons. _J. Mater. Chem. C_ 2015, _3_ , 778–788.
  * Demirci et al. [2017] Demirci, S.; Avazlı, N.; Durgun, E.; Cahangirov, S. Structural and electronic properties of monolayer group III monochalcogenides. _Phys. Rev. B_ 2017, _95_ , 115409.
  * Novoselov et al. [2004] Novoselov, K. S.; Geim, A. K.; Morozov, S. V.; Jiang, D.-e.; Zhang, Y.; Dubonos, S. V.; Grigorieva, I. V.; Firsov, A. A. Electric field effect in atomically thin carbon films. _Science_ 2004, _306_ , 666–669.
  * Geim [2009] Geim, A. K. Graphene: status and prospects. _Science_ 2009, _324_ , 1530–1534.
  * Neto et al. [2009] Neto, A. C.; Guinea, F.; Peres, N. M.; Novoselov, K. S.; Geim, A. K. The electronic properties of graphene. _Rev. Mod. Phys_ 2009, _81_ , 109.
  * Lee et al. [2008] Lee, C.; Wei, X.; Kysar, J. W.; Hone, J. Measurement of the elastic properties and intrinsic strength of monolayer graphene. _Science_ 2008, _321_ , 385–388.
  * Li and Chen [2016] Li, L. H.; Chen, Y. Atomically thin boron nitride: unique properties and applications. _Adv. Funct. Mater._ 2016, _26_ , 2594–2608.
  * Mak et al. [2010] Mak, K. F.; Lee, C.; Hone, J.; Shan, J.; Heinz, T. F. Atomically thin MoS2: a new direct-gap semiconductor. _Phys. Rev. Lett._ 2010, _105_ , 136805.
  * Wang et al. [2012] Wang, Q. H.; Kalantar-Zadeh, K.; Kis, A.; Coleman, J. N.; Strano, M. S. Electronics and optoelectronics of two-dimensional transition metal dichalcogenides. _Nat. Nanotechnol._ 2012, _7_ , 699–712.
  * Tongay et al. [2014] others,, et al. Monolayer behaviour in bulk ReS2 due to electronic and vibrational decoupling. _Nat. Commun._ 2014, _5_ , 1–6.
  * Cahangirov et al. [2009] Cahangirov, S.; Topsakal, M.; Aktürk, E.; Şahin, H.; Ciraci, S. Two-and one-dimensional honeycomb structures of silicon and germanium. _Phys. Rev. Lett._ 2009, _102_ , 236804.
  * Deng et al. [2018] Deng, Y.; Yu, Y.; Song, Y.; Zhang, J.; Wang, N. Z.; Sun, Z.; Yi, Y.; Wu, Y. Z.; Wu, S.; Zhu, J.; et al., Gate-tunable room-temperature ferromagnetism in two-dimensional Fe3GeTe2. _Nature (London)_ 2018, _563_ , 94–99.
  * Li et al. [2021] Li, F.; Ren, Y.; Wan, W.; Liu, Y.; Ge, Y. Strain tunable intrinsic ferromagnetic in 2D square CrBr2. _AIP Adv._ 2021, _11_ , 115220.
  * Xuhan et al. [2020] Xuhan, Z. et al. Atomically Thin 1T-FeCl2 Grown by Molecular-Beam Epitaxy. _J. Phys. Chem. C_ 2020, _124_ , 9416–9423.
  * Ceyhan et al. [2021] Ceyhan, E.; Yagmurcukardes, M.; Peeters, F.; Sahin, H. Electronic and magnetic properties of single-layer FeCl2 with defects. _Phys. Rev. B_ 2021, _103_ , 014106.
  * Grimme et al. [2010] Grimme, S.; Antony, J.; Ehrlich, S.; Krieg, H. A consistent and accurate ab initio parametrization of density functional dispersion correction (DFT-D) for the 94 elements H-Pu. _J. Chem. Phys._ 2010, _132_ , 154104\.
  * Kresse and Furthmüller [1996] Kresse, G.; Furthmüller, J. Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set. _Phys. Rev. B_ 1996, _54_ , 11169.
  * Kresse and Hafner [1993] Kresse, G.; Hafner, J. Ab initio molecular dynamics for liquid metals. _Phys. Rev. B_ 1993, _47_ , 558.
  * Perdew et al. [1996] Perdew, J. P.; Burke, K.; Ernzerhof, M. Generalized gradient approximation made simple. _Phys. Rev. Lett._ 1996, _77_ , 3865.
  * Blöchl [1994] Blöchl, P. E. Projector augmented-wave method. _Phys. Rev. B_ 1994, _50_ , 17953.
  * Henkelman et al. [2006] Henkelman, G.; Arnaldsson, A.; Jónsson, H. A fast and robust algorithm for Bader decomposition of charge density. _Comput. Mater. Sci._ 2006, _36_ , 354–360.
  * Togo and Tanaka [2015] Togo, A.; Tanaka, I. First principles phonon calculations in materials science. _Scr. Mater._ 2015, _108_ , 1–5.
  * Lanzillo et al. [2015] Lanzillo, N. A.; Simbeck, A. J.; Nayak, S. K. Strain engineering the work function in monolayer metal dichalcogenides. _J. Phys. Condens. Matter_ 2015, _27_ , 175501.
  * Pierucci et al. [2018] Pierucci, D.; Zribi, J.; Henck, H.; Chaste, J.; Silly, M.; Bertran, F.; Fevre, P.; Gil, B.; Summerfield, A.; Beton, P.; Novikov, S.; Cassabois, G.; Rault, J.; Ouerghi, A. Van der Waals epitaxy of two-dimensional single-layer h-BN on graphite by molecular beam epitaxy: Electronic properties and band structure. _Appl. Phys. Lett._ 2018, _112_ , 253102.
  * Bivour et al. [2015] Bivour, M.; Temmler, J.; Steinkemper, H.; Hermle, M. Molybdenum and tungsten oxide: High work function wide band gap contact materials for hole selective contacts of silicon solar cells. _Sol. Energy Mater. Sol. Cells_ 2015, _142_ , 34–41.
  * Schulz et al. [2016] Schulz, P.; Tiepelt, J. O.; Christians, J. A.; Levine, I.; Edri, E.; Sanehira, E. M.; Hodes, G.; Cahen, D.; Kahn, A. High-work-function molybdenum oxide hole extraction contacts in hybrid organic–inorganic perovskite solar cells. _ACS Appl. Mater. Interfaces_ 2016, _8_ , 31491–31499.
  * Ma et al. [2018] Ma, Y.; Kou, L.; Huang, B.; Dai, Y.; Heine, T. Two-dimensional ferroelastic topological insulators in single-layer Janus transition metal dichalcogenides M SSe (M= Mo, W). _Phys. Rev. B_ 2018, _98_ , 085420.
  * Şahin et al. [2009] Şahin, H.; Cahangirov, S.; Topsakal, M.; Bekaroglu, E.; Akturk, E.; Senger, R. T.; Ciraci, S. Monolayer honeycomb structures of group-IV elements and III-V binary compounds: First-principles calculations. _Phys. Rev. B_ 2009, _80_ , 155453.
  * Gercek [2007] Gercek, H. Poisson’s ratio values for rocks. _Int. J. Rock Mech. Min. Sci._ 2007, _44_ , 1–13.
  * Yagmurcukardes et al. [2016] Yagmurcukardes, M.; Senger, R. T.; Peeters, F. M.; Sahin, H. Mechanical properties of monolayer GaS and GaSe crystals. _Phys. Rev. B_ 2016, _94_ , 245407.
  * Ionescu and Riel [2011] Ionescu, A. M.; Riel, H. Tunnel field-effect transistors as energy-efficient electronic switches. _Nature (London)_ 2011, _479_ , 329–337.
  * Lu and Seabaugh [2014] Lu, H.; Seabaugh, A. Tunnel field-effect transistors: State-of-the-art. _IEEE J. Electron Devices Soc._ 2014, _2_ , 44–49.
  * Xia et al. [2018] Xia, C.; Du, J.; Li, M.; Li, X.; Zhao, X.; Wang, T.; Li, J. Effects of Electric Field on the Electronic Structures of Broken-gap Phosphorene/SnX2 (X= S, Se) van der Waals Heterojunctions. _Phys. Rev. Appl._ 2018, _10_ , 054064.


VAL: Interactive Task Learning with GPT Dialog Parsing

Lane Lawley

Christopher J. MacLellan

Georgia Institute of Technology
85 5th St. NW

Reinforcement learning often requires millions of examples to produce static, black-box models. In contrast, interactive task learning (ITL) emphasizes incremental knowledge acquisition from limited instruction provided by humans in modalities such as natural language. However, in practice, ITL systems often suffers from brittle, error-prone language parsing. Large language models (LLMs) are resistant to brittleness but are not interpretable and cannot learn incrementally. We present VAL, an ITL system with a new philosophy for LLM/symbolic integration. By using LLMs only for specific tasks—such as predicate and argument selection—within an algorithmic framework, VAL reaps the benefits of LLMs to support interactive learning of hierarchical task knowledge from natural language. Acquired knowledge is human interpretable and generalizes to support execution of novel tasks without additional training. We studied users' interactions with VAL in a video game setting, finding that most users could successfully teach VAL using language they felt was natural.

Many existing methods for teaching machines to perform tasks are based on reinforcement learning (RL), often requiring millions of examples to produce static, black-box behavioral policies.
In contrast, the interactive task learning (ITL) framework emphasizes incremental, few-data learning via interaction with human tutors in modalities such as natural language.
However, much existing work on ITL in the language modality suffers from brittle, error-prone language parsing techniques.
Large language models (LLMs) are more resistant to this brittleness, but cannot reliably form structured concepts, ground their output in the world around them, or update their own knowledge incrementally.

Here, we present VAL, an interactive task learning system based on a new philosophy of LLM/symbolic integration.
VAL learns by mapping natural language instructions to symbolic hierarchical task representations.
VAL can then generalize and apply the learned HTNs to execute tasks in novel settings without additional examples.
By using LLMs only for extremely specific tasks—such as predicate mapping, argument selection, and and paraphrase checking—within a primarily algorithmic framework, VAL reaps the linguistic benefits of LLMs to enable natural instructional dialogs while still learning interpretable, reliable knowledge structures.

We studied user interactions with VAL in a video game task instruction setting, and observed that most users were able to successfully teach VAL and felt their interactions with VAL were natural.[An alternative abstract, using only the 1,000 most common English words, is presented in Appendix <ref>.]

<concept_desc>Computing methodologies Discourse, dialogue and pragmatics</concept_desc>
<concept_desc>Human-centered computing Text input</concept_desc>
<concept_desc>Computing methodologies Planning with abstraction and generalization</concept_desc>
<concept_desc>Computing methodologies Information extraction</concept_desc>
<concept_desc>Human-centered computing User interface programming</concept_desc>

[500]Computing methodologies Discourse, dialogue and pragmatics
[500]Human-centered computing Text input
[300]Computing methodologies Planning with abstraction and generalization
[300]Computing methodologies Information extraction
[300]Human-centered computing User interface programming

An illustration of one iteration of the basic VAL learning procedure. A more comprehensive process description is given by Algorithm <ref> and Section <ref>.
A cartoonish flowchart illustration of a human being instructing the VAL system, which asks a clarifying question after processing the input through several targeted GPT prompts.

14 September 2023
[revised]at some point after that, probably
[accepted]soon after revision, hopefully! :)

§ INTRODUCTION

The Interactive Task Learning (ITL) approach introduced by <cit.> articulates a research vision dedicated to the goal of enabling machines to interactively learn general tasks in natural, human-like ways. It posits that acquired task knowledge should be general enough to be applicable in novel situations, and that it should be interpretable and modifiable, so that human teachers can actively shape the machine's understanding. ITL highlights many instructional modalities for the acquisition of task knowledge, including gestures, diagrams, and language—these modalities are also formally accounted for in the Natural Training Interaction (NTI) framework due to <cit.>. In this work, we focus on the language modality, endeavoring toward acquisition of human-interpetable, modular, hierarchical task knowledge from natural dialogs between humans and machines.

An example dialog with VAL. The current dialog state is a confirmatory prompt for the text segmentation step performed by one of VAL's GPT subroutines. This step performs action discretization, anaphora resolution (of “in there”), and temporal ordering.
An example dialog with VAL. The user asks VAL to “turn the pot on after putting an onion in there”, and VAL responds with the broken down steps of the command.

There has been considerable research into language-driven task acquisition, which we detail further in Section <ref>. Each of these systems, however, suffers the characteristic brittleness of the classical syntactic and semantic parsers they invariably use to convert natural instructional dialogs into task knowledge. Syntactic forms vastly outnumber their semantic counterparts, and problems like anaphora resolution, paraphrasing, and grammatical and transcription errors frequently claim the “natural” in “natural language” as their first casualty. These systems not only fall subject to myriad parser errors on malformed input, but also lack the ability to perform adequate paraphrase resolution when parsed verb structures do not match known ones exactly, generally relying on manually constructed knowledge bases of synonyms to solve even a portion of this problem.
These issues make it unlikely that these systems will successfully achieve the ITL vision with real users—currently, they only work with users in the lab that know their idiosyncratic syntax.

Here, we present VAL, the Verbal Apprentice Learner, an alternate strategy for mitigating the sheer variety of natural language. By exploiting the virtual mastery of the form of natural English apparently acquired by large language models <cit.>, we demonstrate the acquisition of task knowledge in a far less restrictive, far more natural dialog setting than possible in prior work. We integrate a GPT-family language model <cit.> in a careful and principled way, using it to map natural dialog into a symbolic domain.
The GPT model is used for one of a limited number of sub-tasks: the semantic parsing of text into predicate-argument structures, the semantic unification of those structures to already-known actions with the same meaning, and other narrowly scoped procedures enumerated in Section <ref>.
The linguistic flexibility provided by GPT allows VAL to deal with error-laden and grammatically unrestricted input dialog, and to perform paraphrasal mappings that take into account meaning; together, these applications of GPT vastly widen the set of possible inputs to the task learning system with minimal upfront engineering cost.

We feel VAL is uniquely suitable for task knowledge acquisition: unlike LLMs alone, VAL can learn long-term, re-usable task structures with just a few examples; and unlike hierarchical task networks alone, VAL offers a rich natural language interface to guide task execution and learning. We provide evidence of VAL's usability collected from user studies, analyzing both subjective responses and objective signals.

§.§ Key Contributions

This paper makes the following key contributions:

* A neuro-symbolic hybrid approach to interactive task learning (Section <ref>) in which:

* a classical learning algorithm is augmented with subroutines implemented as highly specific GPT prompts, and;

* unknown actions are recursively reduced to known ones via natural dialog.

* The VAL system (Section <ref>), an implementation of this approach with an emphasis on mitigating “cascading errors” through confirmations and undo operations.

* A user study (Section <ref>) of VAL's usability and efficacy in a video game task learning environment, evaluating both user-reported experience and objective measures of efficacy.

§ RELATED WORK

Past work on ITL using a verbal modality has almost exclusively mapped language to semantics using parsers that are either entirely classical, such as dependency parsers, or utilize a mix of classical and neural methods. We describe some of this past work here, in each case highlighting differences from our approach.

M: The game is tic-tac-toe.
ßR: I don't know that game, how many players are there?ß
M: Two.
ßR: Please start by teaching me the name of a legal action in the game.ß
M: Place
ßR: What are the verb and parameter arguments associated with this action?ß
M: move 1 to 2

A conversation with the Rosie system <cit.> demonstrating the rigid nature of the interaction. M denotes a user message, and R denotes a message from Rosie.
A conversation with Rosie, another task learning system. The user and Rosie go back and forth using less-than-natural language.

Early work with Instructo-Soar <cit.> investigated how to acquire hierarchical task descriptions for use in the Soar cognitive architecture <cit.>.
A decade later, this approach has evolved into Rosie <cit.>, a system that can learn game and robotics task knowledge from language instruction.
However, as <cit.> point out, Rosie's language understanding facility is limited to a “restricted subset” of English, limiting user interactions and potentially requiring user training in what Rosie can understand. An example can be seen in Figure <ref>, in which Rosie asks for very limited answers to very esoteric questions, e.g., “What are the verb and parameter arguments associated with this action?”. We seek to remedy this, making interactions with our agent more natural, and requiring less training and mental modeling on the part of the human interlocutor.

PUMICE <cit.> uses a "floating parser" design <cit.>, which, while providing a degree of grammatical flexibility, is still bound to exact lexical values and orderings; synonyms, paraphrases, idioms, etc. require manual engineering to overcome. PUMICE, like our system, uses recursive clarification for task learning, and also introduces a demonstration modality, which we have not yet incorporated.

Building on PUMICE, ONYX <cit.> is a task teaching system situated primarily in a data visualization application. Users teach ONYX to perform data visualization operations in that environment utilizing a mix of demonstration and natural language instruction. The ONYX natural language modality is capable of handling vague or misunderstood instructions by prompting the user to perform a demonstration instead. ONYX uses Microsoft's LUIS language understanding service <cit.> to perform intent classification, but in understanding the content of the request, it uses a bigram dependency kernel <cit.> based on a classical dependency parse. ONYX requires a pre-constructed natural language lexicon. While we seek to improve on ONYX's natural language understanding component, we do not yet target the demonstration modality which ONYX uses as a fallback for language misunderstanding.

<cit.> introduce a system similar to ours, in which hierarchical task structures are acquired and generalized from dialog via recursive clarification. Their system also automatically learns preconditions and effects for actions, which our system does not yet. However, their use of an OpenCCG parser and manually constructed, domain-specific lexicon introduce the same dialog limitations as most other language-based ITL systems.

VAL's niche is linguistic flexibility; while these other systems include features we do wish to add to VAL, like condition learning and additional instructional modalities, we seek here mainly to establish that the careful use of large language models in dialog understanding presents a much more flexible and holistic alternative to the complex and brittle parsing and grounding systems in common use today. Our implementation of VAL also provides, and studies, user-centric features of task learning, such as an undo button and a real-time display of currently known procedures.

§ VAL SYSTEM DESIGN

U: Cook an onion.
V: How do I cook an onion?
U: First, get an onion. Then, put it in the pot and turn it on.
V: How do I get an onion?
U: Go to the onion and hit space.

An excerpt of an interaction with VAL to teach the plan shown in Figure <ref>. U denotes a user message, and V denotes a VAL message.
A conversation with VAL. The user and VAL discuss how to cook an onion; the user breaks the task down into steps, which VAL begins to recursively seek clarification for.

VAL acquires hierarchical task knowledge by exploiting the hierarchical nature of clarification in dialog. Users give VAL commands, and when VAL is unable to relate that command to existing task knowledge, it asks the user to define the steps of the command. This process continues recursively until all tasks have been “grounded out” in terms of known actions, initially comprising a set of environment-dependent primitive actions.

To facilitate natural language as a medium for these dialogs, VAL uses an array of GPT-powered subroutines to perform tasks like action segmentation, predicate selection, and argument grounding. However, these subroutines are largely translational between the user input and the underlying symbolic knowledge structures; the overall task learning algorithm itself is not executed by GPT, and when VAL acts in the environment, or when users want to audit VAL's knowledge, the black-box GPT subroutines are not involved.

While VAL's recursive task acquisition mechanism is inspired by prior work, this synthesis of GPT with the symbolic task acquisition process is its main contribution, and hopefully serves as another step toward truly general neuro-symbolic hybrid systems.

A high-level diagram of VAL's components: the GPT subroutines (Section <ref>), the main “VALgorithm” (Section <ref>), the HTN knowledge base (Section <ref>), and, outside of VAL, an example environment from our user study (Section <ref>).
An architectural diagram of VAL's main components. A central interaction manager interfaces with GPT and with an HTN knowledge base, as well as with the external environment.

§.§ Knowledge Representation

An example HTN plan for cooking, learned by VAL in the Overcooked-AI environment. The learned task  decomposes into other learned tasks, which themselves decompose into primitive actions.
An example HTN plan for cooking, learned by VAL. The cook action breaks down into get, put, and turnOn actions, each of which break down further into the atomic commands moveTo and pressSpace.

VAL's task knowledge representation is based on the Hierarchical Task Network (HTN) formalism <cit.>, a tree structure in which tasks decompose into sequences of subtasks, each task contains preconditions and effects, and leaves are “primitives”  given as part of the environment. HTNs are widely applicable to planning, and have been used in many domains, from video games <cit.> to user preference modeling <cit.> to transport logistics <cit.>.

An example of a VAL HTN in the cooking video game domain of our user study is shown in Figure <ref>. While the general HTN formalism includes precondition and effect modeling, this study only focuses on evaluating the acquisition of non-conditional hierarchical task structures from dialog.

§.§ Hierarchical Task Acquisition

Tasks are conveyed from the user to VAL starting with an initial command provided by the user. The command can be an action or a sequence of multiple actions. All actions represented in the command are matched to corresponding predicate-argument action structures already learned by VAL, including the initial primitive actions. If any actions in a command cannot be matched to semantically similar known actions, they are recursively clarified with new subcommands solicited from the user; the recursive clarification of actions into sequences of new actions forms the hierarchical structure of the learned task. One step of this process is depicted in Figure <ref>, and the full procedure is described in Algorithm <ref>.

Once tasks are acquired, they are added to VAL's knowledge base and can be chosen and executed for future requests. These requests fill in their generalized values and traverse the task tree to the leaves to send primitive actions to the environment. For more information on this process, see Section <ref>.

§.§ Plan Generalization

When new plan knowledge is acquired, a GPT subroutine is used to determine which arguments to generalize and which to hold constant. This means that learned tasks can be reused, e.g., after being told how to cook an onion in the Overcooked video game environment (see Section <ref>), VAL should be able to cook a tomato without seeking further clarification. The generalization component, , is described in Section <ref>.

§.§ GPT Integration Philosophy

VAL is an AI system, rather than just a model: it comprises a combination of GPT components, a classical learning algorithm, and connecting code to interface with a given environment. We use GPT models to solve atomic, linguistic subroutines within the well-defined “VALgorithm”. This allows us to obtain more focused, reliable output than if we had used language models to solve the entire task end-to-end. It also allows us to perform better error correction on a more predictable set of error classes. When language models alone are used for this task, without the entire VAL architecture, they frequently invent new actions not yet learned, or attempt to fabricate non-applicable domain knowledge, such as washing a knife in the Overcooked video game despite the game lacking any washing operations.

The tasks performed by the language model here, though, are by no means trivial; extraction of predicate-argument structures from text, and the mapping of those structures to known actions, both require resolution of a considerable number of linguistic tasks, such as syntactic parsing, anaphora resolution, paraphrase resolution, action segmentation, and more; the full set of tasks performed by GPT in VAL is detailed in Section <ref>.

* Syntactic parsing, a necessary step of semantic parsing in other dialog-based instructional systems, is a challenging task that is highly prone to grammatical and spelling errors, often requiring significant engineering effort to overcome them. GPT's virtual mastery of the form of natural language, even when that language may be ungrammatical, allows us to avoid most of this engineering effort.

* Anaphora resolution, one of several tasks performed by segmentGPT , allows constructions like Pick up the pepper and put it on the counter to decompose into unambiguous structures like  and .

* Predicate naming, performed by nameGPT , is important in distinguishing predicates with similar verbs, as verb senses may often include mandatory prepositional arguments, e.g.,  vs. .

* Paraphrase resolution, performed by paraphraseGPT , is one of the most challenging aspects of unifying natural speech with formal knowledge representations. While other task learning systems struggle with the sheer phrasal variance of natural language, GPT autonomously makes these determinations, such as identifying that both pull your hand back and bring it back to the original position invoke the known action .

Our integration of GPT into an otherwise classical algorithm has allowed us to capitalize on the fluency of large language models without falling victim to the myriad failures that can arise from using them to complete non-linguistic tasks end-to-end—especially recursive tasks and tasks requiring well-defined output.

§.§.§ Programmatic Scolding

As we developed VAL, we moved between two language model bases for the GPT components:  and . These models frequently exhibit overly cautious behavior: for example,  will often refuse to choose cooking-related actions for cooking-related inputs in the Overcooked game setting, citing its lack of physical body as an AI language model as a reason for its inability to cook food.

This was quite frustrating, and given the black-box nature of the models and the great influence of reinforcement learning from human feedback on their training <cit.>, stymied us for some time. Eventually, we settled on a silly but effective solution we deem programmatic scolding. In programmatic scolding, responses containing apology words or the string as an AI language model are automatically detected and responded to with a message scolding the language model for being culturally insensitive in its refusal to answer. We found these additional scolding prompts would lead the language model to produce the desired output. Literally scolding the model to produce well defined outputs is not a desirable research strategy, but underscores the difficulty of controlling LLM outputs.

§ VAL SYSTEM ARCHITECTURE

§.§ The VALgorithm

Like in the hierarchical task acquisition work by <cit.>, VAL acquires tasks from natural language descriptions by grounding actions and arguments and recursively seeking new definitions for any unknown actions in terms of known ones. However, aside from the GPT subroutines that make VAL's implementation of these actions unique, the actual VAL algorithm (“VALgorithm”) differs in its own ways, and so we describe it here, and in the pseudocode of Algorithm <ref>.

VAL begins by segmenting the input into individual, temporally ordered steps. Each of these steps is then mapped independently to known actions. If a suitable match is found, arguments are selected to fill its slots. The filled predicate-argument representation is then verbalized into a sentence, and that sentence is compared with the initial input segment; if they are determined to be paraphrases of one another, the mapped action and filled argument slots are used. However, if the matched predicate is deemed too semantically different from the input segment after matching, or if no match is initially found, then a definition is sought for it, a name is created for the new action, and a subset of the used arguments are selected as the variable arguments for the new action.

Each purple box   in Algorithm <ref> represents a GPT subroutine, each of which is defined below, in Section <ref>.

§.§ GPT Subroutines

While the VALgorithm is straightforward and interpretable on its face, its subroutines encapsulate much of the complexity inherent to any natural language understanding task. Below, we enumerate and describe each of the GPT subroutines essential to VAL's ability to understand naturally phrased instructions and relate them to its symbolic models of the environment and of actions within it. We implemented each of these subroutines as a prompt, occasionally with a set of examples; we did not use fine-tuning for any GPT subroutines.

splits a command into atomic steps. Because each atomic step will be matched individually to known actions, co-references must be resolved as well, so that pronouns in later steps are replaced with their referents.  also expands repeated actions, such as those with adverbs like “twice”, into their own steps, and orders the steps chronologically, resolving temporal relations like “before” and “after” in the input.

selects a known action corresponding to an input segment produced by segmentGPT, or determines that no known action is suitable. In the latter case, VAL will attempt to acquire a new action (or, if confirmatory dialogs are enabled, ask if a suitable matching action exists).  does not jointly select arguments for the chosen action; that subsequent task is factored out and performed by .

selects from a list of known world objects to fill argument slots for a predicate chosen by .  also has access to the original text segment.  must correctly interpret plurals, restrictors, and other quantifiers in the input: for example, a single noun phrase like “the numerators” may expand to fill two argument slots, or “any red vegetable” may be used to select “tomato”.  only selects arguments when a known action has been chosen first; to choose a set of arguments for a newly introduced action, VAL uses .

§.§.§  and

is used in tandem with  to verify matches before committing to them; occasionally,  will select an action with an inexact semantic match, such as the selection of the action  for the input segment “get an onion”. The reasons for this are not totally clear[However, when we extend the  prompt to a conversational setting with ChatGPT and ask why it chose what it did, it will often justify it by saying things like “in order to get an onion, you must first go to it”. It is unclear, though, whether this reasoning informed its initial choice or is merely a consequence of needing to justify it retrospectively.], but we find that these incorrect choices can often be avoided by the following three-step process:

* Verbalize the action and arguments chosen by  and  into a natural language sentence using .

* Compare that sentence to the original user input segment using .

* Accept the match if and only if  deems them a match.

When confirmatory dialogs are enabled,  and  are not used, as the user provides suitability information directly.

produces a predicate name for the original text snippet responsible that led to the currently unnamed task being learned.

If  produces a name identical to a known name, progressively higher numbers are appended, beginning with 2; however, this is a shallow solution to a relatively deep problem, as we will discuss in Section <ref>.

selects arguments for newly learned actions after  decides on a predicate name for them. It is only allowed to choose from the set of arguments that were used in the full expansion of the task. We use GPT to filter this set because selecting which arguments should be generalized and which should remain constant is a task that requires both lexical and semantic knowledge. However,  occasionally omits an important argument, leaving it constant; or includes an argument that will never vary, such as the pot in the  action.

§ E“VAL”UATION

A screenshot of the Overcooked-AI game environment used in VAL's user studies.
A screenshot of the Overcooked-AI game environment used in VAL's user studies. Two cartoon sprites of chefs sit in kitchen-like environments, with vegetables and cooking tools available to them.

To evaluate VAL, we conducted an IRB-approved user study, in which we recruited 12 participants to teach VAL a task in a video game environment and take a survey regarding their experience. This study had two aims:

* to evaluate the participants' subjective experiences via the survey, and

* to evaluate the objective performance of VAL's components via signals recorded both from the interaction logs and from VAL's implementation.

In the latter case, we largely relied on confirmatory dialogs (see Section <ref>), via which users either approve or correct decisions made by VAL's components, as a source of ground truth for measuring component success. We also measured signals for in-game actions successfully completed by VAL, instances of users clicking “undo”, and any system crashes. We report and analyze the survey results in Section <ref>, and all other results in Section <ref>.

§.§ Environment

For our user study environment, we chose Overcooked-AI <cit.>, an open-source, simplified implementation of the popular video game Overcooked <cit.>.
An increasing amount of contemporary task learning and task performance work has focused on this environment <cit.>.
After showing each user a tutorial for the game and allowing them to practice the task manually, we asked them to teach VAL to make onion soup with one onion, which involves putting the onion in the pot, turning the pot on, fetching a plate, interacting with the pot while holding the plate, and taking the plate to the drop-off location.

§.§.§ Primitive Actions

As initial primitive actions, we implemented  and . While the latter straightforwardly lives up to its name, we chose to implement the former as a breadth-first search within the environment; while work has been done on HTN representations of pathfinding <cit.>, we decided that verbal acquisition of such networks would be too complex for average users to teach in an initial study. The selection of primitives is an important question in integrating VAL with a new environment; it is important to find a balance between ease of implementation and ease of teaching. In this case, we felt that the task of making soup remained suitably interesting with low-level movement primitives introduced.

Two confirmatory dialog examples for segmentGPT   (left) and genGPT   (right). Examples of all types of confirmatory dialog are available in Appendix <ref>.
Some example confirmatory dialogs used in the VAL user study. VAL asks questions to confirm its GPT subroutine results, and the user responds “yes” or “no”, optionally providing a manual correction in the event they select “no”.

§.§ Study Design Features

§.§.§ Confirmatory Dialogs

Interactive task learning is a sequential process in which an early error can compromise all downstream steps and the final result. Furthermore, we required a way to evaluate the performance of VAL's individual GPT subroutines relative to human judgment. To address both of these issues, the VAL user study makes use of confirmatory dialogs for GPT subroutine results. These prompt the user to either confirm or correct the result returned by the subroutine. Examples of confirmatory dialogs can be seen in Figure <ref> and Appendix <ref>.

As these dialogs interrupt the flow of the interaction and may frustrate users with their frequency and repetitiveness, it would be ideal not to ever require them in a non-study setting. In Section <ref>, we discuss a strategy that seems to reduce the need for these dialogs in one component, and our plans to pursue similar strategies for other components.

The VAL dialog interface was adapted from the Simple Chat UI system developed by Sajad Hashemian <cit.>. We contributed the design and implementation of the confirmatory dialogs within the Simple Chat UI message bubbles, which contain only text by default.

§.§.§ Knowledge Display

The knowledge display (see Section <ref>) shown on the study page. In this example,  and  are primitive actions, and  is a learned action taught by the user.
An enumeration of three currently known actions from a VAL interaction: moveTo, pressSpace, and get.

At all times, a knowledge display box (see Figure <ref>) was visible on the study page, showing which generalized actions had been learned by VAL, including both primitive and learned actions.

§.§.§ Real-Time Action Performance

As VAL grounds the task out in terms of primitive actions, a decision must be made as to when to send each primitive action to the game environment.
In early testing, users found it confusing to wait until VAL had learned an entire complex task tree before it took any actions in the game.
Therefore, we opt to send each primitive as it's added to the tree, rather than sending all primitives once the tree has been completed.
This allows the user to monitor the task progress during instruction and confirm that the right actions are being carried out.

§.§.§ The Undo Button

To further enable error correction during user sessions, we implemented an undo button in the VAL dialog interface, which is visible in Figure <ref>.
David R. Hill <cit.> offered the guidance that, in a dialog setting with external effects, what should be undoable is each effect, rather than each message.[Prof. Hill also noted that an undo feature “takes a lot of effort to implement”, which finding is corroborated by our work.]
In a VAL session, all modification of game or agent knowledge state is a reaction to a user input, whether that be a command or a confirmatory dialog response.
Therefore, when the undo button is clicked, we opt to rewind both the game and agent to their states at the previous prompt for user input.

§.§ Experimental Setup

We recruited 12 participants by word of mouth from university communities.
After signing consent forms, they were given two static webpage tutorials: one for making soup in the Overcooked-AI environment, and one for teaching an unrelated task to VAL.
In between the two tutorials, they were also asked to practice making soup manually in the game environment.
After completing both tutorials, the participants were asked to instruct VAL to make onion soup, and then presented with a side-by-side view of a VAL dialog (e.g., Figure <ref>), a display of VAL's knowledge (see Figure <ref>), and the game (e.g., Figure <ref>).
The participants were informed that they could end the study at any time, regardless of progress, and proceed to the survey.

The post-session survey asked non-identifiable questions about relevant prior experience, seven-point Likert scale opinion questions, and open-ended short-answer opinion questions. The full survey form is attached as Appendix <ref>, and a link to the full, anonymized set of results, including short-answer responses, will be provided if and when this paper is no longer under blind review.

§.§ User Survey Results

Participant Likert scale scores for the questions asked in our post-study survey. Red/left is bad, and blue/right is good. Tick marks are sometimes aligned at half-intervals due to neutral responses straddling the center line.
A representation of the user-provided Likert scores for usability questions in the VAL user study. The figure shows that the majority of responses indicated positive sentiments regarding VAL's usability.

§.§.§ Prior Experience

We asked each participant about their prior experience levels with interaction with large language models, computer programming, and video games like Overcooked-AI. All participants had at least “a lot” of experience with programming; most participants only had “a little” experience interacting with large language models; and most participants had at least “a moderate amount” of experience with video games like Overcooked-AI. The full results for these questions are presented in Appendix <ref>.

§.§.§ Opinions on Sessions

The questions we asked participants about their sessions were derived in part from prior work on task learning system usability, although some were original to this work.

While some questions are inspired by prior work, the questions are highly subjective and situational, and so responses cannot be directly compared as if they were on the same scale.

The prompts we gave, and any applicable works inspiring those questions, were:

* VAL was able to correctly perform what I asked it to <cit.>

* I found the display of VAL's current knowledge easy to understand <cit.>

* VAL processed my explanations quickly <cit.>

* I found the display of VAL's current knowledge useful in guiding the interaction <cit.>

* I found it easy to learn to use VAL <cit.>

* I found my entire experience teaching VAL to be straightforward

* I felt VAL understood what I said to it

* VAL learned the task without me needing to correct it often

The responses are visualized in Figure <ref>.
For the most part, user experience was positive; all questions had more agree-type responses than disagree-type responses.
In keeping with the observed game milestone signals, almost all participants reported being able to complete the task.
The worst responses were to the prompts VAL learned the task without me needing to correct it often and I felt VAL understood what I said to it; this highlights the need for the confirmatory dialogs found in the study, although finding ways to reduce them may still be beneficial.
In fact, the high number of confirmatory dialogs may have contributed to the slightly higher proportion of negative scores for I found my entire experience teaching VAL to be straightforward as well.

§.§ System Metric Results

GPT Subroutine     Success Rate

segmentGPT      [l]93% user approval

mapGPT      [l]82% user approval ()
97% user approval ()

groundGPT      [l]88% user approval

genGPT      [l]81% user approval

verbalizeGPT  + paraphraseGPT     [l]79% true positive rate
99% true negative rate

GPT subroutine success rates across all the times they were activated within the twelve user sessions.
A table displaying success rates for the GPT subroutines used by VAL.

§.§.§ GPT Subroutine Success Rates

Table <ref> shows the success rates for four of the seven GPT subroutines described in Section <ref>. We did not measure the success of , as predicate name choice was not subject to user approval or disapproval in our study. However, there is clear evidence from question (2) on the opinion survey that users found the display of VAL's current knowledge easy to understand. As the names of the learned tasks in the display were the output of nameGPT, this suggests its outputs are interpretable and in alignment with user's expectations.

The success rates of  and  are not independently defined, as these two components work together to filter spurious matches produced by . As the design of the study required manual user approval for  decisions, these two components were unused. However, we calculated the decisions that would have been made according to these two components, and compared them to the user decisions that were actually made. We found a false negative rate of 21% and a false positive rate of 1% relative to the user-provided ground truth. The low false positive rate suggests that roughly four fifths of confirmatory dialogs for  outputs may be avoidable, with minimal impact to performance, by simply always accepting negative paraphrase decisions made by these two components.

§.§.§ Programmatic Scolding Occurrences

Programmatic scolding (see Section <ref>) was triggered a total of 29 times, split across three participants. All 29 scolds occurred in the  participant group.

§.§.§ Game Milestone Signals

During the sessions, we recorded signals from the game engine corresponding to certain “milestone” events: picking up an onion, putting an onion in the pot, turning the pot on, plating the cooked soup, and delivering the soup.

Of 12 total participants, nine were able to instruct VAL to complete every milestone; two were able to instruct VAL to cook the soup, but not to plate or deliver it; and one participant was unable to complete any milestones.

§.§.§ Undos

10 of 12 participants used the undo feature. The median number of undos was 6, and the mean was 9.

§.§.§ System Crashes

Of the three participants who did not signal all five milestones, two experienced crashes in the VAL system that halted all progress. One participant experienced a crash after completing all milestones.

The root cause of two of these three crashes was a malformed string returned by a GPT subroutine. The third crash was due to a platform-specific string encoding issue in the experimental environment that was not observed in the development environment.

§.§.§ Choice of Base GPT Model

Any autoregressive language model can be used to implement VAL's GPT-based subroutines. Of our 12 participants, seven interacted with a -based VAL system, and the other five interacted with a -based one.

The switch to GPT-4 had a statistically significant ($p \le .05$) effect[p-values were calculated using $t$-tests for all metrics except approve-reject decisions, for which we used binary $\chi^{2}$ tests.] on:

* the frequency of programmatic scolding (decreased; good)[In fact, there was no scolding at all in the -based sessions.]

* the number of crashes (decreased; good),

* the success rate of  (increased; good),

* the user-perceived processing speed (increased; bad),

* the ease of learning to use VAL (increased; good), and

* the straightforwardness of the entire experience (increased; good).

The other categories tested for significance were:

* the number of messages sent by the user per completed milestone,

* the false positive and false negative rates of  filtering when compared user decisions,

* the number of clicks of the “undo” button,

* all GPT subroutines other than , and

* all of the other questions in the survey.

§.§ Discussion

This section reflects on the quantitative results above and on the short-answer responses given in the survey. We break the discussion into themes digested from these results.

§.§.§ Naturalness of Interactions

Most participants responded that they felt empowered to use natural phrasing with VAL, and that VAL largely understood what they meant. However, some reported issues: two participants told VAL “I want to teach you to ”, only for VAL to interpret the utterance as a request to perform a “teach” action. One participant also felt that VAL needed explanations at too high a granularity; while we expect that, in repeated uses of the same VAL model, many actions could be re-used to reduce this effect, it is true that VAL starts only with its primitives, and requires explicit explanations for any other actions. It may be worth exploring ways to perform environmental search, or to use language models, to provide initial, non-binding suggestions for how to implement some higher-level actions, in order to reduce the burden on the user.

§.§.§ Theory of Mind

Several users reported that they felt unsure of what exactly VAL modeled about the world, and of what operations it could perform on that model. For example, one user asked VAL to edit one of the steps in its current plan, which VAL cannot currently do. Another user was unsure if VAL would face an object after moving to it, and attempted to ask VAL this question; VAL does not have access to this information, and is not designed to respond to questions of this kind, and so gave a confusing response.

These situations highlight the importance of mutual theory of mind in conversational agents <cit.>; it is important not only for VAL to model the intent of the human instructor, but for the instructor to be able to model VAL's capabilities and likely responses as well. The more restricted the agent's capabilities, the more important this becomes. We intend to look into ways to augment the displayed knowledge structures, as well as the tutorial process, to  give users a better sense of VAL's cognitive machinations.

§.§.§ Tedium of Confirmatory Dialogs

Many users complained that the confirmatory dialogs used to correct and evaluate GPT subroutine outputs were tedious. This was expected to some degree, and while they were necessary for this study, we discuss here some ways we might reduce the number of confirmatory dialogs in actual deployment.

First, as we discussed in Section <ref>, the use of  and  to check the output of  was quite successful, and could reduce the number of  confirmation dialogs by roughly 80% while hardly increasing the error rate. Using additional GPT subroutine calls to check the output of the other subroutines may lead to similar results across the board, further reducing these dialogs.

Caching user responses may also be useful; as the system stands, users are asked to confirm each time, even if duplicate commands are issued. This happened many times, for example, with the command “press space”.

In general, confirmatory dialogs present a trade-off: tedium for correctness. However, given the sequential nature of the dialog, one final optimization to consider may be a stricter threshold for requesting confirmation earlier in a dialog, while allowing the undo button to become the primary mechanism of error correction later in a dialog.

§.§.§ GPT Model Size

As we discuss in Section <ref>, increasing the size of the base GPT model increased performance in five metrics while only decreasing processing speed. In fact, every metric saw improvement; those reported earlier were simply the only ones to meet our statistical significance threshold.

However, there are ethical concerns about this method of scaling VAL's performance, the first being the environmental cost of training and using larger and larger language models <cit.>. Additionally, the models we used are not open-source, and the centralization of foundational technology behind the veil of a corporate API offers almost no transparency, both into how the technology works and into how the data sent by users is being used or stored <cit.>.

While this work is a proof of concept using the highest-performing existing language models, we now advocate—and intend to transition to—the use of open-source models <cit.> to power systems like VAL, even if it will take time for their performance to catch up to that demonstrated in this work.

§ CONCLUSION

VAL offers a new angle on the long-standing problem of interactive task learning: using language models in a principled way to enable natural instructional dialog. Using language models allows VAL to comprehend malformed or casual sentences that traditional semantic parsers either cannot, or would require substantial manual engineering to, parse. Furthermore, restricting language model use to a small number of narrow tasks increases interpretability and reliability.

VAL can acquire knowledge incrementally, from few examples, and generalize that knowledge for use in new situations. The natural language interface, confirmatory dialogs, and undo feature increase VAL's usability relative to existing, more brittle systems. Ultimately, we hope that VAL's usability by those with no formal training in programming or AI will help enable a wide variety of people to not only teach VAL new tasks, but also to learn tasks from VAL in educational and job training settings.

The user study we conducted supports VAL's usability, but also underscores avenues for future work on VAL: adding condition learning, incorporating additional modalities like demonstration and vision, helping users form a better theory of VAL's mind, reducing GPT subroutine errors, and optimizing the number of confirmatory dialogs are all substantial and important future projects to improve VAL. We hope that the VAL approach continues to evolve and provide a framework for future work on interactive task learning.

§ EXAMPLES OF ALL CONFIRMATORY DIALOGS

new-action confirmation

Task correctness confirmation

§ VAL PARTICIPANT SURVEY FORM

§ PRIOR EXPERIENCE SURVEY QUESTION RESULTS

§ ALTERNATIVE ABSTRACT USING ONLY 1,000 MOST COMMON WORDS

A drawing showing one step of our computer learning. Plan <ref> and Part <ref> do a better job of explaining the full way our computer learns how to do things from talking to people.
A simple picture showing a human being explaining how to do something to our computer, which asks a follow-up question after thinking about the words using several “large attention” pieces with their own small, focused jobs.

Until now, when people have tried to make computers you can talk with to explain how to do things, the computers have used old-school ways of understanding the words. The old ways they use sometimes can't understand people's words for no good reason. These days, some people use large “attention is all you need” approaches to word understanding instead, but those approaches sometimes see things that aren't there, or try to do things they can't do. In this paper, we're going to talk about a new way, where we use an old-school approach to learning, but add in a lot of large-attention pieces who each have very small and focused jobs. The attention pieces help the old-school piece do its job better, and the old-school piece makes sure the attention pieces don't say weird stuff or see things that aren't there. This lets us have a computer that can turn words into an understanding of how to play a game. We asked some people to help us explain to the computer how to play the game, and most of them said it was easy to do.


Comment on “Absence versus Presence of Dissipative Quantum Phase Transition in
Josephson Junctions”

In a recent Letter [1], Masuki, Sudo, Oshikawa, and Ashida studied a Josephson
junction, with Josephson energy $E_{\text{J}}$ and charging energy
$E_{\text{C}}$, shunted by an ohmic transmission line with conductance
$\alpha(2e)^{2}/h$. Their model includes a realistic high frequency cutoff of
order $\alpha E_{c}$, that is typically smaller than the plasma frequency $W$.
The authors present a phase diagram showing surprising features, not
anticipated in the established literature [2]. For $E_{\text{J}}/E_{\text{C}}$
above a certain value, they find that the junction remains superconducting for
all $\alpha$, while below this value, they find that the insulating phase
leads to re-entrant superconductivity at small $\alpha$. In this Comment, we
show that their Numerical Renormalization Group (NRG) implementation is
uncontrolled, and that there is no evidence for the re-entrant
superconductivity in the phase diagram presented in Fig. 1a of [1].

NRG involves approximately diagonalizing the low energy sector of a sequence
of Hamiltonians $H_{N}$ defined through the recursion $H_{N+1}=H_{N}+\Delta
H_{N+1}$ where the term $\Delta H_{N+1}$ couples the system to new degrees of
freedom at successively lower energy scales. An essential requirement is that
$\Delta H_{N+1}$ should constitute a perturbation to $H_{N}$ whose strength
decreases exponentially fast at increasing $N$. For instance, in Wilson’s
treatment of the Kondo problem, $\Delta
H_{N+1}=t_{N+1}\sum_{\sigma=\pm}(c_{N\pm}^{\dagger}c_{N+1\pm}+\text{h.c})$
with $t_{N}\sim\Lambda^{-N/2}$, $\Lambda>1$, and $\\{c_{N}\\}$ a set of
fermionic annihilation operators. This requirement is referred to as scale
separation, and allows one to obtain accurate ground state properties of
$\lim_{N\to\infty}H_{N}$ while only retaining a fixed number of low energy
states of $H_{N}$ at each step.

Figure 1: Top panel: First 9 excitation energies v. NRG step $N$, scaled with
$\Lambda^{N}$. Results obtained with the NRG scheme in [1] for the cosine and
quadratic potential are compared to exact results for the quadratic potential.
Choice of parameters: $n_{S}=50$ kept states, $n_{B}=300$ bosonic states for
$N=0$ and $n_{B}=15$ for $N>0$, $\Lambda=2.0$, $\alpha=10$,
$E_{\text{C}}=0.01W$, $E_{\text{J}}/E_{\text{C}}=10$. Bottom panel:
$\left<\cos(\varphi)\right>$ v. $\alpha$, for
$E_{\text{J}}/E_{\text{C}}=0.15$, like the triangles in the top panel of Fig.
4 of [1]. The blue dots reproduce the result of [1]. (All parameters are as in
[1]). The yellow squares and green diamonds were obtained by increasing the
truncation parameter $n_{B}$ for $N>0$ from $15$ (the value used in [1]) to
respectively $29$ and $43$. The inset zooms in on the two smallest values of
$\alpha$. It shows that here $\left<\cos(\varphi)\right>$ is still not
converged at $n_{B}=43$, and likely tends to zero as $n_{B}$ is increased.

In Ref. [1], $\Delta H_{N+1}$ contains an order $\Lambda^{0}$ term
$-E_{\text{J}}\left(\cos\Xi_{N+1}-\cos\Xi_{N}\right)$ where mode $a_{N+1}$
contributes to the phase variable
$\Xi_{N+1}=\Xi_{N}+i\xi_{N+1}(a_{N+1}^{\dagger}-a_{N+1})$, with the amplitude
$\xi_{N+1}\simeq 2\sqrt{(\Lambda-1)/(\Lambda+1)}/\sqrt{\alpha}$ at large $N$,
which violates scale separation. That the NRG scheme fails is evident from the
low-energy spectrum of $\Lambda^{N}H_{N}$ in the right panel of Fig. S2 in the
supplementary material of [1]. All excitation energies of $\Lambda^{N}H_{N}$,
computed by NRG, start growing like $\Lambda^{N}$ after $N=30$. The spectrum
of $H_{N}$ obtained in NRG thus saturates after $N=30$. In contrast, the true
spectrum of $H_{N}$ has excitation energies down to $\Lambda^{-N}$ for
arbitrarily large $N$, due to low frequency modes. To investigate this
further, we first employed the NRG scheme of [1] in the regime of large
$\alpha$ and large $E_{\text{J}}/E_{\text{C}}$, where the junction is
certainly superconducting, allowing us also to expand the cosine potential
$-E_{\text{J}}\cos(\Xi)\simeq E_{\text{J}}(\Xi^{2}/2-1)$ to quadratic order in
order to gain analytic insights. We compared low energy spectra obtained with
the NRG scheme of [1] for both the cosine and quadratic potentials, to the
exact spectrum obtained for the quadratic potential. If the NRG scheme is
well-behaved, the three calculations should yield nearly the same result.
However, as we show in the top panel of Fig. 1, the NRG results (for both
potentials) start to diverge from the exact spectrum after the seventh RG
step, in the same manner as the exponential flow seen in the right panel of
Fig. S2 in the supplementary material accompanying [1]. We conclude that the
NRG scheme proposed in [1] is unreliable.

We must further point out that, independently of the above, the calculation of
the phase diagram in the left panel in Fig. 1a of [1] is flawed in another
way. Even if, for the sake of the argument, one were to trust the employed NRG
scheme, the re-entrant superconductivity seen at small $\alpha$ and small
$E_{\text{J}}/E_{\text{C}}$ is a finite-size artefact. The blue dots in the
bottom panel of Fig. 1 were calculated with our code implementing the NRG
scheme of [1], and reproduce the result for $\left<\cos(\varphi)\right>$ v.
$\alpha$ at $E_{\text{J}}/E_{\text{C}}=0.15$ presented in the upper panel of
Fig. 4 of [1] exactly 111To achieve this agreement, we had to reverse engineer
a mistake in the numerics of [1], whose presence is revealed by the spectra of
Fig. S2 in the supplementary material to [1]. At low $N$, the spectrum should
be $n\gamma_{0}$, $n=1,2,\ldots$. According to Eq. S50 one should have
$\gamma_{0}=4.33$ in Fig. S2. Instead, one sees $\gamma_{0}=0.5$. There is in
fact a typo in Eq. (S50): $(1+\Lambda+\Lambda^{-2})/3$ should be replaced with
$(1+\Lambda^{-1}+\Lambda^{-2})/3$. Correcting the typo however does not give
$\gamma_{0}=0.5$ as in Fig. S2. We therefore tried various plausible mistakes,
and we found one that reproduced exactly the NRG results in [1]. Apparently,
the numerics in [1] were performed using $(1+\Lambda^{-3})/3$ instead of
$(1+\Lambda+\Lambda^{-2})/3$ in Eq. (S50) that defines $\gamma_{n}$. When the
correct expression for $\gamma_{n}$ is used, there are quantitative changes to
numerical results, but no change to our conclusions.. It is obtained by
allowing a maximum of $14$ bosonic excitations in in each mode $N\geq 1$ (i.e.
$n_{B}=15$ in the notation of [1]). In [1], the region between $\alpha=0.1$
and $\alpha=0.2$, where $\left<\cos(\varphi)\right>$ vanishes, is identified
as the insulating phase. In the exact Hamiltonian, there is no sharp cut-off
on the number of bosons allowed in any mode. We have therefore repeated the
calculation at larger $n_{B}$, and see that the region where
$\left<\cos(\varphi)\right>$ vanishes, grows to include the interval
$\alpha\in[0,0.2]$ as $n_{B}$ increases. Thus, the re-entrant
superconductivity claimed in [1] at small $\alpha$ is a finite size artefact
that goes away when a sufficiently large number of bosonic excitations per
mode is allowed in the NRG scheme.

We point out in closing that the lack of scale separation in the NRG of the
family of models studied in [1] is a known problem, that needs to be cured by
introducing an infrared cut-off [4], restricting the study of the physics
above this scale. While related Josephson Hamiltonians displaying spin-boson
physics are immune to these limitations [5], calculating ground state
properties of the boundary sine Gordon model using NRG remains challenging.

Théo Sépulcre1, Serge Florens2 and Izak Snyman3

1Wallenberg Centre for Quantum Technology, Chalmers University of Technology,
SE-412 96 Gothenburg, Sweden
2Univ. Grenoble Alpes, CNRS, Grenoble INP, Institut Néel, 38000 Grenoble,
France
3Mandelstam Institute for Theoretical Physics, School of Physics, University
of the Witwatersrand, Johannesburg, South Africa

## References

  * Masuki  _et al._ [2022] K. Masuki, H. Sudo, M. Oshikawa, and Y. Ashida, Absence versus presence of dissipative quantum phase transition in josephson junctions, Phys. Rev. Lett. 129, 087001 (2022).
  * Schön and Zaikin [1990] G. Schön and A. D. Zaikin, Quantum coherent effects, phase transitions, and the dissipative dynamics of ultra small tunnel junctions, Physics Reports 198, 237 (1990).
  * Note [1] To achieve this agreement, we had to reverse engineer a mistake in the numerics of [1], whose presence is revealed by the spectra of Fig.+.1667emS2 in the supplementary material to [1]. At low $N$, the spectrum should be $n\gamma_{0}$, $n=1,2,\ldots$. According to Eq.+.1667emS50 one should have $\gamma_{0}=4.33$ in Fig.+.1667emS2. Instead, one sees $\gamma_{0}=0.5$. There is in fact a typo in Eq.+.1667em(S50): $(1+\Lambda+\Lambda^{-2})/3$ should be replaced with $(1+\Lambda^{-1}+\Lambda^{-2})/3$. Correcting the typo however does not give $\gamma_{0}=0.5$ as in Fig.+.1667emS2. We therefore tried various plausible mistakes, and we found one that reproduced exactly the NRG results in [1]. Apparently, the numerics in [1] were performed using $(1+\Lambda^{-3})/3$ instead of $(1+\Lambda+\Lambda^{-2})/3$ in Eq.+.1667em(S50) that defines $\gamma_{n}$. When the correct expression for $\gamma_{n}$ is used, there are quantitative changes to numerical results, but no change to our conclusions.
  * Freyn and Florens [2011] A. Freyn and S. Florens, Numerical renormalization group at marginal spectral density: Application to tunneling in luttinger liquids, Phys. Rev. Lett. 107, 017201 (2011).
  * Kaur  _et al._ [2021] K. Kaur, T. Sépulcre, N. Roch, I. Snyman, S. Florens, and S. Bera, Spin-boson quantum phase transition in multilevel superconducting qubits, Phys. Rev. Lett. 127, 237702 (2021).