# On universal butterfly and antisymmetric magnetoresistances

H. T. Wu  Department of Physics, The Hong Kong University of Science and
Technology, Clear Water Bay, Kowloon, Hong Kong, China  HKUST Shenzhen
Research Institute, Shenzhen, 518057, China  Tai Min  Center for Spintronics
and Quantum Systems, State Key Laboratory for Mechanical Behavior of
Materials, Xi’an Jiaotong University, No.28 Xianning West Road, Xi’an,
Shaanxi, 710049, China  Z. X. Guo  Center for Spintronics and Quantum Systems,
State Key Laboratory for Mechanical Behavior of Materials, Xi’an Jiaotong
University, No.28 Xianning West Road, Xi’an, Shaanxi, 710049, China  X. R.
Wang<EMAIL_ADDRESS>Department of Physics, The Hong Kong University of Science
and Technology, Clear Water Bay, Kowloon, Hong Kong, China  HKUST Shenzhen
Research Institute, Shenzhen, 518057, China

###### Abstract

Butterfly magnetoresistance (BMR) and antisymmetric magnetoresistance (ASMR)
are about a butterfly-cross curve and a curve with one peak and one valley
when a magnetic field is swept up and down along a fixed direction. Other than
the parallelogram-shaped magnetoresistance-curve (MR-curve) often observed in
magnetic memory devices, BMR and ASMR are two ubiquitous types of MR-curves
observed in diversified magnetic systems, including van der Waals materials,
strongly correlated systems, and traditional magnets. Here, we reveal the
general principles and the picture behind the BMR and the ASMR that do not
depend on the detailed mechanisms of magnetoresistance: 1) The systems exhibit
hysteresis loops, common for most magnetic materials with coercivities. 2) The
magnetoresistance of the magnetic structures in a large positive magnetic
field and in a large negative magnetic field is approximately the same. With
the generalized Ohm’s law in magnetic materials, these principles explain why
most BMR appears in the longitudinal resistance measurements and is very rare
in the Hall resistance measurements. Simple toy models, in which the Landau-
Lifshitz-Gilbert equation governs magnetization, are used to demonstrate the
principles and explain the appearance and disappearance of BMR in various
experiments. Our finding provides a simple picture to understand
magnetoresistance-related experiments.

##  I Introduction

Magnetoresistance (MR) is an important quantity that is often used to probe
and to understand the electronic properties of a condensed matter [1]. Weak
field MRs at low temperature are a standard probe for extracting quantum
coherence length and time of metals [2], and high field MRs are a powerful
tool for measuring the Fermi surfaces of metals [1]. In magnetic materials
with magnetic hysteresis, MR-curves can be classified into several types. One
commonly-observed curve in magnetic memory devices is the parallelogram shape
as shown in Fig. 1. As an example, let us consider one type of memory devices
shown in Fig. 1(B) with a magnetic fixed layer, whose magnetization is pinned
by either an exchange bias from another antiferromagnetic layer or by its
bulky volume, a magnetic free layer, whose magnetization can be changed by an
external force such as a magnetic field, and a spacer layer of either metal or
insulator separating two magnetic layers. No matter what is the source of
resistance and MR in particular, the device has a higher and a lower resistive
states, respectively, when two magnetizations are antiparallel or parallel to
each other [3, 4, 5]. Due to the coercivity of magnetic materials, when a
magnetic field parallel to the magnetization of the fixed layer is swept up
and down, a magnetic hysteresis loop is formed as the device moves between the
two resistive states. This results in a parallelogram MR-curve. Other
commonly-observed MR-curves are a butterfly-cross called butterfly
magnetoresistance (BMR) of either upward (A) and downward (B) ones [6, 7, 8,
9] as shown in Fig. 2(A-B), and an MR-curve with one peak and one valley,
called antisymmetric magnetoresistance (ASMR), as shown in Fig. 2(C-D) [10].
BMR was found in various magnetic materials, including van der Waals layered
magnetic materials and strongly correlated materials, as well as many
traditional magnetic materials [6, 7, 8, 11, 9, 12, 13, 14, 15, 16, 17, 18,
19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36], at
both high and low temperatures, in strong and weak magnetic fields, while
people observed less common ASMR in topological Hall effect materials [37, 38,
39, 40], antiferromagnetic topological insulators [41], magnetic multilayers
[42, 43] and $\rm FeGeTe$ heterostructures [10]. The observation of BMR can
date back to the 1950s [35, 36].

Figure 1: (A) Schematics of parallelogram MR-curves when a device moves
between a higher resistive state and a lower one in a field sweeping-up and
sweeping-down process. (B) Illustration of a memory device with two stable
resistive states. The device is in a lower (higher) resistive state when the
magnetization of the free-layer is parallel (antiparallel) with that of the
fixed layer. Figure 2: Illustration of butterfly magnetoresistance (BMR) and
antisymmetric magnetoresistance (ASMR) curves. The orange and blue curves
indicate resistance in the sweeping-up and sweeping-down processes,
respectively. (A)-(B) Upward (A) and downward (B) butterfly MR-curves that
display a butterfly cross. (C)-(D) Two possible ASMR-curves that have the
feature of one peak and one valley.

Although both BMR and ASMR were widely observed, the explanations in the
literature, often involving detailed microscopic MR mechanisms, are different
for different systems. For layered films such as metallic multilayers
$\rm[Fe/Cr]_{n}$, $\rm[Co/Cu]_{n}$, and van der Waals $\rm ClI_{3}$ layers
[28, 31, 32, 33], complicated strong or weak electron scatterings involved
magnetizations of adjacent layers were used to explain all kinds of MR-curves.
In magnetic nanowires [24], FeO film [11], $\rm Co/HfO_{2}/Pt$ sandwich
structures [19], etc., BMR was attributed to the anisotropic MR effects that
depend on the relative current and magnetization orientation. Electron-magnon
scattering in systems like $\rm Fe_{5}GeTe$ van der Waal nanostructures [6],
FePt films and nanowires [17, 20], and 2D layers of $\rm Ag_{2}CrO_{2}$
antiferromagnetic films [7], where resistance depends not only on
magnetization but also on the applied fields and the temperature, is
associated with BMR observation. In traditional magnets like $\rm Fe_{3}O_{4}$
films, the electron scattering, and tunnelling at the interfaces of nanograins
[34] or scattering by the magnetization structures induced by fields and anti-
phase boundaries [15] were claimed to be responsible to the observed BMR. In
many 2D materials, BMR in $\rho_{xx}$ is believed to be due to the quantum
anomalous Hall effect (QAHE) [21, 22]. The transverse BMR is reported in some
planar Hall effects [11, 25, 26]. In summary, both BMR and ASMR were
attributed to very detailed microscopic interactions in the literature so far.
The explanations lead to an impression that microscopic interactions are
essential for these universal curves. People did relate the BMR to
magnetization reversal and hysteresis. Magnetization reversal undoubtedly
occurs in all magnetic materials, but BMR sometimes occurs, and other time
does not. A simple universal route leading to their observation is lacking.

Here we would like to ask whether the universal BMR and ASMR have a simple
general route independent of the origins of MR. This is a sensible question
because most MR-curves of all magnetic materials with magnetic hysteresis, if
not all, can be grouped into one of the above three types or their variations:
Parallelogram-shape, BMR, and ASMR. Since the parallelogram-shaped MR-curves
have a simple picture mentioned above, there is no reason to believe that BMR
and ASMR would be different.

##  II The physics of BMR and ASMR

The resistance is a state function. For a magnetic system of a given
magnetization distribution (magnetic/spin structure) and given external
conditions such as the temperature, strains, external magnetic field, etc.,
the resistance is fixed. Under a given magnetic field, a system may have one
or more than one possible stable/metastable magnetic structure. If a system
has only one stable magnetic structure, then the MR curve, no matter how
complicated it might be, has no hysteresis. Otherwise, the MR curve has
hysteresis when an external magnetic field is swept up and down in a fixed
direction. Of course, hysteresis is a general feature of magnetic materials
due to its coercivity.

An MR curve reflects the evolution path of the magnetic structure of a system.
Whether an MR-curve is a parallelogram, a BMR, or an ASMR depends on whether
the resistance of magnetic structures in a large positive magnetic field and
in a large negative magnetic field are similar or different. When the
resistances of large positive and negative magnetic fields are not too
different, an MR-curve will be either a BMR or an ASMR, independent of the
specific origin of the resistance. If the MR passes through two higher (lower)
resistance states in sweeping-up and sweeping-down processes, the MR-curve
displays two crossed peaks (valleys) and results in an (A) upward (downward)
BMR, as shown in Fig. 2(A) or (B). However, if the MR passes through one
higher and one lower resistance state in sweeping-up and sweeping-down
processes, respectively, the MR-curve displays one peak and one valley and
becomes ASMR, as shown in Fig. 2(C) or (D). This simple picture is behind
various magnetoresistance-related experiments on microscopic mechanisms
although MR-curves can have different shapes from system to system. The
coercivity field largely determines their locations of MR-loops while
microscopic details modify their shapes, not their overall features.

Figure 3: MH-curves and MR-curves for various field directions, crystalline
anisotropy, and sample thickness. (A) $m_{z}$ vs. $H$ along various
directions. MH-curves display hysteresis when the field is not perpendicular
to the easy-axis ($\hat{x}$). (B) $m_{z}$ vs $H$ along $\hat{z}$ for various
crystalline anisotropies. As $K_{\rm u}$ decreases, hysteresis disappears. (C)
$m_{z}$ vs $H$ along $\hat{z}$ for various sample thickness and for a fixed
crystalline anisotropy. Hysteresis becomes wider as thickness grows. (D)-F)
Corresponding MR-curves of (A)-(C) with the resistance of
$\zeta(\mathbf{m})=m_{x}^{2}$. A BMR appears only when a hysteresis exists.

Furthermore, with the generalized Ohm’s law in magnetic material, these
principles can explain why most BMRs occur in longitudinal resistance
measurements and are very rare in Hall resistance measurements. For a given
magnetic material, its resistance, in general, depends on the magnetization
when all other material parameters and their environment are fixed. Without
losing any generality, let us define the current direction along the $x$-axis
and transverse voltage measurement along the $y$-direction. The longitudinal
and transverse resistance can be expressed as $R_{xx}=R_{1}+A_{1}M_{x}^{2}$
and $R_{xy}=R_{1}M_{z}+A_{1}M_{x}M_{y}$, respectively according to the
generalized Ohm’s law in amorphous or polycrystalline magnetic materials [44,
45, 46]. $R_{1}$ and $A_{1}$ are material parameters whose values depends on
microscopic interactions. They describe the anomalous Hall effect and the
usual anisotropic MR (as well as the planar Hall effect), respectively. The
above resistances are general for homogeneous systems and independent of
electron scattering mechanisms that give rise to the resistance. For
inhomogeneous systems, the generalized Ohm’s law should refer to the
resistivity, and magnetization and coefficients in resistances formula above
should be properly averaged. When a magnetic field $H$ is swept up and down
along a direction not exactly perpendicular to the magnetic easy-axis, the
stable magnetic structures in a large positive magnetic field and a large
negative magnetic field are two opposite magnetizations of
$(M_{x0},M_{y0},M_{z0})$ and $(-M_{x0},-M_{y0},-M_{z0})$. The system
transforms from one state into the other through different paths in sweeping-
up and sweeping-down processes. Since $R_{xx}$ is a function of $M_{x}^{2}$,
no matter what $M_{x0}$ is, the resistances, $R_{1}+A_{1}M_{x0}^{2}$, in the
two extreme states are the same. When $M_{x}(H)$ moves between $M_{x0}$ and
$-M_{x0}$, $R_{xx}$ forms peaks and valleys and results in a BMR or an ASMR.
Unlike $R_{xx}$ which depends only on $M^{2}_{x}(H)$, $R_{xy}$ depends on
$M_{x}(H)M_{y}(H)$ and $M_{z}(H)$ at the same time. $R_{xy}$ at the two
extreme fields takes different values of $2R_{1}M_{z0}$. This explains why
most BMR appears in longitudinal resistance measurements but is rare in
$R_{xy}$-measurements. However, when the magnetic field is in the $xy$-plane,
i.e., the plane of applied current and voltage measurement. $M_{z0}$ is zero
such that the two opposite stable magnetization states have approximately the
same resistances. $R_{xy}$ can have peak and valley, resulting in either a BMR
or an ASMR. This is why these two phenomena can be observed in some planar
Hall measurements [11, 26, 25].

##  III Demonstration of principles with toy models

Whether a BMR or an ASMR appears depends only on whether the evolution of the
magnetization has a hysteresis, and whether the resistances in two extreme
states in large positive and negative magnetic fields are similar. In
experiments, various factors can affect the appearance of BMRs, including
anisotropy, thickness, temperature, etc. [6, 7, 9, 19, 28]. A BMR appears
usually in a system with a strong anisotropy and at the low temperature. It
appears sometimes in a thicker sample [9] and sometimes in a thinner one [6].
People knew that all these factors somehow affect the magnetization reversal
through which system changes facilitate or prohibit the presence of a BMR, but
a universal simple picture showing how and why a BMR occurs and does not occur
in a specific system is lacking. Here, with the simple principles mentioned
above, we use toy models to show these factors actually influence BMRs by
changing the easy-axis and hysteresis, which are essential for BMRs and ASMRs.

Our toy model is for a ferromagnetic sample whose magnetic energy is

$\mathscr{E}=\int_{V}\\{A|\nabla\mathbf{m}|^{2}-K_{\rm
u}m_{z}^{2}-\mu_{0}M_{\rm s}[\mathbf{H}+\mathbf{H}_{\rm
d}]\cdot\mathbf{m}\\}\,{\rm d}\mathbf{x}^{3},$ (1)

where $\mathbf{m}$, $A$, $K_{\rm u}$, $\mu_{0}$, $M_{\rm s}$, $\mathbf{H}$ and
$\mathbf{H}_{\rm d}$ are the magnetization unit vector, the Heisenberg
exchange stiffness, the perpendicular magneto-crystalline anisotropy, the
vacuum permeability, the saturation magnetization, the external magnetic
field, and the dipolar field, respectively. If not stated otherwise, the
sample size is $\rm 100\times 20\times 2\,nm^{3}$, and model parameters are
$A=3\,\rm pJ/m$, $M_{\rm s}=0.86\,\rm MA/m$, and $K_{\rm u}=0.3\,\rm
MJ/m^{3}$, around typical values of common magnetic materials. We also
consider a weak disorder to mimic a realistic situation. Random granular
sample of average $10\,$nm grains are generated in films by Voronoi
tessellation. Anisotropies of grains vary randomly by $10\%$ around its mean
value. The non-linear Landau-Lifshitz-Gilbert (LLG) equation governs the spin
dynamics of the model,

$\begin{split}\frac{\partial\mathbf{m}}{\partial
t}=-\gamma\mathbf{m}\times\mathbf{H}_{\rm
eff}+\alpha\mathbf{m}\times\frac{\partial\mathbf{m}}{\partial t},\end{split}$
(2)

where $\gamma$, $\alpha$ and ${\bf H}_{\rm eff}$ are respectively the
gyromagnetic ratio, Gilbert damping constant, and the effective field. ${\bf
H}_{\rm eff}(\mathbf{x})=\frac{2A}{\mu_{0}M_{\rm
s}}\nabla^{2}\mathbf{m}(\mathbf{x})+\frac{2K_{\rm u}}{\mu_{0}M_{\rm
s}}[\mathbf{m}(\mathbf{x})\cdot\mathbf{u}]\mathbf{u}+{\bf H}+{\bf H}_{\rm d}$
includes the exchange field, the magneto-crystalline anisotropy field, the
external magnetic field H, and the dipolar field ${\bf H}_{{\rm d}}$. Eq. (2)
is numerically solved by Mumax3 [47]. A large $\alpha=1$ is used to speed up
the search for static solutions at given fields. The value of $\alpha$ shall
not affect the principles of BMR and ASMR. The unit cell in the simulations is
a cube of side $2\,$nm. The selection of the toy model is not exclusive. Other
models could also be used to demonstrate the principles (see discussion). We
use a dimensionless quantity $\zeta$ to mimic MR. $\zeta$ is a function of
$m_{x}$, $m_{y}$, and $m_{z}$ ($\zeta=m_{x}^{2}$ for example). Coefficients
related to detail mechanisms are neglected in this simple model. The MR ratio
is just its normalization of MR$=(\zeta_{\rm max}-\zeta)/\zeta_{\rm max}$.
$\zeta$ can well capture the shapes of BMR, ASMR, and parallelogram-shaped MR-
curves. To consider the contributions from all local magnetization-dependent
resistivity, we average $\zeta(\mathbf{m})$ over the whole sample,

$\zeta=\frac{1}{V}\int_{V}\zeta(\mathbf{m})\,{\rm d}V.$ (3)

where $V$ is the total sample volume [24, 30, 48].

Figure 4: A BMR and an ASMR model for field-sweeping in the plane. The fields
are swept along the direction that is $0.1^{\circ}$ tilted from $\hat{x}$ to
$\hat{z}$, and the current is along $x$-direction. (A) $M_{x}$ as a function
of $H$. A hysteresis appears with a coercivity field around $60\,$mT. (B) A
downward BMR is obtained in the anisotropic magnetoresistance with two high
resistance states at high fields. (C) An upward BMR is observed in the planar
Hall resistance with two low resistance states at large positive and negative
fields. (D) The ASMR curve is obtained for the quantum anomalous Hall systems
due to $M_{z}$ components with opposite signs in sweeping-up and sweeping-down
processes, respectively. The resistance vanishes at large fields. Peaks and
valleys are around the coercivity field. Figure 5: A BMR model for a field
along $0.1^{\circ}$ from $\hat{z}$ in the $xz-$plane. (A) A square-like
hysteresis loop in $M_{z}$H-curve with a coercivity field around $130\,$mT.
(B)-(C) Downward (B) and upward (C) BMR are obtained for the anisotropic
magnetoresistance. The current is along the $x$ and $z$ direction,
respectively. (D)-(E) The BMR from two different formulas for scatter-induced
resistance discussed in the main text. They show minor differences in
curvatures. Peaks and valleys are around the coercivity fields, which is
different from that of Fig. 4.

It is known that the magnetization dynamical path relies on the angle between
the applied field and the easy-axis [49]. Hysteresis loops appear only when
the field is not perpendicular to the easy-axis. Otherwise, the system has
only one stable structure, and magnetization is reversible such that no BMRs
and ASMRs are possible. Since a field was swept in all directions in all kinds
of experiments and the easy-axis is sensitive to many factors such as
thickness, temperature, etc., it is not surprising to see the appearance and
disappearance of a BMR in similar measurements on similar samples, but with
different details. To mimic the phenomenon, we consider a sample of $K_{\rm
u}=0$ for simplicity. The demagnetization factors of $\hat{x}$, $\hat{y}$, and
$\hat{z}$ are $N_{x}=0.86$, $N_{y}=0.12$, and $N_{z}=0.02$, and easy-axis
aligns along $\hat{x}$. We sweep the field in different directions in the xz-
plane with an angle $\Theta_{H}$ to $\hat{x}$. MH-curves for
$\Theta_{H}=0^{\circ},60^{\circ},$ and $90^{\circ}$ are shown in Fig. 3(A).
For $\Theta_{H}$ not equal to $90^{\circ}$, an MH-curve displays hysteresis
loops because of the easy $x$-axis. For $\Theta_{H}=90^{\circ}$, or applied
field perpendicular to its easy-axis, there is only one stable state at each
given field such that the MH-curve is reversible and there is no hysteresis
loop. The appearance of the BMR is closely related to that of hysteresis. We
consider a resistance of $\zeta(\mathbf{m})=m_{x}^{2}$ for example, which can
represent the anisotropic MR effect with a current along $\hat{x}$. The
obtained curves are shown in Fig. 3(D). When $\Theta_{H}\neq 90^{\circ}$, this
model satisfies the principle that has the same resistance states at both
large positive and negative fields, and the MR-curve is BMR. As $\Theta_{H}$
approaches $90^{\circ}$, it degenerates from BMR to a simple curve. The vanish
of the BMR is due to the disappearance of hysteresis, regardless of specific
resistance mechanisms. This provides a new simple picture that can explain
BMR’s appearance and disappearance when the magnetic field changes its
direction in the experiments and simulations [11, 12, 24, 30].

Crystalline anisotropy varies from sample to sample and leads to the
appearance and the disappearance of the BMR and the ASMR. Perpendicular
magnetic anisotropy can both increase and decrease [50] with film thickness.
For materials whose perpendicular anisotropy decreases or even vanishes as
sample thickness increases, increase of thickness may results in vanishing BMR
in the perpendicular field-sweeping. For thin films of perpendicular
crystalline anisotropy that is insensitive to film thickness, the opposite
behaviour can occur: Easy-axis changes from perpendicular to in-plane
directions as sample thickness decreases. With the toy model, our theory can
explain disappearance of a BMR when a sample thickness both increases [6, 28]
or decreases. Figure 3(B) is the MH-curves for the field along the $\hat{z}$
for $K_{\rm u}=0.27\,\rm MJ/m^{3}$ [$2K_{\rm u}/(\mu_{0}M_{\rm s}^{2})=0.6$],
$0.36\,\rm MJ/m^{3}$ [$2K_{\rm u}/(\mu_{0}M_{\rm s}^{2})=0.8$], and $0.40\,\rm
MJ/m^{3}$ [$2K_{\rm u}/(\mu_{0}M_{\rm s}^{2})=0.9$] with other parameters
unchanged. When $K_{\rm u}$ is significantly smaller than
$1/2\mu_{0}M_{s}^{2}$, anisotropy is dominated by the demagnetization, and the
easy-axis lies in the plane. There is no hysteresis since the field is
perpendicular to the easy-axis. MR-curves, $\zeta(\mathbf{m})=m_{x}^{2}$ is
shown in Fig. 3(E). The BMR and hysteresis disappear simultaneously. For
another set of samples of $1\,$nm, $1.5\,$nm and $2\,$nm thick whose $K_{\rm
u}=0.36,\rm MJ/m^{3}$ [$2K_{\rm u}/(\mu_{0}M_{\rm s}^{2})=0.8$], MH-curves for
field-sweeping along $\hat{z}$ are shown in Fig. 3(C). Figure 3(F) is the MR-
curves of $\zeta(\mathbf{m})=m_{x}^{2}$. With the increase of thickness, the
hysteresis loop becomes fatter and the BMR is more pronounced because the
perpendicular anisotropy is enhanced. The sharper BMR in a thicker film is
previously attributed to the increase of anti-phase domain size [9], very
different from our simple universal picture.

The change of crystalline anisotropy can come from other sources. For example,
anisotropy $K_{u}$ of some materials decreases with a power of $M(T)$ [51],
which is sensitive to the temperature near the Curie temperature. The change
of $K_{u}$ can be substantial. For example, the magnetic anisotropy of
$1.2\,$nm CoFeB film could drop by 50% as temperature increases from $300\,$K
to $400\,$K [52]. As the temperature increases, perpendicular magnetic
anisotropy gets smaller, and the easy-axis changes from out-of-plane to in-
plane. Hysteresis, as well as BMR, thus no longer exists. In contrast to our
universal picture of the BMR, the disappearance of the BMR at higher
temperatures was attributed to the variation of electron scattering, which, in
turn, was attributed to the vanish of partially disordered states [6, 7, 9,
19].

The resistance is a state function. BMR or ASMR curves are the manifestations
of magnetic hysteresis. BMR and ASMR shapes and loop positions depend on
resistance mechanisms and detailed magnetic properties such as coercivity
fields. To further demonstrate this point, we use various configurations and
resistance mechanisms to generate all kinds of BMR’s and ASMR’s with our toy
models. First, we consider the in-plane fields. A smaller $K_{\rm u}=0.3\,\rm
MJ/m^{3}$ is used such that the easy-axis aligns with the $\hat{x}$. We sweep
fields along the direction $0.1^{\circ}$ from the $\hat{x}$ in the $xz$-plane.
The system displays a hysteresis in its MH-curve as shown in Fig. 4(A) with a
coercivity field around $60\,$mT. If we apply a current along the
$x$-direction and consider the resistance of $\zeta(\mathbf{m})=m_{x}^{2}$,
the MR goes down and up. This results in a two-valley butterfly cross
illustrated in Fig. 4(B). BMR valleys appear around coercivity fields. If we
consider the resistance of $\zeta(\mathbf{m})=m_{x}m_{y}$, the form of the
planar Hall resistance. A transverse BMR can be obtained as shown in Fig. 4(C)
that qualitatively agrees with the in-plane sweeping experiment [11]. If we
consider resistance in the form of $\zeta(\mathbf{m})=m_{z}$, similar to the
anomalous Hall effect, a transverse ASMR is obtained, as shown in Fig. 4(D).
Although the curve shapes are different, peaks and valleys appear all around
coercivity fields.

To demonstrate the same BMR principles in the field sweeping along the
perpendicular direction of a film, a larger $K_{\rm u}=4.8\,\rm MJ/m^{3}$ is
used in order to maintain $\hat{z}$ as the easy-axis. We sweep fields along
the direction that is $0.1^{\circ}$ tilted from $\hat{z}$ in the $xz$-plane.
An MH-curve shows hysteresis in sweeping processes as shown in Fig. 5(A) with
a coercivity field around $130\,$mT, which is sharper than Fig. 4(A). Let’s
still consider the current aligning along $x$-direction and the anisotropic
magnetoresistance of $\zeta(\mathbf{m})=m_{x}^{2}$. The resistance goes up and
down, resulting in an upward BMR illustrated in Fig. 5(B). If the current is
perpendicular to the film, the magnetoresistance becomes
$\zeta(\mathbf{m})=m_{z}^{2}$. the MR goes down and up, resulting in two
valleys in the MR-curves, i.e., a downward BMR shown in Fig. 5(C), which
qualitatively agrees with experiments [19]. In this configuration, peaks and
valleys appear at the coercivity fields, which differ from those of Fig. 4.
One can also reproduce BMR with other microscopic mechanisms, such as magnon-
electron scattering, where the external fields tune the magnetization-
dependent resistance [7, 17, 20]. If the resistance is linearly in the field,
i.e. $\zeta(\mathbf{m})=m_{z}\mu_{0}H$, a BMR curve similar to that in Ref.
[17, 20] can be reproduced as shown in Fig. 5(D). If we consider the MR in the
form of $\zeta(\mathbf{m})=-m_{z}\sqrt{\Delta/(\Delta+2)}$, where
$\Delta\propto H-H_{\rm a}$ measures the difference between the applied field
and the anisotropy field, a BMR of Fig. 6(E) can be obtained, where we choose
$\Delta=\mu_{0}(H-H_{\rm a})$ as a demonstration. This result qualitatively
agrees with experiments [7].

##  IV discussions and conclusion

We used toy models governed by the LLG equation to demonstrate BMR and ASMR in
various systems, because it is compatible with experiments involving
incoherent magnetization reversal such as those in Refs. [17, 20, 24], Other
models could also be used in different scenarios. For example, BMR has also
been observed in systems described by coherent-rotational models [53, 54], and
other special reversal process such as domain wall nucleation and motion
described by the Kondorsky model [24]. Between two states at large positive
and negative fields, there is a hysteresis in these systems, also consistent
with the general picture here. If one uses the similar procedure as that in
the third section, BMR or ASMR can also appear as long as the resistances is a
function of magnetization, and the resistance of two states at high fields are
not too different, see Supplementary Information for details.

Figure 6: (A) Schematics of BMR in a bilayer. The system moves between two
similar low resistive states by passing through a high resistive state during
the field sweeping. (B) Illustration of resistive states. The device is in a
higher (lower) resistive state when the magnetization of two layers are
parallel (antiparallel) with each other.

The rules of BMR and ASMR revealed by the toy models are general. For example,
in a sample of two free layers, in contrast to a memory cell where only one is
free, as sketched in Fig. 1(B), the system has a lower resistive state of two
magnetizations parallel to each other and a higher resistive state of two
antiparallel magnetization. When the magnetic field is swept along the
$z$-direction, the system moves between two stable low resistive states of
magnetizations along $\hat{z}$ and $-\hat{z}$. The magnetization of two layers
is no longer parallel to each other during the magnetization reversals, and
the resistance forms two peaks and an upward BMR [28, 31, 32, 33] as shown in
Fig. 6. We also consider a system with the anomalous quantum Hall effect.
Assume that the magnetic field is swept along $z$-direction, and the system
switches between the ferromagnetic states of $m_{z}=1$ and $m_{z}=-1$, both of
which have $R_{xx}$ equal to 0. However, in sweeping processes, the presence
of magnetic structure leads to a finite $R_{xx}$, resulting in two peaks and
an upward BMR [21, 22]. $\rm Fe_{3}O_{4}$ films have the same resistance
states at both high positive and high negative fields that decrease with a
field strength in the same slope [15]. Consequently, it displays a BMR. In
some chiral magnetic materials, the topological Hall resistance is related to
the topological charge. The system reverses between the states of $m_{z}=1$
and $m_{z}=-1$, which have zero topological charge and zero topological Hall
resistances. In the sweeping-up and sweeping-down process, topological charges
with opposite signs are generated in the system that produces one peak and one
valley on the MR-curve manifesting an ASMR [37, 38, 39].

It may be important to emphasize that the exact shapes and locations in a BMR
and ASMR are not our concerns here. The hysteresis loop of BMR and ASMR could
be very irregular in different systems. Their general features do not rely on
any symmetries, as shown in the toy models of Figs. 4 and 5. When the
resistance mechanism has inversion symmetry, the system is more likely to
display ASMR as the toy model shown in Fig. 4(D). In general, any curve can
always be decomposed into symmetric and antisymmetric components, as was
usually done in experiments. Nevertheless, this kind of decomposition is not
meaningful unless one can attribute each of them to a specific source. This
is, of course, an interesting question but not the aim of this paper.

ASMR is less common than BMR in both $R_{xx}$ and $R_{xy}$. In many
experiments, the magnetic field is swept perpendicular to the currents. For
the former, $M_{x}(H)$ reverses between $\pm\\{M_{x}\\}_{\rm max}$. These two
opposite magnetizations have the largest $R_{xx}$, so in sweeping-up and
sweeping-down processes, $R_{xx}$ can only decrease first and then increase
and form two valleys and a downward BMR. For the latter, $M_{x}(H)$ changes
from 0 to 0 through a path. The initial and final magnetizations have the
lowest $R_{xx}$. So whether in the sweeping-up or sweeping-down process,
$R_{xx}$ can only rise and then fall, which is manifested as an upward BMR.
For magnetic materials that obey the generalized Ohm’s law, ASMR that requires
one peak and one valley is unlikely to occur in $R_{xx}$ for either of the two
common experimental settings.

In conclusion, similar to parallelogram MR, BMR and ASMR are universal MR
behavior independent of the resistance origins. They appear as long as a
system exhibits a hysteresis loop under sweeping-up and sweeping-down of a
magnetic field, and its MR is approximately the same when the magnetization
direction is reversed. From the generalized Ohm’s law in magnetic materials,
BMR should be very common in longitudinal resistance and could als occur in
transverse resistance when the magnetic field is in the plane of applied
current and voltage measurement. The coercivity fields and microscopic details
are encoded in the positions and shapes of a BMR and ASMR. Although there is
no inconsistent with the universal BMR/ASMR theory presented here, the
explanations of many BMR and ASMR in literature [6, 7, 9, 10, 11, 19, 28, 30,
31, 32, 33] are not the same as ours.

###### Acknowledgements.

This work is supported by the Ministry of Science and Technology through grant
2020YFA0309600, the NSFC Grants (12074301, 11974296), and Hong Kong RGC Grants
(No. 16300522, 16302321, 16301518, and 16301619).

## References

  * Grosso and Parravicini [2000] G. Grosso and G. P. Parravicini,  _Solid state physics_ (Elsevier Pte Ltd., Singapore, 2000).
  * Huckestein [1995] B. Huckestein, Rev. Mod. Phys. 67, 357 (1995).
  * Kundt [1893] A. Kundt, Wied. Ann 49, 257 (1893).
  * Smith and Sears [1929] A. W. Smith and R. W. Sears, Phys. Rev. 34, 1466 (1929).
  * Pugh and Lippert [1932] E. M. Pugh and T. W. Lippert, Phys. Rev. 42, 709 (1932).
  * Ohta  _et al._ [2021] T. Ohta, M. Tokuda, S. Iwakiri, K. Sakai, B. Driesen, Y. Okada, K. Kobayashi, and Y. Niimi, AIP Advances 11, 025014 (2021).
  * Taniguchi  _et al._ [2020] H. Taniguchi, M. Watanabe, M. Tokuda, S. Suzuki, E. Imada, T. Ibe, T. Arakawa, H. Yoshida, H. Ishizuka, K. Kobayashi, and Y. Niimi, Scientific Reports 10, 2525 (2020).
  * Mukherjee  _et al._ [2010] K. Mukherjee, S. D. Das, N. Mohapatra, K. K. Iyer, and E. V. Sampathkumaran, Phys. Rev. B 81, 184434 (2010).
  * Li  _et al._ [2009a] P. Li, L. T. Zhang, W. B. Mi, E. Y. Jiang, and H. L. Bai, Journal of Applied Physics 106, 033908 (2009a).
  * Hu  _et al._ [2020] G. Hu, Y. Zhu, J. Xiang, T.-Y. Yang, M. Huang, Z. Wang, Z. Wang, P. Liu, Y. Zhang, C. Feng, D. Hou, W. Zhu, M. Gu, C.-H. Hsu, F.-C. Chuang, Y. Lu, B. Xiang, and Y.-L. Chueh, ACS Nano 14, 12037 (2020).
  * Stankiewicz  _et al._ [2006] J. Stankiewicz, F. Jiménez-Villacorta, and C. Prieto, Phys. Rev. B 73, 014429 (2006).
  * Zhou  _et al._ [2018] L. Zhou, H. Song, K. Liu, Z. Luan, P. Wang, L. Sun, S. Jiang, H. Xiang, Y. Chen, J. Du, H. Ding, K. Xia, J. Xiao, and D. Wu, Science Advances 4, eaao3318 (2018).
  * Gilbert  _et al._ [2015] D. A. Gilbert, B. B. Maranville, A. L. Balk, B. J. Kirby, P. Fischer, D. T. Pierce, J. Unguris, J. A. Borchers, and K. Liu, Nature Communications 6, 8462 (2015).
  * Chen  _et al._ [2018] W. Chen, L. Qian, and G. Xiao, Phys. Rev. B 98, 174402 (2018).
  * Li  _et al._ [2009b] P. Li, L. T. Zhang, W. B. Mi, E. Y. Jiang, and H. L. Bai, Journal of Applied Physics 106, 033908 (2009b).
  * Ennen  _et al._ [2016] I. Ennen, D. Kappe, T. Rempel, C. Glenske, and A. Hütten, Sensors 16 (2016), 10.3390/s16060904.
  * Mihai  _et al._ [2008] A. P. Mihai, J. P. Attané, A. Marty, P. Warin, and Y. Samson, Phys. Rev. B 77, 060401 (2008).
  * Liu  _et al._ [2021] C. Liu, Y. Wang, M. Yang, J. Mao, H. Li, Y. Li, J. Li, H. Zhu, J. Wang, L. Li, Y. Wu, Y. Xu, J. Zhang, and Y. Wang, Nature Communications 12, 4647 (2021).
  * Li  _et al._ [2017] L. Li, Y. Liu, J. Teng, S. Long, Q. Guo, M. Zhang, Y. Wu, G. Yu, Q. Liu, H. Lv, and M. Liu, Nanoscale Research Letters 12, 210 (2017).
  * Nguyen  _et al._ [2011] V. D. Nguyen, L. Vila, P. Laczkowski, A. Marty, T. Faivre, and J. P. Attané, Phys. Rev. Lett. 107, 136605 (2011).
  * Deng  _et al._ [2020] Y. Deng, Y. Yu, M. Z. Shi, Z. Guo, Z. Xu, J. Wang, X. H. Chen, and Y. Zhang, Science 367, 895 (2020).
  * Chang  _et al._ [2013] C.-Z. Chang, J. Zhang, X. Feng, J. Shen, Z. Zhang, M. Guo, K. Li, Y. Ou, P. Wei, L.-L. Wang, Z.-Q. Ji, Y. Feng, S. Ji, X. Chen, J. Jia, X. Dai, Z. Fang, S.-C. Zhang, K. He, Y. Wang, L. Lu, X.-C. Ma, and Q.-K. Xue, Science 340, 167 (2013).
  * Huang  _et al._ [2020] B. Huang, M. A. McGuire, A. F. May, D. Xiao, P. Jarillo-Herrero, and X. Xu, Nature Materials 19, 1276 (2020).
  * Lu and Wang [2009] J. Lu and X. Wang, Journal of Magnetism and Magnetic Materials 321, 2916 (2009).
  * Li  _et al._ [2019] X. Li, C. Collignon, L. Xu, H. Zuo, A. Cavanna, U. Gennser, D. Mailly, B. Fauqué, L. Balents, Z. Zhu, and K. Behnia, Nature Communications 10, 3021 (2019).
  * Tang  _et al._ [2003] H. X. Tang, R. K. Kawakami, D. D. Awschalom, and M. L. Roukes, Phys. Rev. Lett. 90, 107201 (2003).
  * Miao  _et al._ [2022] Y.-Q. Miao, J.-J. Guo, Z.-Y. Luo, M.-Z. Zhong, B. Li, X.-G. Wang, Y.-Z. Nie, Q.-L. Xia, and G.-H. Guo, Frontiers in Physics 10 (2022), 10.3389/fphy.2022.847402.
  * Klein  _et al._ [2018] D. R. Klein, D. MacNeill, J. L. Lado, D. Soriano, E. Navarro-Moratalla, K. Watanabe, T. Taniguchi, S. Manni, P. Canfield, J. Fernández-Rossier, and P. Jarillo-Herrero, Science 360, 1218 (2018).
  * Zhang  _et al._ [2014] Z. Zhang, X. Feng, M. Guo, K. Li, J. Zhang, Y. Ou, Y. Feng, L. Wang, X. Chen, K. He, X. Ma, Q. Xue, and Y. Wang, Nature Communications 5, 4915 (2014).
  * Wegrowe  _et al._ [1999] J.-E. Wegrowe, D. Kelly, A. Franck, S. E. Gilbert, and J.-P. Ansermet, Phys. Rev. Lett. 82, 3681 (1999).
  * Bass and Pratt [1999] J. Bass and W. Pratt, Journal of Magnetism and Magnetic Materials 200, 274 (1999).
  * Miyazaki and Tezuka [1995] T. Miyazaki and N. Tezuka, Journal of Magnetism and Magnetic Materials 139, L231 (1995).
  * Binasch  _et al._ [1989] G. Binasch, P. Grünberg, F. Saurenbach, and W. Zinn, Phys. Rev. B 39, 4828 (1989).
  * Lu  _et al._ [2007] Z. L. Lu, M. X. Xu, W. Q. Zou, S. Wang, X. C. Liu, Y. B. Lin, J. P. Xu, Z. H. Lu, J. F. Wang, L. Y. Lv, F. M. Zhang, and Y. W. Du, Applied Physics Letters 91, 102508 (2007).
  * Hirsch [1959] A. Hirsch, Physica 25, 581 (1959).
  * Tatsumoto  _et al._ [1960] E. Tatsumoto, K. Kuwahara, and M. Goto, Journal of the Physical Society of Japan 15, 1703 (1960).
  * Wang  _et al._ [2018] L. Wang, Q. Feng, Y. Kim, R. Kim, K. H. Lee, S. D. Pollard, Y. J. Shin, H. Zhou, W. Peng, D. Lee, W. Meng, H. Yang, J. H. Han, M. Kim, Q. Lu, and T. W. Noh, Nature Materials 17, 1087 (2018).
  * Zhang  _et al._ [2021] X. Zhang, S. C. Ambhire, Q. Lu, W. Niu, J. Cook, J. S. Jiang, D. Hong, L. Alahmed, L. He, R. Zhang, Y. Xu, S. S.-L. Zhang, P. Li, and G. Bian, ACS Nano 15, 15710 (2021).
  * Qin  _et al._ [2019] Q. Qin, L. Liu, W. Lin, X. Shu, Q. Xie, Z. Lim, C. Li, S. He, G. M. Chow, and J. Chen, Advanced Materials 31, 1807008 (2019).
  * Niu  _et al._ [2021] W. Niu, Z. Cao, Y. Wang, Z. Wu, X. Zhang, W. Han, L. Wei, L. Wang, Y. Xu, Y. Zou, L. He, and Y. Pu, Phys. Rev. B 104, 125429 (2021).
  * He  _et al._ [2018] Q. L. He, G. Yin, L. Yu, A. J. Grutter, L. Pan, C.-Z. Chen, X. Che, G. Yu, B. Zhang, Q. Shao, A. L. Stern, B. Casas, J. Xia, X. Han, B. J. Kirby, R. K. Lake, K. T. Law, and K. L. Wang, Phys. Rev. Lett. 121, 096802 (2018).
  * Cheng  _et al._ [2005] X. M. Cheng, S. Urazhdin, O. Tchernyshyov, C. L. Chien, V. I. Nikitenko, A. J. Shapiro, and R. D. Shull, Phys. Rev. Lett. 94, 017203 (2005).
  * Lin [2007] W. Lin, Magnetization Reversal, and Generation and Detection of Pure Spin-Current (2007).
  * Zhang  _et al._ [2017] Y. Zhang, X. S. Wang, H. Y. Yuan, S. S. Kang, H. W. Zhang, and X. R. Wang, Journal of Physics: Condensed Matter 29, 095806 (2017).
  * Zhang  _et al._ [2016] Y. Zhang, X. R. Wang, and H. W. Zhang, Europhysics Letters 113, 47003 (2016).
  * Wang [2022] X. R. Wang, Chinese Physics Letters 39, 027301 (2022).
  * Vansteenkiste  _et al._ [2014] A. Vansteenkiste, J. Leliaert, M. Dvornik, M. Helsen, F. Garcia-Sanchez, and B. Van Waeyenberge, AIP Advances 4, 107133 (2014).
  * Chung and Hsu [2008] T. Y. Chung and S. Y. Hsu, Journal of Applied Physics 103, 07C506 (2008).
  * Sun and Wang [2005] Z. Z. Sun and X. R. Wang, Phys. Rev. B 71, 174430 (2005).
  * Carcia  _et al._ [1985] P. F. Carcia, A. D. Meinhaldt, and A. Suna, Applied Physics Letters 47, 178 (1985).
  * Schlickeiser  _et al._ [2014] F. Schlickeiser, U. Ritzmann, D. Hinzke, and U. Nowak, Phys. Rev. Lett. 113, 097201 (2014).
  * Lee  _et al._ [2017] K.-M. Lee, J. W. Choi, J. Sok, and B.-C. Min, AIP Advances 7, 065107 (2017).
  * Zhao  _et al._ [2007] G. Zhao, X. Wang, Y. Feng, and C. Huang, IEEE transactions on magnetics 43, 2908 (2007).
  * Zhao  _et al._ [2019] G. P. Zhao, L. Zhao, L. C. Shen, J. Zou, and L. Qiu, Chinese Physics B 28, 077505 (2019).


# Learning to Execute Actions or Ask Clarification Questions

Zhengxiang Shi  Yue Feng  Aldo Lipani
University College London, London, UK
<EMAIL_ADDRESS>

###### Abstract

Collaborative tasks are ubiquitous activities where a form of communication is
required in order to reach a joint goal. Collaborative building is one of such
tasks. We wish to develop an intelligent builder agent in a simulated building
environment (Minecraft) that can build whatever users wish to build by just
talking to the agent. In order to achieve this goal, such agents need to be
able to take the initiative by asking clarification questions when further
information is needed. Existing works on Minecraft Corpus Dataset only learn
to execute instructions neglecting the importance of asking for
clarifications. In this paper, we extend the Minecraft Corpus Dataset by
annotating all builder utterances into eight types, including clarification
questions, and propose a new builder agent model capable of determining when
to ask or execute instructions. Experimental results show that our model
achieves state-of-the-art performance on the collaborative building task with
a substantial improvement. We also define two new tasks, the learning to ask
task and the joint learning task. The latter consists of solving both
collaborating building and learning to ask tasks jointly.

##  1 Introduction

Following instructions in natural language by intelligent agents to achieve a
shared goal with the instructors in a pre-defined environment is a ubiquitous
task in many scenarios, e.g., finding a target object in an environment Nguyen
and Daumé III (2019); Roman et al. (2020), drawing a picture Lachmy et al.
(2021), or building a target structure Narayan-Chen et al. (2019). A number of
machine learning (ML) research projects about following instructions tasks
have been initiated by making use of the video game Minecraft Johnson et al.
(2016); Shu et al. (2018); Narayan-Chen et al. (2019); Guss et al. (2019);
Jayannavar et al. (2020). Building such agents requires to make progress in
grounded natural language understanding – understanding complex instructions,
for example, with spatial relations in natural language – self-improvement –
studying how to flexibly learn from human interactions – synergies of ML
components – exploring the integration of several ML and non-ML components to
make them work together Szlam et al. (2019).

Figure 1: A simple example of builder task: The builder can observe the world
state and dialogue context. For the sake of space, only a part of the dialogue
history is displayed. The utterance in green displays understanding and the
utterance in yellow asks a clarification question.

The recently introduced Minecraft Corpus dataset Narayan-Chen et al. (2019)
proposes a collaborative building task, in which an architect and a builder
can communicate via a textual chat. Architects are provided with a target
structure they want to have built, and the builders are the only ones who can
control the Minecraft avatar in the virtual environment. The task consists in
building 3D structures in a block world-like scenario collaboratively, as
shown in the Figure 1. Earlier works in Minecraft collaborative building tasks
Jayannavar et al. (2020) attempted to build an automated builder agent with a
large action space but failed to allow the builder to take the initiative in
the conversation. However, an intelligent agent should not only understand and
execute the instructor’s requests but also be able to take initiatives, e.g.,
asking clarification questions, in case the instructions are ambiguous. In the
task defined by this dataset, builders may encounter ambiguous situations that
are hard to interpret by just relying on the world state information and
instructions. For example, in Figure 1, we provide a simple case where the
architect fails to provide sufficient information to the builder, such as the
color of the blocks. In this situation, it is clearly difficult for the
builder to know exactly which action should be taken. If, however, the builder
is able to clarify the situation with the architect, this ambiguity can be
resolved. Therefore, builders, besides following architects’ instructions,
should take the initiative in the conversation and ask questions when
necessary.

To this end, in this paper we annotate all builder utterances in the Minecraft
Corpus dataset by categorizing them in the dataset into eight dialogue
utterance types as shown in Table 2, allowing the intelligent agents to learn
when and what to ask given the world state and dialogue context. Particularly,
a builder would ask task-level questions or instruction-level questions for
further clarifications. Experimental results in the Sec. 5.2 show that
determining when to ask clarification questions remains a challenging task.
However, it is worth noting that the clarification questions in the Minecraft
Corpus dataset are more complex and diverse than those in navigation tasks
Roman et al. (2020); Nguyen and Daumé III (2019); Thomason et al. (2020) whose
questions are relatively simpler and mainly about where to go.

Also, we propose a new automated builder agent that learns to map instructions
to actions and decide when to ask questions. Our model utilizes three dialogue
slots, the action type slot, the location slot, and the color slot. This
solution has the benefit of making the learning easier with respect to those
models that work using a large action space Jayannavar et al. (2020). To solve
the collaborative building task, both the dialogue context and the world state
need to be considered. Therefore, to endow our model with the ability to
better learn the representations between the world state and language, our
model implements a cross-modality module, which is based on the cross
attention mechanism. Experimental results on our extended Minecraft Corpus
dataset show that our model achieves state-of-the-art performance with a
substantial improvement for the collaborative building task. We also provide
new baselines for learning to ask task and the joint learning of these two
tasks.

##  2 Related Work and Background

#### Dialogue Tasks.

As virtual personal assistants have now penetrated the consumer market, with
products such as Siri and Alexa, the research community has produced several
works on task-oriented dialogue tasks such as: hotel booking, restaurant
booking, movie recommendation, etc. Budzianowski et al. (2018); Wei et al.
(2018); Wu et al. (2019); Feng et al. (2021, 2022); Kim and Lipani (2022).
These task-oriented dialogues have been modelled as slot filling tasks. These
tasks consist of correctly identifying and extracting information (slots)
useful to solve the task. However, most of these slot filling tasks Coope et
al. (2020); Ni et al. (2020) are considered as semantic tagging or parsing of
natural language and do not normally consider visual information. Moreover,
these tasks focus only on two of the many components needed by conversational
systems: the Natural Language Understanding (NLU) and Dialogue State Tracking
(DST) ones Budzianowski et al. (2018); Williams et al. (2014). Beside these
task-oriented dialogue tasks, the research community has also focused on
instruction following dialogue tasks, such as target completion tasks de Vries
et al. (2017), object finding tasks Roman et al. (2020), and navigation tasks
Thomason et al. (2020). Narayan-Chen et al. (2019) proposed the Minecraft
Corpus dataset, where the task consists in a cooperative asymmetric task
involving an architect and a builder that have to build a target structure
collaboratively. Jayannavar et al. (2020) then built a builder model to follow
the sequential instructions from the architect.

#### Multi-Modal.

Almost all instruction following dialogue tasks need to consider both
contextual information and actions as well as the state of the world Suhr and
Artzi (2018); Suhr et al. (2019); Chen et al. (2019); Lachmy et al. (2021),
which remains a key challenge for instruction following dialogue tasks. In
particular, the Vision-and-Dialog Navigation (VDN) task Chen et al. (2019);
Thomason et al. (2020); Roman et al. (2020); Zhu et al. (2021) where the
question-answering dialogue and visual contexts are leveraged to facilitate
navigation, has attracted increasing research attention. Other tasks, such as
moving blocks tasks Misra et al. (2017) and object finding tasks Janner et al.
(2018), also require the modelling of both contextual information in natural
language as well as the world state representation to be solved.

Table 1: The taxonomy of builder utterances: we categorize them into eight types where instruction-level questions and task-level questions are both a sub-type of clarification questions. There are 4,904 builder utterances in total. Category | Example | Amount | Percentage
---|---|---|---
Instruction-level Questions |  | 1\. What color?
---
2\. Is it flat?
914 | 18.64%
Task-level Questions |  | 1\. What are we building?
---
2\. What’s next?
252 | 5.14%
Verification Questions |  | 1\. Like that or othwr way?
---
2\. Is this the cross you wanted?
1021 | 20.82%
Greeting |  | 1\. Ready?
---
2\. Hello!
808 | 16.48%
Suggestions |  | 1\. In the future we can call these donuts or something
---
2\. No problem, if it’s hard to describe we can just go step by step
59 | 1.23%
Display Understanding |  | 1\. No problem.
---
2\. Knew what you meant
1296 | 26.43%
Status Update |  | 1\. I don’t have enough green to continue.
---
2\. I’ll stay with this perspective
101 | 2.06%
Others |  | 1\. I got my first job from Minecraft.
---
2\. Oh, wwo, sorry!
453 | 9.24%

#### Spatial Reasoning.

Many instruction following dialogue tasks contain texts with spatial-temporal
concepts Chen et al. (2019); Yang et al. (2020). For instance, the Minecraft
Corpus dataset contains utterances with spatial relations, e.g., “go to the
middle and place an orange block two spaces to the left”. Although pre-trained
language models have been used successfully in a wide array of downstream
tasks, interpreting and grounding abstractions stated in natural language,
such as spatial relations, have not been systematically studied and remain
still challenging. Therefore, another challenge for an agent is to follow
instructions which require the learning and understanding of spatio-temporal
linguistic concepts in natural language. To train models able to understand
and reason about spatial references in natural language, Shi et al. (2022)
proposed a benchmark for robust multi-hop spatial reasoning over texts.

#### Learning by Asking Questions.

Determining whether to ask clarification questions and what to ask is critical
for instruction followers to complete the tasks. Several recent studies have
focused on learning a dialogue agent with the ability to interact with users
by both responding to questions and by asking questions to accomplish their
task interactively Li et al. (2017); de Vries et al. (2017); Misra et al.
(2018); Roman et al. (2020). For instance, de Vries et al. (2017) introduced a
game to locate an unknown object via asking questions about objects in a given
image. A decision-maker is introduced to learn when to ask questions by
implicitly reasoning about the uncertainty of the agent. Different from
earlier works Kitaev and Klein (2017); Suhr et al. (2019), recent works on VDN
tasks propose agents that learn to ask a question when the certainty of the
next action is low Thomason et al. (2020); Roman et al. (2020); Chi et al.
(2020). Roman et al. (2020) proposed a two models-based agent with a navigator
model and a questioner model. The former model was responsible for moving
towards the goal object, while the latter model was used to ask questions. Zhu
et al. (2021) proposed an agent that learned to adaptively decide whether and
what to communicate with users in order to acquire instructive information to
help the navigation.

##  3 Dataset and Tasks

###  3.1 The Minecraft Dialogue Corpus

The Minecraft Dialogue Corpus Narayan-Chen et al. (2019) is built upon a
simulated block-world environment with dialogues between an architect and a
builder. This consists of 509 human-human dialogues (15,926 utterances,
113,116 tokens) playing the role of an architect and a builder, and game logs
for 150 target structures of varying complexity (min. 6 blocks, max. 68
blocks, avg. 23.5 blocks), For each target structure at least three dialogues
are collected where each dialogue contains 30.7 utterances (22.5 architect
utterances and 8.2 builder utterances) and 49.5 builder blocks movements on
average.

The architect instructs about a target structure the builder to build it via a
dialogue. Although the architect observes the builder operating in the world,
only the builder can move blocks. The builder has access to an inventory of
120 blocks of six given colors that he or she can place or remove. The
collaborative building task restricts the structures to a build region of size
11 × 9 × 11, and contains 3709, 1331, and 1616 samples for training,
validation, and test sets.

###  3.2 Builder Dialogue Annotation

Table 2: Statistics of the extended Minecraft Dialogue Corpus: "Execution
(Original)" represents that the builder should predict a sequence of building
actions given the dialogue context and the world state in a sample; "Ask for
clarifications" indicates that the builder should ask for more information in
order to execute building actions; "Others" stands for remaining dialogue acts
for the builder such as greetings, chit-chat, and display understanding.

Train Valid Test Execution (Original) 3709 1331 1616 Ask for clarifications
437 151 163 Others 837 267 366 Total 4983 1749 2145

Builders need to be able to decide their actions at any time point rather than
only execute actions with the information about when to execute. Thus, we
annotate all builders’ utterances in the Minecraft Corpus dataset Narayan-Chen
et al. (2019) and categorize all 4,904 builder utterances into 8 utterance
types. These types are partially inspired by the work of Lambert et al.
(2019). Each utterance falls into exactly one category. These categories are
defined as follows: (1) Instruction-level Questions: used to request that the
architect clarifies previous instructions; (2) Task-level Questions: used to
request the architect to give a description about the whole picture of the
building task, e.g., ask for the next instruction or ask to describe how the
target structure should look like; (3) Verification Questions: used to request
the architect to give feedback on previous actions; (4) Greetings: used to
greet each other; (5) Suggestions: used to provide suggestions about building;
(6) Display Understanding: used to express whether previous instructions have
been understood; (7) Status Update: used to describe the current status, e.g.,
tell the architect where they are, their current block stock status, or
whether they have finished a given instruction; (8) Others: not relevant to
the task itself (e.g. chit-chat, expressing gratification, or apologies).

Among these 8 utterance types, the instruction-level questions and the task-
level questions are a sub-type of clarification questions used to further
clarify instructions or the task itself when the information from the
architect is not clear or ambiguous. Based on these annotations, we extend the
original dataset (the first row in Table 2) with two other dialogue acts, ’Ask
for clarifications’ and ’Others’, as shown in the second and third rows of
Table 2. Each ’Ask for clarifications’ sample includes a world state and a
dialogue context followed by a builder utterance labelled as instruction-level
questions or task-level questions; each sample tagged as ’Others’ includes a
world state and a dialogue context followed by other builder utterance types.

Figure 2: The model architecture. The $\oplus$ sign represents the
concatenation operation. This illustration uses the plate notation. There are
a total of $N_{T}+1$ text single modality modules, $N_{G}+1$ grid single
modality modules, $N_{T}$ text cross modality modules, and $N_{T}$ grid cross
modality modules. Arrows indicate the flow of information.

###  3.3 Task Definition

Let $\mathcal{H}$ be the set of all dialogue contexts, $\mathcal{W}$ the set
of all world states, and $\mathcal{A}$ the set of actions, including building
actions (placing a block, removing a block, or a special stop action that
terminates the task) and utterance actions (ask clarification questions or
other utterance categories). Given a dialogue context $h\in\mathcal{H}$, an
initial grid-based world state $w_{0}\in\mathcal{W}$, the target is to predict
the next action type as Execution, Ask for clarifications and Others. When the
prediction of the action type is Execution, a sequence of actions
$\\{a_{i}\\}_{i=1}^{n}$, where $a_{i}\in\mathcal{A}$ and $a_{n}$ is the Stop
action, is needed to be generated to reach the final target structure $w_{n}$.
The action type Execution will update the world state via a deterministic
transition function $T$ such that $w_{i}=T(w_{i-1},a_{i})$ where
$w_{i}\in\mathcal{W}$.

##  4 Method

In this section we introduce the proposed builder model, as shown in Figure 2.
The model comprises four major components: the utterance encoder, the world
state encoder, the fusion module, and the slot decoder. The utterance encoder
(in Sec. 4.1) and world state encoder (in Sec. 4.2) learn to represent the
dialogue context and the world state. These encoded representations are then
fed into the fusion module (in Sec. 4.3) that learns contextualized embeddings
for the grid world and textual tokens through the single and cross modality
modules. Finally, the learned world and text representations are mapped into
the pre-defined slot-values in the slot decoder (in Sec. 4.4).

###  4.1 Dialogue Context Encoder

We add “architect” and “builder” annotations before each architect utterance
$A_{t}$ and each builder utterance $B_{t}$ respectively. Then, the dialogue
utterances are represented as

$D_{t}=\text{``architect''}A_{t}\oplus\text{``builder''}B_{t}$

at the turn $t$, where $\oplus$ is the operation of sequence concatenation.The
entire dialogue context is defined as:

$\displaystyle H=D_{1}\oplus D_{2}\oplus\dots\oplus D_{t}$ (1)

Given the dialogue context $H$, we truncate the tokens from the end of the
dialogue context or pad them to a fixed length as inputs and then use the
dialogue context encoder to encode utterance history into
$U\in\mathbb{R}^{s\times d_{w}}$, where $d_{w}$ is the dimension of the word
embedding and $s$ is the maximum number of tokens for a dialogue context. The
dialogue context encoder can be word embeddings like Glove Pennington et al.
(2014) or contextual word embeddings Devlin et al. (2019), which are both
widely used in the literature Ni et al. (2021a, b); Wang et al. (2020, 2021).

###  4.2 Grid World State Encoder

The world state is represented by a voxel-based grid. We first represent each
grid state as a 7-dimensional one-hot vector that stands for empty or a block
having one of six colors, yielding a 7×11×9×11 world state representation.
Additionally, we truncate the action history to the last five ones, assign an
integer weight in $1,\dots,5$ and then include these weights as a separate
input feature in each grid, resulting in a raw world state input of
$W_{0}\in\mathbb{R}^{8\times 11\times 9\times 11}$. We also represent the last
action as an 11-dimensional vector $a$ where the first two dimensions
represent the placement or removal actions, the next six dimensions represent
the color, and the last three dimensions represent the location of the last
action.

The structure of the world state encoder is similar to Jayannavar et al.
(2020)’s, i.g., consisting of $k$ 3D-convolutional layers ($f_{1}$) with
kernel size 3, stride 1 and padding 1, followed by a ReLU activation function.
Between every successive pair of these layers there is a 1×1×1
3D-convolutional layer ($f_{2}$) with stride 1 and no padding followed by
ReLU:

$\displaystyle W_{i}=\text{ReLU}(f^{i}_{2}(\text{ReLU}(f^{i}_{1}(W_{i-1})))),$
(2) $\displaystyle W_{k}=\text{ReLU}(f^{i}_{1}(W_{k-1})),$ (3)

where $i=1,2,\dots,k-1$. $W_{k}\in\mathbb{R}^{d_{c}\times 11\times 9\times
11}$ is the learned world grid-based representation where $d_{c}$ is the
dimension of each grid representation. Then we concatenate the last action
representation $a\in\mathbb{R}^{11}$ to each grid vectors in $W_{k}$ and
reshape them into $W^{{}^{\prime}}\in\mathbb{R}^{d_{c}^{{}^{\prime}}\times
1089}$, where $d_{c}^{{}^{\prime}}=d_{c}+11$.

###  4.3 Fusion Module

The fusion module comprises four major components: two single modality modules
and two cross-modality modules. The former modules are based on self-attention
layers and the latter on cross-attention layers. These take as input the world
state representation and dialogue history representation. Between every
successive pair of grid single-modality modules or text single-modality
modules there is a cross modality module. We take $N_{G}$ and $N_{T}$ layers
for the grid cross modality module and the text cross modality module. We
first revisit the definition and notations about the attention mechanism
Bahdanau et al. (2015) and then introduce how they are integrated into our
single modality modules and cross-modality modules.

#### Attention Mechanism.

Given a query vector $x$ and a sequence of context vectors
$\\{y_{j}\\}^{K}_{j=1}$, the attention mechanism first computes the matching
score $s_{j}$ between the query vector $x$ and each context vector $y_{j}$.
Then, the attention weights are calculated by normalizing the matching score:
$a_{j}=\frac{exp(s_{j})}{\sum_{j=1}^{K}exp(s_{j})}$. The output of an
attention layer is the attention weighted sum of the context vectors:
$Attention(x,{y_{j}})=\sum_{j}a_{j}\cdot y_{j}$. Particularly, the attention
mechanism is called self-attention when the query vector itself is in the
context vectors $\\{y_{j}\\}$. We use the multi-head attention following
Devlin et al. (2019); Tan and Bansal (2019).

#### Single-Modality Module.

Each layer in a single-modality module contains a self-attention sub-layer and
a feed-forward sub-layer, where the feed-forward sub-layer is further composed
of a linear transformation layer, a dropout layer and a normalization layer.
We take $N_{G}+1$ and $N_{T}+1$ layers for the grid single-modality modules
and the text single-modality modules respectively, interspersed with cross-
modality module as shown in Figure 2. Since new blocks can only be feasibly
placed if one of their faces touches the ground or another block in the
Minecraft world, we add masks to all infeasible grids in the grid single-
modality modules. For a set of text vectors $\\{u_{i}^{n}\\}_{i=1}^{s}$ and a
set of grid vectors $\\{w_{j}^{m}\\}_{j=1}^{1089}$ as inputs of $n$-th text
single-modality layer and $m$-th grid single-modality layer, where
$n\in\\{1,\dots,N_{T}+1\\}$ and $m\in\\{1,\dots,N_{G}+1\\}$, we first feed
them into two self attention sub-layers:

$\displaystyle u_{i}^{n}$
$\displaystyle=\text{SelfAttn}_{u}^{n}(u_{i}^{n},\\{u_{i}^{n}\\}),$ (4)
$\displaystyle w_{j}^{m}$
$\displaystyle=\text{SelfAttn}_{w}^{m}(w_{j}^{m},\\{w_{j}^{m}\\},mask)$ (5)

Lastly, the outputs of self attention modules, $u_{i}^{n}$ and $w_{j}^{m}$,
are followed by feed-forward sub-layers to obtain $\hat{u_{i}}^{n}$ and
$\hat{w_{j}}^{m}$.

#### Cross-Modality Module.

Each layer in the cross-modality module consists of one cross-attention sub-
layer and one feed-forward sub-layer, where the feed-forward sub-layers follow
the same setting as the single-modality module. Given the outputs of $n$-th
text single-modality layer, $\\{\hat{u_{i}}^{n}\\}_{i=1}^{s}$, and the $m$-th
grid single-modality layer, $\\{\hat{w_{j}}^{m}\\}_{j=1}^{1089}$, as the query
and context vectors, we pass them through cross-attention sub-layers,
respectively:

$\displaystyle\hat{u_{i}}^{n+1}$
$\displaystyle=\text{CrossAttn}_{u}^{n}(\hat{u_{i}}^{n},\\{\hat{w_{j}}^{m}\\}),$
(6) $\displaystyle\hat{w_{j}}^{m+1}$
$\displaystyle=\text{CrossAttn}_{w}^{m}(\hat{w_{j}}^{m},\\{\hat{u_{i}}^{n}\\}),$
(7)

The cross-attention sub-layer is used to exchange the information and align
the entities between the two modalities in order to learn joint cross-modality
representations. Then the output of the cross-attention sub-layer is processed
by one feed-forward sub-layer to obtain $\\{u_{i}^{n+1}\\}_{i=1}^{s}$ and
$\\{w_{j}^{m+1}\\}_{j=1}^{1089}$, which will be passed to the following singe-
modality modules.

Finally, we obtain a set of word vectors,
$\\{\hat{u_{i}}^{N_{T}+1}\\}_{i=1}^{s}$, and a set of grid vectors,
$\\{\hat{w_{j}}^{N_{G}+1}\\}_{j=1}^{1089}$, that is, $U^{N_{T}}$ and
$W^{N_{G}}$. Since the value of $N_{G}$ and $N_{T}$ could be different, the
modality with more layers would keep using the last single modality module’s
output of another modality as the input of its cross modality modules, as
shown in the Figure 2.

###  4.4 Slot Decoder

The Slot Decoder contains three linear projection layers of trainable
parameters,
$M_{L}\in\mathbb{R}^{d_{c}^{{}^{\prime}}},M_{C}\in\mathbb{R}^{6\times
d_{w}},M_{T}\in\mathbb{R}^{d_{a}\times d_{w}}$ where $d_{a}$ is the number of
action types to predict. We compute the average of
$U^{N_{T}}\in\mathbb{R}^{s\times d_{w}}$ alongside the $s$-dimension to obtain
$u\in\mathbb{R}^{d_{w}}$. Then we compute location logits, color logits, and
action type logits:

$\displaystyle\hat{l}$ $\displaystyle=\text{softmax}(M_{L}\cdot W^{N_{G}}),$
(8) $\displaystyle\hat{c}$ $\displaystyle=\text{softmax}(M_{C}\cdot u),$ (9)
$\displaystyle\hat{t}$ $\displaystyle=\text{softmax}(M_{T}\cdot u),$ (10)

where softmax functions are used to map the extracted information into
$\hat{l}\in\mathbb{R}^{1089}$, $\hat{c}\in\mathbb{R}^{6}$, and
$\hat{t}\in\mathbb{R}^{d_{a}}$.

##  5 Experiment, Results and Discussion

In this section, we first compare our model against the baseline for the
collaborative building task where models only need to learn the instruction
following task (in Sec. 5.1). Then, we train our model to learn when to ask
and evaluate on our extended Minecraft Dialogue Corpus (in Sec. 5.2). Finally,
we evaluate our model’s ability on the combination of the two above-mentioned
tasks (in the Sec. 5.3). All training details are reported in the Appendix.
The software and data are available at:
https://github.com/ZhengxiangShi/LearnToAsk.

Table 3: Evaluation on the collaborative building task.

Model Metric Augmentation None 2x 4x 6x BAP model F1 16.1 18.7 18.4 18.2
Recall 12.6 15.2 14.3 15.0 Precision 22.4 24.5 25.7 23.3 Ours (GloVe) F1 35.0
36.5 37.8 39.4 Recall 28.3 30.1 31.4 33.4 Precision 45.8 46.2 47.6 48.1 Ours
(BERT) F1 34.5 30.1 30.4 35.4 Recall 26.7 23.6 23.4 27.9 Precision 48.7 42.6
43.5 48.5

###  5.1 Collaborative Building Task

#### Settings.

In this task, the models are only trained to generate a sequence of actions
without the need of considering Ask for clarifications and Others. We first
compare our model against the only baseline111https://github.com/prashant-
jayan21/minecraft-bap-models named BAP Jayannavar et al. (2020) by only using
the "Execution (Original)" dataset as in Table 2. Then, we conduct the
experiments by using the augmented data from Jayannavar et al. (2020): the
models are trained and evaluated with the augmented training data: 5,563
(indicated as 2x), 9,272 (4x), and 12,981 (6x) training samples. We provide
the ground-truth previous actions and the world state for the next action
prediction. For the sake of fairness, we retrain the BAP model under the same
setting.

For our models, we present the performance of two different dialogue context
encoders: we use the pre-trained GloVe word embeddings with 300 dimensions
Pennington et al. (2014) as the initial word embeddings followed by a GRU
Chung et al. (2014) and contextual word embeddings using the pre-trained BERT
base model Devlin et al. (2019). For the action type slot, we pre-define three
potential values: placement, removal, and stop. The value of the location slot
can be one of 1,089 candidate voxels and the value of the color slot can be
one of six candidate colors. During training we minimize the sum of the cross
entropy losses of the location slot, the color slot, and the action type slot.
The F1-score metric on the test set is used to evaluate model performance by
comparing the model predictions against the action sequence performed by the
human builder.

#### Results.

In Table 3 we present the results of our model and the baselines for the
collaborative building task on the Minecraft Corpus Dataset. Experimental
results show that our model outperforms the baseline model with a large
margin. Results on the augmented dataset show that the advantage of the data
augmentation is not obvious. The performance using contextualized word
embeddings is poorer. This could be due to the size of the builder model with
the BERT encoder which makes it more difficult to train.

Figure 3: Case study of the collaborative building task in Sec. 5.1: A and B
represents the architect and the builder.

###  5.2 Learning to Ask Task

#### Settings.

In this task, all the models are trained only to predict one type of actions,
Execution, Ask for clarifications and Others, without the need to generate a
sequence of actions. All datasets in the Table 2 are used. In our model, for
the action type slot, we define three potential slot values: Execution, Ask,
and Others. In this experiment, the slots for location and color are not used.
We use the pre-trained GloVe embeddings in the dialogue context encoder.
During the training, the cross entropy loss of the action type is minimized.

Table 4: Test Accuracy of the learning to ask task.

Test Accuracy(%) Prediction Size Execute Ask Others Oracle Execution 93.81
4.33 1.86 1616 Ask 22.09 63.80 14.11 163 Others 35.79 36.89 27.32 366 Overall
Test Acc 80.05 2145

#### Results.

In Table 4, we present the results of our model. Although our model achieves
around 80% overall test accuracy, the correct answers mainly come from the
execution type while the model struggles with the ask and other types. These
two types have in fact a joint test accuracy of 38.6%. Experimental results
demonstrate that the difficulty of the learning to ask task and that there is
still a large room for improvement.

###  5.3 Joint Learning Task

#### Settings.

In this task, the models are trained to not only predict one type of actions
from Execution, Ask for clarifications and Others but also to generate a
sequence of actions. All the datasets in Table 2 are used. In our model, for
the action type slot, we pre-define five potential values: Placement, Removal,
Stop, Ask, and Others. The value of the location slot can be one of 1,089
candidate grid and the value of the color slot is one of six candidate colors.
We still use pre-train GloVe embedding in the dialogue context encoder. During
the training we minimize the sum of the cross entropy losses of the location
slot, the color slot, and the action type slot with weights equal to 0.1, 0.1,
0.8. We provide the ground-truth previous actions and the world state for the
next action prediction.

Table 5: Test accuracy of the joint task.

Test Accuracy(%) Prediction Execution Ask Others Oracle Execution 82.64 8.64
8.72 Ask 8.61 60.93 30.46 Others 25.47 47.07 31.46 Overall Test Acc 72.26

Table 6: The evaluation of the joint learning task.

F1 Recall Precison Ours 28.4 20.9 43.9

#### Results.

In Table 5, we present the results of our model’s test accuracy for each
action type. The model has an 72.3% test accuracy. However, if the execution
of building actions is excluded, its joint test accuracy of ask and other
action types is about 40.5%, indicating that deciding when to take the
initiative remains challenging. In Table 6, we also report the results for the
building task. Not surprisingly, the performance of our model drops slightly
compared to those in Table 3, reflecting the difficulty of jointly learning.

###  5.4 Case Study

Although our model can predict the actions more accurately than the baseline,
for example our model can usually predict the color of the blocks correctly
with about 60% test accuracy rate, it is still non-trivial for our model to
predict the whole action sequence correctly. In Figure 3, the architect
instructed the builder to build a 3x3 square and then our model generated only
parts of the structure successfully.

The dataset noise makes the learning process more challenging: the builder
action sequences are noisy due to, for example, the builder miss-clicking in
the construction process Narayan-Chen (2020). Also, builder action sequences
are often fragmented between utterances due to the frequent interruptions of
the architect. In order to solve these issues a good model should be capable
to learn better representations for higher-level abstractions in natural
language like spatial relation concepts and be more robust to noisy actions
Shi et al. (2022). However, existing models including pre-trained ones Devlin
et al. (2019) fail to learn such representations for spatial reasoning, which
translates into poor performance in these instruction following tasks.

##  6 Conclusion

In this paper, we extend the Minecraft Corpus dataset by labelling builder
utterances into eight types, in which two of them are relevant to
clarification questions. This allows builder models to learn to take
initiatives in the instruction following tasks. Also, we have proposed a new
model that achieves state-of-the-art performance on the Minecraft
collaborative building task with a large improvement. Besides these
contributions, we introduce a new learning to ask task for clarification
questions and a jointly learning task with the collaborative building task. We
leave the generation of clarification questions to future work.

## References

  * Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015.  Neural machine translation by jointly learning to align and translate.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Budzianowski et al. (2018) Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018.  MultiWOZ - a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_.
  * Chen et al. (2019) Howard Chen, Alane Suhr, Dipendra Misra, Noah Snavely, and Yoav Artzi. 2019.  TOUCHDOWN: natural language navigation and spatial reasoning in visual street environments.  In _IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019_.
  * Chi et al. (2020) Ta-Chung Chi, Minmin Shen, Mihail Eric, Seokhwan Kim, and Dilek Hakkani-Tür. 2020.  Just ask: An interactive learning framework for vision and language navigation.  In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI_.
  * Chung et al. (2014) Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014.  Empirical evaluation of gated recurrent neural networks on sequence modeling.  _arXiv preprint arXiv:1412.3555_.
  * Coope et al. (2020) Samuel Coope, Tyler Farghly, Daniela Gerz, Ivan Vulić, and Matthew Henderson. 2020.  Span-ConveRT: Few-shot span extraction for dialog with pretrained conversational representations.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
  * de Vries et al. (2017) Harm de Vries, Florian Strub, Sarath Chandar, Olivier Pietquin, Hugo Larochelle, and Aaron C. Courville. 2017.  Guesswhat?! visual object discovery through multi-modal dialogue.  In _2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_.
  * Feng et al. (2022) Yue Feng, Aldo Lipani, Fanghua Ye, Qiang Zhang, and Emine Yilmaz. 2022.  Dynamic schema graph fusion network for multi-domain dialogue state tracking.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_.
  * Feng et al. (2021) Yue Feng, Yang Wang, and Hang Li. 2021.  A sequence-to-sequence approach to dialogue state tracking.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_.
  * Guss et al. (2019) William H. Guss, Brandon Houghton, Nicholay Topin, Phillip Wang, Cayden Codel, Manuela Veloso, and Ruslan Salakhutdinov. 2019.  Minerl: A large-scale dataset of minecraft demonstrations.  In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019_.
  * Janner et al. (2018) Michael Janner, Karthik Narasimhan, and Regina Barzilay. 2018.  Representation learning for grounded spatial reasoning.  _Transactions of the Association for Computational Linguistics_ , 6\.
  * Jayannavar et al. (2020) Prashant Jayannavar, Anjali Narayan-Chen, and Julia Hockenmaier. 2020.  Learning to execute instructions in a Minecraft dialogue.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
  * Johnson et al. (2016) Matthew Johnson, Katja Hofmann, Tim Hutton, and David Bignell. 2016.  The malmo platform for artificial intelligence experimentation.  In _Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016_.
  * Kim and Lipani (2022) To Eun Kim and Aldo Lipani. 2022.  _A Multi-Task Based Neural Model to Simulate Users in Goal-Oriented Dialogue Systems_. Association for Computing Machinery, New York, NY, USA.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Kitaev and Klein (2017) Nikita Kitaev and Dan Klein. 2017.  Where is misty? interpreting spatial descriptors by modeling regions in space.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_.
  * Lachmy et al. (2021) Royi Lachmy, Valentina Pyatkin, and Reut Tsarfaty. 2021.  Draw me a flower: Grounding formal abstract structures stated in informal natural language.  _arXiv preprint arXiv:2106.14321_.
  * Lambert et al. (2019) Charlotte Lambert, Ariel Cordes, Elli Kaplan, Prashant Jayannavar, and Julia Hockenmaier. 2019.  Virtual world context encoding for grounded dialogue in minecraft.
  * Li et al. (2017) Jiwei Li, Alexander H. Miller, Sumit Chopra, Marc’Aurelio Ranzato, and Jason Weston. 2017.  Learning through dialogue interactions by asking questions.  In _ICLR_.
  * Misra et al. (2017) Dipendra Misra, John Langford, and Yoav Artzi. 2017.  Mapping instructions and visual observations to actions with reinforcement learning.  In _EMNLP_.
  * Misra et al. (2018) Ishan Misra, Ross B. Girshick, Rob Fergus, Martial Hebert, Abhinav Gupta, and Laurens van der Maaten. 2018.  Learning by asking questions.  In _2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018_.
  * Narayan-Chen et al. (2019) Anjali Narayan-Chen, Prashant Jayannavar, and Julia Hockenmaier. 2019.  Collaborative dialogue in Minecraft.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_.
  * Narayan-Chen (2020) Anjali Yuan Narayan-Chen. 2020.  _Towards collaborative dialogue in Minecraft_.  Ph.D. thesis, University of Illinois at Urbana-Champaign.
  * Nguyen and Daumé III (2019) Khanh Nguyen and Hal Daumé III. 2019.  Help, anna! visual navigation with natural multimodal assistance via retrospective curiosity-encouraging imitation learning.  In _EMNLP-IJCNLP_.
  * Ni et al. (2021a) Pin Ni, Gangmin Li, Patrick CK Hung, and Victor Chang. 2021a.  Staresgru-cnn with cmedlms: A stacked residual gru-cnn with pre-trained biomedical language models for predictive intelligence.  _Applied Soft Computing_.
  * Ni et al. (2020) Pin Ni, Yuming Li, Gangmin Li, and Victor Chang. 2020.  Natural language understanding approaches based on joint task of intent detection and slot filling for iot voice interaction.  _Neural Computing and Applications_ , 32(20).
  * Ni et al. (2021b) Pin Ni, Yuming Li, Gangmin Li, and Victor Chang. 2021b.  A hybrid siamese neural network for natural language inference in cyber-physical systems.  _ACM Transactions on Internet Technology (TOIT)_ , 21(2).
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014.  GloVe: Global vectors for word representation.  In _EMNLP_.
  * Roman et al. (2020) Homero Roman Roman, Yonatan Bisk, Jesse Thomason, Asli Celikyilmaz, and Jianfeng Gao. 2020.  Rmm: A recursive mental model for dialogue navigation.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 1732–1745.
  * Shi et al. (2022) Zhengxiang Shi, Qiang Zhang, and Aldo Lipani. 2022.  Stepgame: A new benchmark for robust multi-hop spatial reasoning in texts.  In _Association for the Advancement of Artificial Intelligence_.
  * Shu et al. (2018) Tianmin Shu, Caiming Xiong, and Richard Socher. 2018.  Hierarchical and interpretable skill acquisition in multi-task reinforcement learning.  In _ICLR_.
  * Suhr and Artzi (2018) Alane Suhr and Yoav Artzi. 2018.  Situated mapping of sequential instructions to actions with single-step reward observation.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_.
  * Suhr et al. (2019) Alane Suhr, Claudia Yan, Jack Schluger, Stanley Yu, Hadi Khader, Marwa Mouallem, Iris Zhang, and Yoav Artzi. 2019.  Executing instructions in situated collaborative interactions.  In _EMNLP-IJCNLP_.
  * Szlam et al. (2019) Arthur Szlam, Jonathan Gray, Kavya Srinet, Yacine Jernite, Armand Joulin, Gabriel Synnaeve, Douwe Kiela, Haonan Yu, Zhuoyuan Chen, Siddharth Goyal, et al. 2019.  Why build an assistant in minecraft? _arXiv preprint arXiv:1907.09273_.
  * Tan and Bansal (2019) Hao Tan and Mohit Bansal. 2019.  LXMERT: Learning cross-modality encoder representations from transformers.  In _EMNLP-IJCNLP_.
  * Thomason et al. (2020) Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer. 2020.  Vision-and-dialog navigation.  In _Conference on Robot Learning_. PMLR.
  * Wang et al. (2020) Xi Wang, Iadh Ounis, and Craig Macdonald. 2020.  Negative confidence-aware weakly supervised binary classification for effective review helpfulness classification.  CIKM ’20, page 1565–1574, New York, NY, USA. Association for Computing Machinery.
  * Wang et al. (2021) Xi Wang, Iadh Ounis, and Craig Macdonald. 2021.  Leveraging review properties for effective recommendation.  In _Proceedings of the Web Conference 2021_ , WWW ’21, page 2209–2219, New York, NY, USA. Association for Computing Machinery.
  * Wei et al. (2018) Wei Wei, Quoc Le, Andrew Dai, and Jia Li. 2018.  AirDialogue: An environment for goal-oriented dialogue research.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_.
  * Williams et al. (2014) Jason D Williams, Matthew Henderson, Antoine Raux, Blaise Thomson, Alan Black, and Deepak Ramachandran. 2014.  The dialog state tracking challenge series.  _AI Magazine_ , 35(4).
  * Wu et al. (2019) Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung. 2019.  Transferable multi-domain state generator for task-oriented dialogue systems.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_.
  * Yang et al. (2020) Tsung-Yen Yang, Andrew Lan, and Karthik Narasimhan. 2020.  Robust and interpretable grounding of spatial references with relation networks.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_.
  * Zhu et al. (2021) Yi Zhu, Yue Weng, Fengda Zhu, Xiaodan Liang, Qixiang Ye, Yutong Lu, and Jianbin Jiao. 2021.  Self-motivated communication agent for real-world vision-dialog navigation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_.

##  Appendix A Appendix

Training Details. We examine our model on the extend Minecraft Dialogue Corpus
dataset. Our model’s hyper-parameters are fixed for all three experiments in
the Sec 5 as follows. The number of 3D-conv layers $k$ is 3, the dimension of
each grid representation $d_{c}$ is 300, the number of layers of the grid
cross-modality modules $N_{G}$ is 4, and the number of layers of the text
cross-modality modules $N_{T}$ is 2. The max length of the dialogue context
$s$ is selected as 100 and the dropout rates are all set to 0.2. The number of
heads for the attention mechanism in the text singe and cross modality modules
is set to 2, while the number of heads is set to 1 for the attention mechanism
in the grid singe and cross modality modules. The cross entropy loss from the
location slot is not counted if the ground truth label of the action type is
not ‘placement’ or ’removal’, and the cross entropy loss from the color slot
is not counted if the ground truth label of the action type is not
’placement’. The ground-truth of previous actions and the world state is
provided to the models during the training and testing. For the experiment in
the Sec 5.2 and 5.3, we randomly sample from ’Ask’ and ’Others’ sets in the
training set to make training samples of different action types (’Execution’,
’Ask’, and ’Others’) in the training set balanced. We train our model with
cross entropy loss functions of all slots and a batch size of 50, using Adam
optimizer Kingma and Ba (2015) with a learning rate of 1e-6, $\beta_{1}$ = 0.9
and $\beta_{2}$ = 0.99. We train our model with 50 epochs and select the model
with the highest F1-score on the valid set.