# The Physics of Preference: Unravelling Imprecision of Human Preferences
through Magnetisation Dynamics

Ivan S. Maksymov  Artificial Intelligence and Cyber Futures Institute, Charles
Sturt University, Bathurst, NSW 2795, Australia  Ganna Pogrebna  Artificial
Intelligence and Cyber Futures Institute, Charles Sturt University, Bathurst,
NSW 2795, Australia  The Alan Turing Institute, British Library, London, NW1
2DB, United Kingdom  The University of Sydney Business School, Darlington, NSW
2006, Australia

###### Abstract

Paradoxical decision-making behaviours such as preference reversal often arise
from imprecise or noisy human preferences. By harnessing the physical
principle of magnetisation reversal in ferromagnetic nanostructures driven by
electric current, we developed a model that closely reflects human decision-
making dynamics. Tested against a spectrum of psychological data, our model
adeptly captures the complexities inherent in individual choices. This blend
of physics and psychology paves the way for fresh perspectives on
understanding human decision-making processes.

Introduction—Theoretical and experimental research in economics and psychology
posits that human preferences are inherently noisy and imprecise [1] (see [2,
3] for a definition of noise and imprecision). For example, such phenomena as
Allais Paradox, Ellberg Paradox, Endowment Effect and Preference Reversal (PR)
demonstrate that human decisions deviate from predictions of traditional
rational models [4]. These deviations follow certain patterns [3]. While the
classical theory suggests that individuals consistently maximise their utility
based on stable preferences, experiments present situations where choices are
inconsistent or even contradictory [5, 6].

PR was first observed in the gambling studies involving [7]: (i) a choice
task, where people were asked to decide between a relatively safe lottery with
a low payoff (Probability- or P-bet) and a relatively risky lottery with a
high payoff (Dollar- or $-bet) as well as (ii) a valuation task, where the
same people were asked to indicate lottery prices if they were to sell P-bet
and $-bet. The observed people’s preferences were often contradictory: opting
for the P-bet they put a higher price on the $-bet.

Alongside other behavioural irregularities, PR has reshaped contemporary
discourse in both economics and psychology. Traditional models such as
Expected Utility Theory (EUT), which held that individual choices were
reflective of consistent and stable preferences, have been challenged by
empirical findings [8]. The existence of such paradoxes implied that human
decision-making was more multifaceted than previously assumed.

Understanding and accurately predicting these anomalies is not merely an
academic exercise. As policy-makers strive to devise strategies and
interventions that depend on opinion polls, the need for models predicting
human behaviour is pressing. Yet, capturing the essence of phenomena like PR
within traditional models remains a challenge. For example, transitivity
posits that if an individual prefers $A$ to $B$ and $B$ to $C$, then they
should prefer $A$ to $C$. However, PR defies the transitivity, suggesting that
human preferences might be imprecise.

Figure 1: (a) Idealised deterministic (the dotted line) and probabilistic (the
line with circular markers) PR curves. (b) Simulated magnetisation reversal
behaviour for different values of the static magnetic field $H_{0}$.

This instigates debates on the need to (i) reconsider some axioms of decision
theory [9]; (ii) devise theories relaxing transitivity assumption [10]; (iii)
embed decision theories into models of noise-synthesising stochastic versions
of known theories [5]; and (iv) develop models that assume that preferences
are inherently imprecise [3].

The latter approach has demonstrated that individual choices under risk are
probabilistic rather than deterministic, i.e. the same person may make
different choices when presented with the same problem multiple times.
Experiments show that people’s decisions to accept or refuse sure payoff
amounts in exchange for a risky lottery varies from consistently selecting bet
over cash (when the sure payoff amounts offered against the lottery are not
sufficiently attractive) to consistently selecting cash over bet (when the
sure payoff amounts increase) with some imprecision interval in-between, when
individuals make probabilistic choices between cash and the bet [11, 12, 3].

While these debates are ongoing, it has been noticed that physical processes
can adequately describe many psychological effects and decision-making
phenomena. For example, there exist classical physical models of confirmation
bias [13], public opinion formation [14, 15, 16] and backfire effect [17].
Quantum-mechanical models of decision-making [18, 19, 20] that originate from
the quantum mind hypothesis [21, 22] have also been developed.

However, there remains an imperative to juxtapose the predictions of such
models against empirical findings. In this pursuit, our paper not only
elucidates the connections between physics and decision theory, but also
provides a comprehensive evaluation of a real-life physical model against rich
empirical datasets. Through this endeavour, we aim to herald a paradigm shift
in understanding and predicting human decision-making.

Study 1—In a study of imprecision intervals [11, 12, 3], an individual is
given a number of choices between a lottery $B$ and a series of increasing
sure payoff amounts $A_{j}$, $j=0,\dots,13$. Since each choice is made
multiple times and independently of the previous ones, we model this
individual’s preferences as a probability distribution.

Figure 1a shows two theoretically possible distributions. The dotted line
denotes the deterministic preferences of an individual who chooses $B$ when
$j<6$, never chooses $B$ when $j>6$ and is exactly indifferent between $A$ and
$B$ when $j=6$. The curve with the circular markers denotes the probabilistic
preferences. When the sure thing $ST\leq A_{3}$, the individual always chooses
$B$. When $ST\geq A_{10}$, the individual never chooses $B$. For
$A_{3}<A<A_{10}$ there is an imprecision interval.

The probabilistic distribution in Fig. 1a is drawn as a sigmoid-like curve
since this function robustly fits many intuitions [3] and datasets [11, 12, 3,
5, 9]. Figure 2 shows five examples of experimental curves (median patterns)
plotted using the data obtained from the laboratory decision-making
experiments with human subjects.

All experiments were conducted in the University of Warwick using subject
pools of undergraduate, postgraduate and executive students over the age of
18. In Fig. 2a, 81 subjects were making a series of binary choices between a
lottery which gave £40 with probability 80% or 0 with probability 20% and sure
payoff amounts between £16 and £36 with a step of £2. In Fig. 2b, 101 subjects
were offered a lottery which yielded £40 with probability 30% or 0 with
probability 70% versus sure amounts of money between £4 and £12 with a step of
£1. In Fig. 2c, 101 experimental participants make choices between a lottery
providing £15 with probability 70% and 0 otherwise against sure amounts
between £4 and £12 with a step of £1. In Fig. 2d, 184 subjects face a choice
between a chance of winning £12 with probability 80% and sure amounts from £3
to £11 with a step of £1. In Fig. 2e, 184 people decide between £15 with
probability 25% and sure monetary amounts between £3 to £11 with a step of £1.
In all lotteries, each person made each of the binary choices 4 times. We can
see that all five examples demonstrate robust evidence in favour of the
existence of an imprecision interval, where people make probabilistic choices
between a lottery and sure monetary amounts.

Figure 2: Experimental PR data of the five lotteries. The straight dotted
lines are the prediction of EUT under the assumption that the participants are
risk-neutral.

To model the experimental decision-making behaviour, we employ magnetisation
dynamics in ferromagnetic metal (FM) structures. In general, an electric
current is unpolarised since it involves electrons with a random polarisation
of spins. However, a current that passes through a thin FM film with a fixed
magnetisation direction can become spin-polarised since spins become oriented
predominantly in the same direction [23]. This physical effect is exploited in
a spin transfer torque nano-oscillator (STNO), where a layer with a fixed
direction of magnetisation ${\bf M}$ is separated from a thinner FM layer by a
non-magnetic metal layer [24]. When a spin-polarised current is directed from
the “fixed” magnetisation layer into the “free” magnetisation layer, the
static equilibrium orientation of magnetisation in the “free” layer becomes
destabilised. Depending on the strength of the electric current, the
destabilisation of magnetisation can lead to either stable precession of
magnetisation of the “free” layer about the direction of the effective
magnetic field or to a reversal of the magnetisation direction (Fig. 3).

Figure 3: (a) Sketch of magnetisation vector $M$ precession. Simulated
results: (b) ${\bf M}$ spirals back toward the direction of ${\bf H}_{eff}$
due to the damping. (c) The spin transfer torque compensates the damping,
resulting in a stable precession of ${\bf M}$ around ${\bf H}_{eff}$. (d)
Reversal of the direction of ${\bf M}$. The applied field is $H_{0}=8$ kOe.
The sphere is used for visualisation only.

Using the numerical method described below, in Fig. 1b we plot the dynamics of
the $M_{z}$ component of the magnetisation vector ${\bf M}$ in the “free”
layer as a function of the electric current strength for different value of
the applied magnetic field $H_{0}$. The trajectories of ${\bf M}$ for
$H_{0}=8$ kOe computed at the simulated time of 1 ns are shown in Fig. 3b–c.
Figure 1b demonstrates that the magnetisation dynamics correctly approximates
the sigmoid probabilistic PR curve in Fig. 1a but the strength of the driving
electric current plays the role of parameter $A$. The experimental curves in
Fig. 2 can be regarded as the particular cases of the sigmoid curve and they
can be approximated using magnetisation dynamics at different values of
$H_{0}$, which is demonstrated below.

Study 2—We validate our model using a dataset obtained from a “Deal or No
Deal” video game based on a popular TV show, where contestants choose boxes
with concealed cash amounts and negotiate with a banker to accept an offer or
keep opening boxes. The aim is to secure the highest prize, avoiding smaller
sums. Once a box is opened, its prize is shown and removed from potential
winnings. Essentially, players choose between risky lotteries and certain cash
offers. They also may exchange their box for any unopened one in the game. The
decision to swap the box or to stick to the original one enables us to study
whether the participants behave according to EUT or to Cumulative Prospect
Theory (CPT). A previous study [25] that used TV show data from three
countries demonstrated that: (i) CPT players are more likely to stick to their
original choice of the box due to the embedded assumption of loss aversion and
(ii) EUT players are indifferent between swapping and sticking.

We recruited 78 study participants (members of the general public over 18
years of age) who made a total of 1,698 game decisions, including 486 swap or
stick decisions. The fact that 63% of decisions were stick decisions and 37%
were swap decisions demonstrated a higher consistency with EUT than with CPT.
However, Cumulative Distribution Function (CDF) of swap or stick decisions
revealed an intriguing result: while EUT predicts an approximately equal split
between swapping or sticking decisions (the straight dash-dotted lines in Fig.
4), the distribution of swap or stick decisions (the dotted curve in Fig. 4)
cannot be described by EUT.

Since the magnetisation dynamics depends on the static field $H_{0}$ (Fig.
1b), in Fig. 4 we plot a magnetisation reversal that is recast as
$(M_{z}/M_{s}+1)/2$ for the sake of comparison with the experimental data. The
resulting curve satisfactorily reproduces the experimental decision-making
behaviour demonstrated by the players of the game. The values of $H_{0}$
corresponding to the data points of this curve are, from left to right: 15, 6,
8,75, 9,6, 9,95, 10, 10, 10.75, 11, 11.52, 11.99, 12.1 and 12.5 kOe.

Figure 4: Top $x$ and right $y$ axes: Cumulative probability distribution
function for sticking decisions in the video game. The dash-dotted straight
lines denote the EUT propensity to stick. The dashed curve corresponds to the
experimental data. Bottom $x$ and left $y$ axes: The circular markers denote
the simulated magnetisation reversal. The thin solid line is the guide to the
eye only.

Discussion—The functions of a human brain underpinning the phenomena like PR
can be comprehended in detail through neuroscience methods [26]. Nonetheless,
understanding human behaviour is pivotal not just for neuroscience but also
for fields of finance, politics, religion and sociology. Thus, there is a
demand for phenomenological models that can forecast decisions at both
individual and societal levels quickly and efficiently.

These models should not necessarily be based on neurobiological insights,
mirroring the human cognition (drawing parallels with biological brain
processes might even be unhelpful [27]). The primary criteria for evaluating a
model of decision-making should be the ability of the model to accurately
explain and predict imprecise and noisy decisions. The model we introduce in
this paper follows this philosophy, aligning with the ongoing work in decision
theory where models are not necessarily capturing human brain functions [28].

STNO has been a foundational component in neuromorphic hardware systems that
emulate the functions of the human brain [29, 30]. The operational principles
of STNO obey the laws of quantum mechanics [31], even though our model’s Eq.
(1) is classical. Quantum-mechanical models have demonstrated superior
capacity in depicting decision-making compared with classical models [20].
Therefore, leveraging a realistic STNO structure in our model hints at the
potential of STNOs to act as a neuromorphic system for simulating human
decision-making processes. The so-designed system can be integrated with a
reservoir computing algorithm [32], resulting in a machine learning technique
that has an enhanced learning efficacy compared with classical systems [33]
and also can simulate the human decision-making.

Other quantum-mechanical systems can replicate decision-making [34]. However,
the magnetisation reversal in STNO naturally extends important models of
opinion shifts in social networks based on spin direction reversal [16]. Even
though those models adopted the spin concept from physics, the processes
causing the spin changes were largely synthetic. Thus, integrating a genuine
model of magnetisation reversal into the opinion shift model [16] will enhance
the efficacy of the latter, enabling it to analyse diverse decision-making
scenarios.

Conclusions—We demonstrated that the psychological phenomenon of PR can be
modelled using the physical phenomenon of magnetisation reversal. Using a spin
transfer torque as the mechanism of magnetisation reversal, we revealed the
ability of the magnetisation reversal model to capture complex individual’s
decisions made in psychological experiments. We have also shown that our model
extends beyond PR and encompasses a wide range of decision-making
regularities, reaffirming the observation that human preferences are
imprecise.

While our work bolsters the exploration of physics as a lens to comprehend
human psychology, its tangible implications are profound: the model holds
promise for designing neuromorphic hardware systems rooted in nanomagnetic
devices. Such interdisciplinary convergence has the potential to reshape the
way we interface with both the human mind and advanced computational systems.

The implication of magnetisation reversal as a model for human decision-making
not only accentuates the universality of certain principles across ostensibly
disparate domains but also underscores the rich tapestry of relationships
waiting to be unearthed between the physical and cognitive realms. For
example, the integration of these findings can stimulate a paradigm shift in
how we design and conceptualise next-generation cognitive devices. Moreover,
the ability of physical effects to emulate cognitive processes suggests a
future where the boundaries between cognition and artificial intelligence
blur, leading to new possibilities for collaborative intelligence.

Numerical method—We model the dynamics of magnetisation using Landau-Lifshitz-
Gilbert equation [35]:

$\partial{\bf M}/\partial t=\gamma\left[{\bf H}_{eff}\times\bf M\right]+{\bf
T}_{G}+{\bf T}_{SB}\,,$ (1)

where $\gamma$ is the gyromagnetic ratio. The first term of the right-hand
side of Eq. (1) governs the precession of $\bf M$ (Fig. 1a) about the
direction of the effective magnetic field
$\textbf{H}_{eff}=H_{0}\textbf{e}_{\rm{z}}+\textbf{H}+\textbf{H}_{ex}$, where
$H_{0}$ is the external static magnetic field orientated along the z-axis, H
is the dynamic field due to currents and magnetic sources such as
demagnetising field and eddy current fields and $\textbf{H}_{ex}$ is the
exchange field [36]. The dissipative torque is [35]

${\bf T}_{G}=\alpha_{G}M_{s}^{-1}\left[{\bf M}\times{\partial\bf M}/{\partial
t}\right]\,,$ (2)

where $M_{s}$ is the saturation magnetisation of the “free” magnetisation
layer and $\alpha_{G}$ is Gilbert damping parameter [36, 35]. The Slonczewski-
Berger torque is

${\bf T}_{SB}=\sigma_{0}IM_{s}^{-1}\left[{\bf M}\times\left[{\bf
M}\times\hat{e}_{p}\right]\right]\,,$ (3)

where $I$ is the strength and $\hat{e}_{p}$ is the direction of the spin
polarisation of the current. The parameter $\sigma_{0}$ incorporates
fundamental physical constants and the thickness of the “free” magnetisation
layer [35]. Equation (1) is solved consistently with Maxwell’s equations using
a finite-difference time-domain (FDTD) method and material parameters
presented in [36]. We model a point-contact STNO structure experimentally
studied in [24]. Since we employ a one-dimensional FDTD scheme, the strength
of the current $I$ required to produce a magnetisation reversal in our model
is lower than that in [24].

## References

  * Lindman [1971] H. Lindman, Inconsistent preferences among gambles, J. Experiment. Psych. 89, 390 (1971).
  * Bhatia and Loomes [2017] S. Bhatia and G. Loomes, Noisy preferences in risky choice: A cautionary note, Psychol. Rev. 124, 678 (2017).
  * Loomes and Pogrebna [2017] G. Loomes and G. Pogrebna, Do preference reversals disappear when we allow for probabilistic choice?, Manage. Sci. 63, 166 (2017).
  * Moffatt  _et al._ [2009] P. Moffatt, C. Starmer, R. Sugden, N. Bardsley, R. Cubitt, and G. Loomes,  _Experimental economics: Rethinking the rules_ (Princeton University Press, 2009).
  * Blavatskyy and Pogrebna [2010a] P. R. Blavatskyy and G. Pogrebna, Models of stochastic choice and decision theories: Why both are important for analyzing decisions, J. Appl. Econom. 25, 963 (2010a).
  * Bardsley  _et al._ [2009] N. Bardsley, R. Cubitt, G. Loomes, P. Moffatt, C. Starmer, and R. Sugden, Experimental economics, in  _Experimental Economics_ (Princeton University Press, 2009).
  * Lichtenstein and Slovic [1971] S. Lichtenstein and P. Slovic, Reversals of preference between bids and choices in gambling decisions, J. Experiment. Psych. 89, 46 (1971).
  * Grether and Plott [1979] D. Grether and C. Plott, Economic theory of choice and the preference reversal phenomenon, Amer. Econom. Rev. 69, 623 (1979).
  * Butler and Pogrebna [2018] D. J. Butler and G. Pogrebna, Predictably intransitive preferences, Judgm. Decis. Mak. 13, 217 (2018).
  * Loomes and Sugden [1982] G. Loomes and R. Sugden, Regret theory: An alternative theory of rational choice under uncertainty, Econ. J. 92, 805 (1982).
  * Mosteller and Nogee [1951] F. Mosteller and P. Nogee, An experimental measurement of utility, J. Pol. Econ. 59, 371 (1951).
  * Loomes and Pogrebna [2014] G. Loomes and G. Pogrebna, Testing for independence while allowing for probabilistic choice, J. Risk Uncertain. 49, 189 (2014).
  * Gronchi and Strambini [2017] G. Gronchi and E. Strambini, Quantum cognition and Bell’s inequality: A model for probabilistic judgment bias, J. Math. Psychol. 78, 65 (2017).
  * Galam [2005] S. Galam, Heterogeneous beliefs, segregation, and extremism in the making of public opinions, Phys. Rev. E 71, 046123 (2005).
  * Castellano  _et al._ [2009] C. Castellano, S. Fortunato, and V. Loreto, Statistical physics of social dynamics, Rev. Mod. Phys. 81, 591 (2009).
  * Redner [2019] S. Redner, Reality-inspired voter models: A mini-review, C. R. Phys. 20, 275 (2019).
  * Hohmann  _et al._ [2023] M. Hohmann, K. Devriendt, and M. Coscia, Quantifying ideological polarization on a network using generalized Euclidean distance, Sci. Adv. 9, eabq2044 (2023).
  * Atmanspacher and Filk [2010] H. Atmanspacher and T. Filk, A proposed test of temporal nonlocality in bistable perception, J. Math. Psychol. 54, 314 (2010).
  * Aerts  _et al._ [2014] D. Aerts, S. Sozzo, and J. Tapia, Identifying quantum structures in the Ellsberg paradox, Int. J. Theor. Phys 53, 3666 (2014).
  * Busemeyer and Bruza [2012] J. R. Busemeyer and P. D. Bruza,  _Quantum Models of Cognition and Decision_ (Oxford University Press, 2012).
  * Khrennikov [2006] A. Khrennikov, Quantum-like brain: “interference of minds”, Biosystems 84, 225 (2006).
  * Mindell [2012] A. Mindell,  _Quantum Mind: The Edge Between Physics and Psychology_ (Deep Democracy Exchange, 2012).
  * Ralph and Stiles [2008] D. C. Ralph and M. D. Stiles, Spin transfer torques, J. Magn. Magn. Mater. 320, 1190 (2008).
  * Rippard  _et al._ [2004] W. H. Rippard, M. R. Pufall, S. Kaka, S. E. Russek, and T. J. Silva, Direct-current induced dynamics in Co90Fe10/Ni80Fe20 point contacts, Phys. Rev. Lett. 92, 027201 (2004).
  * Blavatskyy and Pogrebna [2010b] P. Blavatskyy and G. Pogrebna, Endowment effects? “Even” with half a million on the table!, Theory Decis. 68, 173 (2010b).
  * Sharma and Khan [2021] S. N. Sharma and A. Khan, Intertemporal preference reversals are associated with early activation of insula and sustained preferential processing of immediate rewards in visual cortex, Sci. Rep. 11, 22277 (2021).
  * Taddeo [2016] M. Taddeo, On the risks of relying on analogies to understand cyber conflicts, Minds Mach. 26, 317 (2016).
  * Wakker [2010] P. P. Wakker,  _Prospect theory: For risk and ambiguity_ (Cambridge University Press, 2010).
  * Furuta  _et al._ [2018] T. Furuta, K. Fujii, K. Nakajima, S. Tsunegi, H. Kubota, Y. Suzuki, and S. Miwa, Macromagnetic simulation for reservoir computing utilizing spin dynamics in magnetic tunnel junctions, Phys. Rev. Appl. 10, 034063 (2018).
  * Riou  _et al._ [2019] M. Riou, J. Torrejon, B. Garitaine, F. A. Araujo, P. Bortolotti, V. Cros, S. Tsunegi, K. Yakushiji, A. Fukushima, H. Kubota, S. Yuasa, D. Querlioz, M. D. Stiles, and J. Grollier, Temporal pattern recognition with delayed-feedback spin-torque nano-oscillators, Phys. Rep. Appl. 12, 024049 (2019).
  * Brataas [2022] A. Brataas, Quantum scattering theory of spin transfer torque, spin pumping, and fluctuations, Phys. Rev. B 106, 064402 (2022).
  * Maksymov [2023] I. S. Maksymov, Analogue and physical reservoir computing using water waves: Applications in power engineering and beyond, Energies 16, 5366 (2023).
  * Dudas  _et al._ [2023] J. Dudas, B. Carles, E. Plouet, F. A. Mizrahi, J. Grollier, and D. Marković, Quantum reservoir computing implementation on coherently coupled quantum oscillators, NPJ Quantum Inf. 9, 64 (2023).
  * Naruse  _et al._ [2014] M. Naruse, W. Nomura, M. Aono, M. Ohtsu, Y. Sonnefraud, A. Drezet, S. Huant, and S.-J. Kim, Decision making based on optical excitation transfer via near-field interactions between quantum dots, J. Appl. Phys. 116, 154303 (2014).
  * Slavin and Tiberkevich [2009] A. Slavin and V. Tiberkevich, Nonlinear auto-oscillator theory of microwave generation by spin-polarized current, IEEE Trans. Magn. 45, 1875 (2009).
  * Maksymov and Kostylev [2013] I. S. Maksymov and M. Kostylev, Impact of conducting nonmagnetic layers on the magnetization dynamics in thin-film magnetic nanostructures, J. Appl. Phys. 113, 043927 (2013).


# Unsupervised Representation Disentanglement of Text:
An Evaluation on Synthetic Datasets

Lan Zhang♠ Victor Prokhorov♣ Ehsan Shareghi♠♣
♠ Department of Data Science & AI, Monash University
♣ Language Technology Lab, University of Cambridge
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

To highlight the challenges of achieving representation disentanglement for
text domain in an unsupervised setting, in this paper we select a
representative set of successfully applied models from the image domain. We
evaluate these models on 6 disentanglement metrics, as well as on downstream
classification tasks and homotopy. To facilitate the evaluation, we propose
two synthetic datasets with known generative factors. Our experiments
highlight the existing gap in the text domain and illustrate that certain
elements such as representation sparsity (as an inductive bias), or
representation coupling with the decoder could impact disentanglement. To the
best of our knowledge, our work is the first attempt on the intersection of
unsupervised representation disentanglement and text, and provides the
experimental framework and datasets for examining future developments in this
direction.111Code and datasets are available at
https://github.com/lanzhang128/disentanglement

##  1 Introduction

Learning task-agnostic unsupervised representations of data has been the
center of attention across various areas of Machine Learning and more
specifically NLP. However, little is known about the way these continuous
representations organise information about data. In recent years, the NLP
community has focused on the question of design and selection of suitable
linguistic tasks to probe the presence of syntactic or semantic phenomena in
representations as a whole (Bosc and Vincent, 2020; Voita and Titov, 2020;
Torroba Hennigen et al., 2020; Pimentel et al., 2020; Hewitt and Liang, 2019;
Ettinger et al., 2018; Marvin and Linzen, 2018; Conneau et al., 2018).
Nonetheless, a fine-grain understanding of information organisation in
coordinates of a continuous representation is yet to be achieved.

Arguably, a necessity to move in this direction is agreeing on the cognitive
process behind language generation (fusing semantic, syntactic, and lexical
components), which can then be reflected in the design of representation
learning frameworks. However, this still remains generally as an area of
debate and perhaps less pertinent in the era of self-supervised masked
language models and the resulting surge of new state-of-the-art results.

Even in the presence of such an agreement, learning to disentangle the surface
realization of the underlying factors of data (e.g., semantics, syntactic,
lexical) in the representation space is a non-trivial task. Additionally,
there is no established study for evaluating such models in NLP. A handful of
recent works have looked into disentanglement for text by splitting the
representation space into _predefined_ disentangled subspaces such as style
and content Cheng et al. (2020); John et al. (2019), or syntax and semantics
Balasubramanian et al. (2021); Bao et al. (2019); Chen et al. (2019), and rely
on _supervision_ during training. However, a generalizable and realistic
approach needs to be _unsupervised_ and capable of identifying the underlying
factors solely via the regularities presented in data.

In areas such as image processing, the same question has been receiving a lot
of attention and inspired a wave of methods for learning and evaluating
unsupervised representation disentanglement (Ross and Doshi-Velez, 2021;
Mathieu et al., 2019; Kim and Mnih, 2018; Burgess et al., 2018; Higgins et
al., 2018, 2017) and creation of large scale datasets Dittadi et al. (2021).
It has been argued that disentanglement is the means towards representation
interpretability Mathieu et al. (2019), generalization Montero et al. (2021),
and robustness Bengio et al. (2013); Bengio (2013). However, these benefits
are yet to be realized and evaluated in text domain.

In this work we take a representative set of _unsupervised_ disentanglement
learning frameworks widely used in image domain (§2.1) and apply them to two
artificially created corpora with known underlying generative factors (§3).
Having known generative factors (while being ignored during the training
phase) allows us to evaluate the performance of these models on imposing
representation disentanglement via 6 disentanglement metrics (§2.2; §4.1).
Additionally, taking the highest scoring models and corresponding
representations, we investigate the impact of representation disentanglement
on two downstream text classification tasks (§4.3), and dimension-wise
homotopy (§4.4).

We show that existing disentanglement models, when evaluated on a wide range
of metrics, are inconsistent and highly sensitive to model initialisation.
However, where disentanglement is achieved, it shows its positive impact on
improving downstream task performance. Our work highlights the potential and
existing challenges of disentanglement on text. We hope our proposed datasets,
accessible description of disentanglement metrics and models, and experimental
framework will set the path for developments of models specific to for text.

##  2 Disentanglement Models and Metrics

Let $\mathbf{x}$ denote data points and $\mathbf{z}$ denote latent variables
in the latent representation space, and assume data points are generated by
the combination of two random process: The first random process samples a
point $\mathbf{z}^{(i)}$ from the latent space with prior distribution of
$\mathbf{z}$, denoted by $p(\mathbf{z})$. The second random process generates
a point $\mathbf{x}^{(i)}$ from the data space, denoted by
$p(\mathbf{x}|\mathbf{z}^{(i)})$.

We consider $\mathbf{z}$ as a disentangled representation for $\mathbf{x}$, if
the changes in single latent dimensions of $\mathbf{z}$ are sensitive to
changes in single generative factors of $\mathbf{x}$ while being relatively
invariant to changes in other factors Bengio et al. (2013). Several
probabilistic models are designed to reveal this process, here we look at some
of the most widely used ones.

###  2.1 Disentanglement Models

A prominent approach for learning disentangled representations is through
adjusting Variational Auto-Encoders (VAEs) (Kingma and Welling, 2014)
objective function, which decompose the representation space into
independently learned coordinates. We start by introducing vanilla VAE, and
then cover some of its widely used extensions that encourage disentanglement:

##### VAE

uses a combination of a probabilistic encoder
$q_{\phi}(\mathbf{z}|\mathbf{x})$ and decoder
$p_{\theta}(\mathbf{x}|\mathbf{z})$, parameterised by $\phi$ and $\theta$, to
learn this statistical relationship between $\mathbf{x}$ and $\mathbf{z}$. The
VAEs are trained by maximizing the lower bound of the logarithmic data
distribution $\log p(\mathbf{x})$, called evidence lower bound,

$\begin{aligned} \mathbb{E}_{q_{\phi}(\mathbf{z}|\mathbf{x})}\big{[}\log
p_{\theta}(\mathbf{x}|\mathbf{z})\big{]}-\mathbb{D}_{KL}(q_{\phi}(\mathbf{z}|\mathbf{x}),p(\mathbf{z}))\end{aligned}$

The first term of is the expectation of the logarithm of data likelihood under
the posterior distribution of $z$. The second term is KL-divergence, measuring
the distance between the posterior distribution
$q_{\phi}(\mathbf{z}|\mathbf{x})$ and the prior distribution $p(\mathbf{z})$
and can be seen as a regularisation.

#####  $\mathbf{\beta}$-VAE

(Higgins et al., 2017) adds a hyperparameter $\beta$ to control the
regularisation from the KL-term via the following objective function:

$\begin{aligned} \mathbb{E}_{q_{\phi}(\mathbf{z}|\mathbf{x})}\big{[}\log
p_{\theta}(\mathbf{x}|\mathbf{z})\big{]}-\beta\mathbb{D}_{KL}(q_{\phi}(\mathbf{z}|\mathbf{x}),p(\mathbf{z}))\end{aligned}$

Reconstructing under $\beta$-VAE (with the right value of $\beta$) framework
encourages encoding data points on a set of representational axes on which
nearby points along those dimensions are also close in original data space
Burgess et al. (2018).

##### CCI-VAE

(Burgess et al., 2018) extends $\beta$-VAE via constraint optimisation:

$\begin{aligned} \mathbb{E}_{q_{\phi}(\mathbf{z}|\mathbf{x})}\big{[}\log
p_{\theta}(\mathbf{x}|\mathbf{z})\big{]}-\beta\left|\mathbb{D}_{KL}(q_{\phi}(\mathbf{z}|\mathbf{x}),p(\mathbf{z}))-C\right|\end{aligned}$

where $C$ is a positive real value which represents the target KL-divergence
term value. This has an information-theoretic interpretation, where the placed
constraint $C$ on the KL term is seen as the amount of information transmitted
from a sender (encoder) to a receiver (decoder) via the message ($\mathbf{z}$)
Alemi et al. (2018), and impacts the sharpness of the posterior distribution
Prokhorov et al. (2019). This constraint allows the model to prioritize
underlying factors of data according to the availability of channel capacity
and their contributions to the reconstruction loss improvement.

##### MAT-VAE

Mathieu et al. (2019) introduces an additional term to $\beta$-VAE,
$\mathbb{D}_{MMD}(q_{\phi}(\mathbf{z}),p_{\theta}(\mathbf{z}))$,

$\begin{split}\mathbb{E}_{q_{\phi}(\mathbf{z}|\mathbf{x})}[\log
p_{\theta}(\mathbf{x}|\mathbf{z})]-\beta\mathbb{D}_{KL}(q_{\phi}(\mathbf{z}|\mathbf{x}),p(\mathbf{z}))\\\
-\lambda\mathbb{D}_{MMD}(q_{\phi}(\mathbf{z}),p(\mathbf{z}))\end{split}$

where $\mathbb{D}_{MMD}$ is computed using maximum mean discrepancy (Gretton
et al. (2012), MMD) and $\lambda$ is the scalar weight. This term regularises
the aggregated posterior $q_{\phi}(\mathbf{z})$ with a factorised spike-and-
slab prior Mitchell and Beauchamp (1988), which aims for disentanglement via
clustering and sparsifying the representations of $\mathbf{z}$.

####  2.1.1 Issue of KL-Collapse

In text modelling, the presence of powerful auto-regressive decoders poses a
common optimisation challenge for training VAEs called posterior collapse,
where the learned posterior distribution $q_{\phi}(\mathbf{z}|\mathbf{x})$,
collapses to the prior $p(\mathbf{z})$. Posterior collapse results in the
latent variables $\mathbf{z}$ being ignored by the decoder. Several strategies
have been proposed to alleviate this problem from different angles such as
choice of decoders (Yang et al., 2017; Bowman et al., 2016), adding more
dependency between encoder and decoder (Dieng et al., 2019), adjusting the
training process (Bowman et al., 2016; He et al., 2019), imposing direct
constraints to the KL term (Pelsmaeker and Aziz, 2020; Razavi et al., 2019;
Burgess et al., 2018; Higgins et al., 2017). In this work, both $\beta$-VAE
(with $\beta<1$) and CCI-VAE are effective methods to avoid KL-collpase.

###  2.2 Disentanglement Metrics

In this section we provide a short overview of six widely used disentanglement
metrics, highlighting their key differences and commonalities, and refer the
readers to the corresponding papers for exact details of computations.

Eastwood and Williams (2018) define three criteria for disentangled
representations: _disentanglement_ , which measures the degree of one
dimension only encoding information about no more than one generative factor;
_completeness_ , which measures whether a generative factor is only captured
by one latent variable; _informativeness_ , which measures the degree by which
representations capture exact values of the generative factors.222These
criteria are referred to modularity, compactness and explicitness by Ridgeway
and Mozer (2018). They design a series of classification tasks to predict the
value of a generative factor based on the latent code, and extract the
relative importance of each latent code for each task to calculate
disentanglement and completeness scores. Informativeness score is measured by
the accuracy of the classifier directly. Other existing metrics reflect at
least one of these three criteria, as summarised in Table 1.

Higgins et al. (2017) focus on disentanglement and propose to use the absolute
difference of two groups of representations with the same value on one
generative factor to predict this generative factor. For perfectly
disentangled representations, latent dimensions not encoding information about
this generative factor would have zero difference. Hence, even simple linear
classifiers could easily identify the generative factors based on the changes
of values. Kim and Mnih (2018) consider both disentanglement and completeness
by first finding the dimension which has the largest variance when fixing the
value on one generative factor, and then using the found dimension to predict
that generative factor. Kumar et al. (2018) propose a series of classification
tasks each of which uses a single latent variable to predict the value of a
generative factor and treat the average of the difference between the top two
accuracy scores for each generative factor as the final disentanglement score.

Apart from designing classification tasks for disentanglement evaluation,
another method is based on estimating the mutual information (MI) between a
single dimension of the latent variable and a single generative factor. Chen
et al. (2018) propose to use the average of the gap (difference) between the
largest normalised MI (by the information entropy of the generative factor)
and the second largest normalised MI over all generative factors as the
disentanglement score, whereas the modularity metric of Ridgeway and Mozer
(2018) measures whether a single latent variable has the highest MI with only
one generative factor and none with others.

Metric | Dis. | Com. | Info. | Ex.1$\uparrow$ | Ex.2$\uparrow$
---|---|---|---|---|---
Higgins et al. (2017) | Yes | No | No | 100 | 100
Ridgeway and Mozer (2018) | Yes | No | No | 100 | 100
Kim and Mnih (2018) | Yes | Yes | No | 100 | 100
Chen et al. (2018) | No | Yes | No | 81.05 | 5.73
Eastwood and Williams (2018) | Yes | Yes | Yes | 66.47 | 63.45
Kumar et al. (2018) | No | Yes | Yes | 4.68 | 3.98

Table 1: The disentanglement (Dis.), completeness (Com.), and informativeness
(Info.) criteria reflected in six metrics. The Ex.1 and Ex.2 columns are
corresponding metrics’ scores (%) on two ideally disentangled representations.

The algorithmic details for computing the above metrics are provided in
Appendix A.

##### Empirical Difference.

To highlight the empirical difference between these metrics, we use a toy set
built by permuting four letters: A B C D. Each letter representing a
generative factor with 20 choices of assignments (i.e, $X=\\{X1,\ldots,X20\\}$
where $X\in\\{A,B,C,D\\}$). We consider two settings where each generative
factor is embedded in a single dimension (denoted by Ex.1), or two dimensions
(denoted by Ex.2). In each setting we uniformly sample 20 values from -1 to 1
to represent 20 assignments per factor and use them to allocate the
assignments into distinctive bins per each corresponding dimension. By
concatenating dimensions for each generative factor, we construct two ideal
disentangled representations for data points in this toy dataset, amounting to
4 and 8 dimensional representations, respectively. Using these representations
(skipping the encoding step), we measured the above metrics. Table 1 (Ex.1 and
Ex.2 columns) summarises the results, illustrating that out of the 6 metrics,
Higgins et al. (2017); Ridgeway and Mozer (2018); Kim and Mnih (2018) are the
only ones that reach the potential maximum (i.e., 100), while Chen et al.
(2018) exhibits its sensitivity towards _completeness_ when we allocate two
dimensions per factors.

##### Data Requirement.

Measuring the mentioned disentanglement metrics requires a dataset satisfying
the following attributes:

  1. 1.

A set $\mathbb{F}$ where each of its elements is a generative factor which
should be disentangled through representations;

  2. 2.

For each element $f_{i}\in\mathbb{F}$, a value space $\mathbb{V}_{i}$ which is
the domain of $f_{i}$;

  3. 3.

For each value $v_{ij}\in\mathbb{V}_{i}$, a sample space $\mathbb{S}_{ij}$
which contains observations who has value $v_{ij}$ on generative factor
$f_{i}$ while everything else is arbitrary.

We present two synthetic datasets (§3) that meet these criteria and use them
in our experiments (§4).

##  3 Generative Synthetic Datasets

The use of synthetic datasets is the common practice for evaluating
disentanglement in image domain (Dittadi et al., 2021; Higgins et al., 2017;
Kim and Mnih, 2018). Generative simplistic datasets in image domain define
independent generative factors (e.g. shape, color) behind the data generation.
However, a comparable resource is missing in text domain. We develop two
synthetic generative datasets with varying degrees of difficulty to analyse
and measure disentanglement: The YNOC dataset (§3.1) which has only three
structures and generative factors appearing in every sentence, and the POS
dataset (§3.2) which has more structures while some generative factors are not
guaranteed to appear in every sentence. The YNOC dataset offers a simpler
setting for disentanglement.

###  3.1 YNOC Dataset

Sentences in YNOC are generated by 4 generative factors: Year (Y), Name (N),
Occupation (O), and City (C), describing the occupation of a person. Since we
often use different means to express the same message, we considered three
templates to generate YNOC sentences:

Template I.

_in Y, N was a/an O in C._

Template II.

_in Y’s C, N was a/an O._

Template III.

_N was a/an O in C in Y._

The templates were then converted into real sentences using 10 years, 40
names, 20 occupations, and 30 cities. This amounted to a total of 720K
sentences, split as (60%,20%,20%) into training, validation, and test sets.

Simple Sentence Structures | # of Sentences
---|---
n. v. n. end-punc. | 200
n. v. adj. n. end-punc. | 1,000
n. adv. v. n. end-punc. | 1,000
n. adv. v. adj. n. end-punc. | 5,000
n. v. prep. n. end-punc. | 1,000
n. v. prep. adj. n. end-punc. | 5,000
n. adv. v. prep. n. end-punc. | 5,000
n. adv. v. prep. adj. n. end-punc. | 25,000
adj. n. v. n. end-punc. | 1,000
adj. n. v. adj. n. end-punc. | 4,000
adj. n. adv. v. n. end-punc. | 5,000
adj. n. adv. v. adj. n. end-punc. | 20,000
adj. n. v. prep. n. end-punc. | 5,000
adj. n. v. prep. adj. n. end-punc. | 20,000
adj. n. adv. v. prep. n. end-punc. | 25,000
adj. n. adv. v. prep. adj. n. end-punc. | 100,000
n. [dogs cats foxes horses tigers]
v. [want need have get require]
adv. [really recently gradually frequently eventually]
adj. [happy big small beautiful fantastic]
prep. [on in for to of]
conj1. [although because when where whereas]
conj2. [and or]
comma [,]
end-punc. [. !]
Table 2: Simple sentence structures and the vocabulary used for each POS tag
in our synthetic dataset.

###  3.2 POS Dataset

We use part-of-speech (POS) tags to simulate the structure of sentences and
define a base grammar as _“n. v. n. end-punc.”_ , where ‘n.’ denotes noun,
‘v.’ denotes verb and ‘end-punc.’ denotes the punctuation which appears at the
end of sentences. Then we define simple sentence structures as _“(adj.) n.
(adv.) v. (prep.) (adj.) n. end-punc.”_ , where ‘adj.’ denotes adjective,
‘adv.’ denotes adverb, ‘prep.’ denotes preposition, and ‘()’ marks the
arbitrary inclusion/removal of the corresponding POS tag. We populate the
structures with $2^{4}=16$ simple structures presented in Table 2.

Next, we define complex sentence structures as combinations of two simple
sentence structures by applying one of the following three rules:

Rule I.

_conj1. S1 comma S2 end-punc._

Rule II.

_S1 conj1. S2 end-punc._

Rule III.

_S1 comma conj2. S2 end-punc._

where ‘conj1.’ and ‘conj2.’ denote two different kinds of conjunction, ‘comma’
denotes ‘,’ and ‘S1’ and ‘S2’ are two simple sentence structures without ‘end-
punc.’ We limit the number of POS tags that appear in ‘S1’ and ‘S2’ to 9 to
control the complexity of generating sentences and obtain 279 complex
structures in total. A maximum of 5 words is chosen for each POS to construct
our sentences.

The frequency of appearance for each word in a sentence is limited to one.
Although this construction does not focus on sentences being “realistic”, it
simulate natural text in terms of the presence of an underlying grammar and
rules over POS tags.333For structures which can produce more than 10k
sentences (e.g. longer structures), we randomly choose 10k. We deliberately
ignore semantics, since isolating semantics in terms of generative factors
potentially involves analysis over multiple dimensions (combinatorial space)
and quantifying grouped disentanglement requires suitable disentanglement
metrics to be developed. We leave further exploration of this to our future
work.

We split the dataset into training, validation and test sets with proportion
60%, 20%, 20%. This proportion is used for every structure to ensure they have
representative sentences in each portion of the data splits. The final size of
(training, validation, test) sets are (1723680, 574560, 574560). All three
sets are unbiased on word selection for each POS tag: e.g., all 5 noun POS
vocabs from Table 2 have equal frequency (i.e., 20%). Exactly the same
proportions are preserved for validation and test sets.

Through the process of the generation, we can define each POS tag as one
ground truth generative factor for sentences.444While we consider POS tags as
the generative factors in this paper, further sub-categorisation of POS tags
based on position (e.g., first-noun and second-noun, etc) or grammatical roles
(e.g., subject-noun and object-noun, etc) is a possibility for future
investigation. Because the choices of words for different POS tags are
independent, these generative factors are independent. However, for the same
POS, the choices of words are dependent and POS tags are dependent on the
structures as well. It is noteworthy that in contrast to the image domain
where all generative factors are always present in the data, in POS dataset
this cannot be guaranteed, making it a more challenging setting.

##  4 Experiments and Analysis

| YNOC |  | POS
---|---|---|---
Model | KL | Rec.$\downarrow$ | AU$\uparrow$ | Top-3$\uparrow$ |  | KL | Rec.$\downarrow$ | AU$\uparrow$ | Top-3$\uparrow$
AE | - | $8.87_{\pm 0.66}$ | $4.0_{\pm 0.0}$ | 1 |  | - | $4.91_{\pm 1.83}$ | $8.0_{\pm 0.0}$ | 3
Vanilla-VAE | $0.02_{\pm 0.02}$ | $13.48_{\pm 0.02}$ | $0.4_{\pm 0.5}$ | 0 |  | $0.01_{\pm 0.00}$ | $19.57_{\pm 0.00}$ | $0.2_{\pm 0.4}$ | 3
$\beta$-VAE ($\beta=0.2$) | $4.25_{\pm 0.31}$ | $9.72_{\pm 0.25}$ | $1.0_{\pm 0.0}$ | 3 |  | $11.19_{\pm 2.88}$ | $12.03_{\pm 2.04}$ | $2.8_{\pm 0.7}$ | 3
$\beta$-VAE ($\beta=0.4$) | $3.44_{\pm 0.23}$ | $10.32_{\pm 0.23}$ | $1.2_{\pm 0.4}$ | 1 |  | $7.75_{\pm 0.69}$ | $13.87_{\pm 0.85}$ | $2.6_{\pm 0.5}$ | 3
$\beta$-VAE ($\beta=0.8$) | $1.39_{\pm 0.41}$ | $12.14_{\pm 0.40}$ | $1.0_{\pm 0.0}$ | 1 |  | $5.61_{\pm 0.78}$ | $14.26_{\pm 0.72}$ | $1.8_{\pm 0.4}$ | 1
CCI-VAE ($C=5$) | $5.00_{\pm 0.00}$ | $9.51_{\pm 0.30}$ | $1.8_{\pm 1.0}$ | 1 |  | $5.04_{\pm 0.03}$ | $15.01_{\pm 0.30}$ | $2.2_{\pm 0.4}$ | 0
CCI-VAE ($C=10$) | $10.00_{\pm 0.00}$ | $9.48_{\pm 0.49}$ | $3.4_{\pm 0.5}$ | 2 |  | $10.01_{\pm 0.01}$ | $12.76_{\pm 1.18}$ | $4.0_{\pm 1.3}$ | 1
MAT-VAE ($\beta=0.1,\lambda=0.1$) | $6.11_{\pm 0.39}$ | $9.49_{\pm 0.17}$ | $1.0_{\pm 0.0}$ | 2 |  | $22.14_{\pm 2.92}$ | $8.47_{\pm 2.28}$ | $3.0_{\pm 0.0}$ | 3
MAT-VAE ($\beta=0.01,\lambda=0.1$) | $15.38_{\pm 1.86}$ | $7.12_{\pm 0.32}$ | $3.2_{\pm 0.7}$ | 7 |  | $45.48_{\pm 1.65}$ | $3.47_{\pm 0.99}$ | $8.0_{\pm 0.0}$ | 1

Table 3: Results are calculated on the test set. We report mean value and
standard deviation across 5 runs.

Figure 1: Disentanglement scores across six metrics on top: YNOC dataset and
bottom: POS dataset. For better illustration, we multiply the scores of
Eastwood and Williams (2018) and Kumar et al. (2018) by 10.

| AE | VAE |  $\beta$-VAE | CCI-VAE | MAT-VAE
---|---|---|---|---|---
|  |  | $\beta=0.2$ | $\beta=0.4$ | $\beta=0.8$ | $C=5$ | $C=10$ | $\beta=0.1,\lambda=0.1$ | $\beta=0.01,\lambda=0.1$
YNOC | $0.22_{\pm 0.03}$ | $0.03_{\pm 0.02}$ | $0.30_{\pm 0.03}$ | $0.30_{\pm 0.02}$ | $0.30_{\pm 0.05}$ | $0.32_{\pm 0.04}$ | $0.30_{\pm 0.01}$ | $0.36_{\pm 0.03}$ | $0.43_{\pm 0.09}$
POS | $0.30_{\pm 0.05}$ | $0.21_{\pm 0.03}$ | $0.25_{\pm 0.00}$ | $0.27_{\pm 0.01}$ | $0.29_{\pm 0.04}$ | $0.29_{\pm 0.05}$ | $0.28_{\pm 0.01}$ | $0.29_{\pm 0.00}$ | $0.28_{\pm 0.01}$

Table 4: Hoyer scores are calculated on the test set. We report mean value and
standard deviation across 5 runs.

In this section, we examine the introduced disentanglement models on text. We
measure the disentanglement scores of each model on our two synthetic datasets
and quantify how well-correlated these metrics are with reconstruction loss,
active units, and KL (§4.1). We then look at various strategies for coupling
the latent code during decoding and highlight their impacts on training and
disentanglement behaviors (§4.2). We continue our analysis by showing how the
representation learned by the highest scoring model (on disentanglement
metrics) performs compared to vanilla VAE in two text classification tasks
(§4.3), and finish our analysis by looking at these models’ generative
behaviors (§4.4).

##### Training Configuration.

We adopt the VAE architecture from Bowman et al. (2016), using a LSTM encoder-
decoder. Unless stated otherwise, (word embedding, LSTM, representation
embedding) dimensionalities for YNOC and POS datasets are (4D, 32D, 4D) and
(4D, 64D, 8D), respectively, and we use the latent code to initialize the
hidden state of the LSTM decoder. We use greedy decoding. All models are
trained from multiple random starts using Adam (Kingma and Ba, 2015) with
learning rate 0.001 for 10 epochs. We set batch size to 256 and 512 for YNOC
and POS, respectively.

###  4.1 Disentanglement Metrics

Taking the models (§2.1) and also an Autoencoder (AE) as a baseline we use the
YNOC and POS datasets to report average KL-divergence (KL), reconstruction
loss (Rec.), and number of active units (AU)555$i$ is active if ${\rm
Covariance}_{\mathbf{x}}(\mathbb{E}_{i\sim
q\left(i|\mathbf{x}\right)}\left[i\right])>0.01$. in Table 3, and illustrate
disentanglement metrics’ scores in Figure 1.

As demonstrated in Table 3, different models pose various behaviors,
noteworthy of those are: (1) the positive correlation of $C$ with AU which
intuitively means the increase of channel capacity demands more dimensions of
the representation to carry information which then translates into having a
better reconstruction of data, (2) the negative correlation between the
increase of $\beta$ and decrease of reconstruction loss, (3) the best Rec. and
AU are achieved by AE and MAT-VAE whereas the worst one is achieved by the
(collapsed) vanilla-VAE, (4) the MAT-VAE ($\beta=0.01,\lambda=0.1$) model
which induces more sparse representations 666Sparsity is measured using Hoyer
Hurley and Rickard (2009). In this paper we report this as the average Hoyer
over data points’ posterior means. Hoyer for data point $x_{i}$ with posterior
mean $\mu_{i}$ is calculated as
$\frac{\sqrt{d}-||\bar{\mu}_{i}||_{1}/||\bar{\mu}_{i}||_{2}}{\sqrt{d}-1},$
where $d$ is the dimensionality of the representations and
$\bar{\mu}_{i}=\mu_{i}/\sigma(\mu)$, where $\mu=\\{\mu_{1},...,\mu_{n}\\}$,
and $\sigma(.)$ is the standard deviation. performs the best on both datasets,
indicating the positive impact of representation sparsity as an inductive
bias.

As illustrated in Figure 1, the difference between means of each
disentanglement score on various models is relatively small, and due to large
standard deviation on metrics, it is difficult to single out a superior model.
This verifies findings of Locatello et al. (2019) on image domain. In Table 3
(Top-3 column) we report the number of appearances of a model among the top 3
highest scoring models on at least one disentanglement metric. The ranking
suggests that $\beta$-VAE with smaller $\beta$ values reach better
disentangled representations, and MAT-VAE performing superior on YNOC and
poorly on POS, highlighting its more challenging nature. For MAT-VAE we also
observe an interesting correlation between sparsity and disentanglement: for
instance on YNOC, MAT-VAE ($\beta=0.01,\lambda=0.1$) achieves the highest
Hoyer (See Table 4) and occurs 7 times among Top-3 (see Table 3).
Interestingly, the success of MAT-VAE does not translate to POS dataset, where
it underperforms AE. These two observations suggest that sparsity could be a
facilitator for disentanglement, but achieving a stable level of sparsity
remains as a challenge. The more recent development in the direction of
sparsity, HSVAE Prokhorov et al. (2020), addresses the stability issue of MAT-
VAE but we leave its exploration to future work.

Figure 2: Correlation coefficients between six disentanglement metrics, Hoyer,
AU, Rec, and KL on Upper Triangle: YNOC dataset and Lower Triangle: POS
dataset.

(a) Different coupling strategies for the latent code and decoder (§4.2). Gray
box denotes decoder.

(b) Absolute differences between disentanglement metrics’ scores of Init.
coupling and others (§4.2).

Figure 3: Different coupling strategies for the latent code and decoder and
their impacts on disentanglement on POS and YNOC.

|  | Coupling Methods
---|---|---
|  | Init. | Concat. | Init.Concat. | Concat. w/o Emb.
YNOC | KL | $1.51{\pm 0.01}$ | $1.52{\pm 0.01}$ | $1.52{\pm 0.01}$ | $1.62{\pm 0.04}$
Rec.$\downarrow$ | $12.04{\pm 0.04}$ | $12.06{\pm 0.03}$ | $12.01{\pm 0.02}$ | $12.29{\pm 0.16}$
AU$\uparrow$ | $1.2{\pm 0.4}$ | $2.0{\pm 0.0}$ | $1.0{\pm 0.0}$ | $1.2{\pm 0.4}$
POS | KL | $5.54{\pm 0.02}$ | $5.53{\pm 0.02}$ | $5.51{\pm 0.00}$ | $5.69{\pm 0.03}$
Rec.$\downarrow$ | $14.54{\pm 0.33}$ | $15.89{\pm 0.26}$ | $15.98{\pm 0.05}$ | $16.48{\pm 0.09}$
AU$\uparrow$ | $2.2{\pm 0.4}$ | $4.0{\pm 0.0}$ | $3.2{\pm 0.4}$ | $3.6{\pm 0.5}$

Table 5: Test set KL, Reconstruction loss, Active Units using 4 coupling
methods (§4.2).

To further analyse the inconsistency between different metrics we calculate
the Pearson product-moment correlation coefficient between them and KL, -Rec,
AU, Hoyer on POS and YNOC datasets. See the heatmap in Figure 2. While text-
specific metrics are yet to be developed, our experiment suggests Higgins et
al. (2017) is a good candidate to try first for text domain as it seems to be
the one with strong correlation with Hoyer, AU, -Rec, and KL and has the
highest level of agreement (overall) with other metrics.

###  4.2 Coupling Latent Code and Decoder

In VAEs, we typically feed the decoder with the latent code as well as word
embeddings during training. The method to couple the latent code with decoder
could have some effects on disentanglement for text. To highlight this, we
train with 4 different coupling strategies: _Init_ , _Concat_ , _Init Concat_
, _Concat w/o Emb_. See Figure 3(a) for an accessible visualisation. To
analyse the impact of coupling, we opt for CCI-VAE which allows the
comparisons to be made for the same value of KL.

We first use _Concat w/o Emb_ to find an optimal KL in vanilla VAEs, which is
then used as the $C$ to train CCI-VAEs using the other coupling metrics on
YNOC and POS datasets. For YNOC, $C=1.5$, and for POS, $C=5.5$. This is to
keep KL-divergence and reconstruction loss at the same level for fair
comparison across different strategies. We report results in Table 5. Among
the investigated coupling methods, the key distinguishing factor for
disentanglement is their impacts on AU which is the highest for _Concat_.

Next, using _Init_ as the baseline, we measure the absolute difference between
disentanglement scores of different coupling methods in Figure 3(b). In
general, using concatenation can bring a large improvement in disentanglement.
Using both initialization and concatenation do not lead to a better result.
Despite our expectation, not feeding word embeddings into decoder during
training does not encourage disentanglement due to the added reliance on the
latent code.

A confounding factor which could pollute this analysis is the role of strong
auto-regressive decoding of VAEs and the type of information captured by the
decoder in such scenario. While a preliminary analysis has been provided
recently Bosc and Vincent (2020), this has been vastly under-explored and
requires more explicit attempts. We leave deeper investigation of this to
future work.

Figure 4: Classification accuracy on DBpedia and Yahoo Question using
different VAE models. Results are reported as mean and std across 3 randomly
initialised runs.

###  4.3 Disentanglement and Classification

To examine the performance of these models on real-world downstream task
setting, we consider the classification task. For our classification datasets,
we use DBpedia (14 classes) and Yahoo Question (10 classes) (Zhang et al.,
2015). Each class of these two datasets has (10k, 1k, 1k) randomly chosen
sentences in (train, dev, test) sets. We train Vanilla-VAE, $\beta$-VAE
($\beta=0.2$), CCI-VAE ($C=10$), and MAT-VAE ($\beta=0.01,\lambda=0.1$) from
Table 3 on DBpedia and Yahoo (without the labels), then freeze the trained
encoders and place a classifier on top to use the mean vector representations
from the encoder as a feature to train a classifier.

We set the dimensionality of word embedding, LSTM, and the latent space to
128, 512, 32, respectively. The VAE models are trained using a batch size of
64, for 6 epochs with Adam (learning rate 0.001). For the classifier, we use a
single linear layer with 1024 neurons, followed by a Softmax and train it for
15 epochs, using Adam (learning rate 0.001) and batch size 512. We illustrate
the mean and standard deviation across 3 runs of models in Figure 4.

We observe that the ranking of classification accuracy among the models on
DBpedia is consistent with their Top-3 performance in Table 3, with MAT-VAE
outperforming the other three variants. We see roughly the same trend for
Yahoo, with MAT-VAE being the dominating model. This indicates that
disentangled representations are likely to be easier to discriminate, although
the role of sparsely learned representations could contribute to MAT-VAE’s
success as well Prokhorov et al. (2020).

###  4.4 Disentanglement and Generation

Start | $z_{1}$ | $[\mathbf{z_{1,1},z_{1,2},z_{1,3}}]$ |  |
---|---|---|---|---
$i=1$ | $z^{\prime}_{1,1}$ | $z_{1,1}$, $z_{1,2},z_{1,3}$ | $\rightarrow$ | $z_{2,1}$, $z_{1,2},z_{1,3}$
$i=2$ | $z^{\prime}_{1,2}$ | $z_{2,1}$, $z_{1,2}$,$z_{1,3}$ | $\rightarrow$ | $z_{2,1}$, $z_{2,2}$$,z_{1,3}$
$i=3$ | $z^{\prime}_{1,3}$ | $z_{2,1},z_{2,2}$, $z_{1,3}$ | $\rightarrow$ | $z_{2,1},z_{2,2}$, $z_{2,3}$
End | $z_{2}$ |  |  | $[\mathbf{z_{2,1},z_{2,2},z_{2,3}}]$

Table 6: An example of a 3D latent code transformation in the dimension-wise
homotopy. In row $i$, $\rightarrow$ denotes the start and end points of
interpolation, solid box denotes the two dimensions being interpolated, and
dashed box denotes the updated dimensions from $i-1$.

To observe the effect of disentanglement in homotopy Bowman et al. (2016), we
use the exactly same toy dataset introduced in §2.1 and assess the homotopy
behaviour of the highest scoring VAE vs. an ideal representation. To conduct
homotopy, we interpolate between two sampled sequences’ representations and
pass the intermediate representations to decoder to generate the output. We
use 4D word embedding, 16D LSTM, 4D latent space. We report the results for
the VAEs scoring the highest on disentanglement (w.r.t. Higgins et al. (2017)
denoted as VAE-Higg) and completeness (w.r.t. Chen et al. (2018) denoted as
VAE-Chen). The VAE-Higg and VAE-Chen are $\beta$-VAE with $\beta=0.4$ and MAT-
VAE with $\beta=0.01,\lambda=0.1$, respectively.

Additionally, to highlight the role of generative factor in generation, we
conduct a dimension-wise homotopy, transitioning from the first to the last
sentence by interpolating between the dimensions one-by-one. This is
implemented as follows: (i) using prior distribution777Instead of prior, we
sample two sentences from test set and use their representations. This is to
avoid the situation where samples are not in the well-estimated region of the
posterior. we sample two latent codes denoted by
$\textbf{z}_{1}=(z_{1,1},z_{1,2},\dots,z_{1,n})$,
$\textbf{z}_{2}=(z_{2,1},z_{2,2},\dots,z_{2,n})$; (ii) for $i$-th dimension,
using
$\textbf{z}_{1,i}^{{}^{\prime}}=(z_{2,1},\dots,z_{2,i-1},z_{1,i},\dots,z_{1,n})$
as the start, we interpolate along the $i$-th dimension towards
$\textbf{z}_{2,i}^{{}^{\prime}}=(z_{2,1},\dots,z_{2,i},z_{1,i+1},\dots,z_{1,n})$.
Table 6 illustrates this for a 3D latent code example.

##### Results:

Table 4.4 reports the outputs for standard homotopy (top block) and dimension-
wise homotopy. The results for standard homotopy demonstrate that the presence
of ideally disentangled representation translates into disentangled generation
in general. However, both VAE-Higg and VAE-Chen seem to mainly be producing
variations of the letter in the first position (letter A) during the
interpolation. The same observation holds in the dimension-wise experiments.
VAE-Chen also produces variations of the letter in the second position (letter
B) along with the variation of letter A, which suggests the lesser importance
of completeness for disentangled representations.

|  Ideal |  VAE-Higg |  VAE-Chen
---|---|---|---
$\mathbf{z_{1}}$ |  A9 B17 C13 D3 |  A12 B14 C14 D12 |  A9 B4 C10 D15
Homotopy |  A20 B17 C1 D3 |  A12 B14 C14 D12 |  A7 B4 C10 D15
A4 B17 C12 D6 |  A8 B14 C14 D12 |  A14 B4 C10 D15
A3 B1 C6 D6 |  A20 B14 C14 D12 |  A20 B19 C10 D15
A13 B1 C6 D20 |  A15 B14 C14 D12 |  A8 B19 C10 D15
$\mathbf{z_{2}}$ |  A15 B2 C8 D10 |  A4 B14 C14 D12 |  A12 B19 C10 D15
$\textbf{z}_{1}$ |  A9 B17 C13 D3 |  A12 B14 C14 D12 |  A9 B4 C10 D15
1-4  Dim 1 |  A20 B17 C13 D3 |  A12 B14 C14 D12 |  A7 B4 C10 D15
A4 B17 C13 D3 |  A8 B14 C14 D12 |  A4 B19 C10 D15
A3 B17 C13 D3 |  A20 B14 C14 D12 |  A8 B19 C10 D15
|  A13 B17 C13 D3 |  A18 B14 C14 D12 |  A12 B19 C10 D15
1-4 $\textbf{z}_{1,2}^{{}^{\prime}}$ |  A15 B17 C13 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
1-4  Dim 2 |  A15 B17 C13 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
A15 B17 C13 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
A15 B17 C13 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
|  A15 B1 C13 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
1-4 $\textbf{z}_{1,3}^{{}^{\prime}}$ |  A15 B2 C13 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
1-4  Dim 3 |  A15 B2 C1 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
A15 B2 C12 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
A15 B2 C6 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
|  A15 B2 C6 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
1-4 $\textbf{z}_{1,4}^{{}^{\prime}}$ |  A15 B2 C8 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
1-4  Dim 4 |  A15 B2 C8 D3 |  A4 B14 C14 D12 |  A12 B19 C10 D15
A15 B2 C8 D6 |  A4 B14 C14 D12 |  A12 B19 C10 D15
A15 B2 C8 D6 |  A4 B14 C14 D12 |  A12 B19 C10 D15
|  A15 B2 C8 D20 |  A4 B14 C14 D12 |  A12 B19 C10 D15
1-4 $\textbf{z}_{2}$ |  A15 B2 C8 D10 |  A4 B14 C14 D12 |  A12 B19 C10 D15

Table 7: The homotopy experiments, comparing an ideal generator and the best
disentangled VAEs according to Higgins et al. (2017) (VAE-Higg) and Chen et
al. (2018) (VAE-Chen).

This indicates that despite the relative superior performance of certain
models on the metrics and classification tasks, the amount of disentanglement
present in the representation is not sufficient enough to be reflected by the
generative behavior of these models. As a future work, we would look into the
role of auto-regressive decoding and teacher-forcing as confounding factors
that can potentially affect the disentanglement process.

##  5 Conclusion and Future Directions

We evaluated a set of recent _unsupervised_ disentanglement learning
frameworks widely used in image domain on two artificially created corpora
with known underlying generative factors. Our experiments highlight the
existing gaps in text domain, the daunting tasks state-of-the-art models from
image domain face on text, and the confounding elements that pose further
challenges towards representation disentanglement in text domain. Motivated by
our findings, in future, we will explore the role of inductive biases such as
representation sparsity in achieving representation disentanglement.
Additionally, we will look into alternative forms of decoding and training
which may compromise reconstruction quality but increase the reliance of
decoding on the representation, hence allowing for a more controlled analysis
and evaluation.

Our synthetic datasets and experimental framework provide a set of
quantitative and qualitative measures to facilitate and future research in
developing new models, datasets, and evaluation metrics specific for text.

## References

  * Alemi et al. (2018) Alexander Alemi, Ben Poole, Ian Fischer, Joshua Dillon, Rif A Saurous, and Kevin Murphy. 2018.  Fixing a broken elbo.  In _International Conference on Machine Learning_ , pages 159–168. PMLR.
  * Balasubramanian et al. (2021) Vikash Balasubramanian, Ivan Kobyzev, Hareesh Bahuleyan, Ilya Shapiro, and Olga Vechtomova. 2021.  Polarized-VAE: Proximity based disentangled representation learning for text generation.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 416–423, Online. Association for Computational Linguistics.
  * Bao et al. (2019) Yu Bao, Hao Zhou, Shujian Huang, Lei Li, Lili Mou, Olga Vechtomova, Xin-yu Dai, and Jiajun Chen. 2019.  Generating sentences from disentangled syntactic and semantic spaces.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 6008–6019, Florence, Italy. Association for Computational Linguistics.
  * Bengio et al. (2013) Y. Bengio, A. Courville, and P. Vincent. 2013.  Representation learning: A review and new perspectives.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 35(8):1798–1828.
  * Bengio (2013) Yoshua Bengio. 2013.  Deep learning of representations: Looking forward.  In _Statistical Language and Speech Processing - First International Conference, SLSP 2013, Tarragona, Spain, July 29-31, 2013. Proceedings_ , volume 7978 of _Lecture Notes in Computer Science_ , pages 1–37. Springer.
  * Bosc and Vincent (2020) Tom Bosc and Pascal Vincent. 2020.  Do sequence-to-sequence VAEs learn global features of sentences? In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4296–4318, Online. Association for Computational Linguistics.
  * Bowman et al. (2016) Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Józefowicz, and Samy Bengio. 2016.  Generating sentences from a continuous space.  In _Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016_ , pages 10–21, Berlin, Germany. ACL.
  * Burgess et al. (2018) Christopher P. Burgess, Irina Higgins, Arka Pal, Loïc Matthey, Nick Watters, Guillaume Desjardins, and Alexander Lerchner. 2018.  Understanding disentangling in $\beta$-vae.  _CoRR_ , abs/1804.03599.
  * Chen et al. (2019) Mingda Chen, Qingming Tang, Sam Wiseman, and Kevin Gimpel. 2019.  A multi-task approach for disentangling syntax and semantics in sentence representations.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2453–2464, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Chen et al. (2018) Ricky T. Q. Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. 2018.  Isolating sources of disentanglement in variational autoencoders.  In _Advances in Neural Information Processing Systems_ , volume 31, pages 2610–2620. Curran Associates, Inc.
  * Cheng et al. (2020) Pengyu Cheng, Martin Renqiang Min, Dinghan Shen, Christopher Malon, Yizhe Zhang, Yitong Li, and Lawrence Carin. 2020.  Improving disentangled text representation learning with information-theoretic guidance.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7530–7541, Online. Association for Computational Linguistics.
  * Conneau et al. (2018) Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni. 2018.  What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2126–2136, Melbourne, Australia. Association for Computational Linguistics.
  * Dieng et al. (2019) Adji B. Dieng, Yoon Kim, Alexander M. Rush, and David M. Blei. 2019.  Avoiding latent variable collapse with generative skip models.  In _The 22nd International Conference on Artificial Intelligence and Statistics, AISTATS 2019_ , volume 89 of _Proceedings of Machine Learning Research_ , pages 2397–2405, Naha, Okinawa, Japan. PMLR.
  * Dittadi et al. (2021) Andrea Dittadi, Frederik Träuble, Francesco Locatello, Manuel Wuthrich, Vaibhav Agrawal, Ole Winther, Stefan Bauer, and Bernhard Schölkopf. 2021.  On the transfer of disentangled representations in realistic settings.  In _International Conference on Learning Representations_.
  * Eastwood and Williams (2018) Cian Eastwood and Christopher K. I. Williams. 2018.  A framework for the quantitative evaluation of disentangled representations.  In _International Conference on Learning Representations_.
  * Ettinger et al. (2018) Allyson Ettinger, Ahmed Elgohary, Colin Phillips, and Philip Resnik. 2018.  Assessing composition in sentence vector representations.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 1790–1801, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * Gretton et al. (2012) Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. 2012.  A kernel two-sample test.  _Journal of Machine Learning Research_ , 13(25):723–773.
  * He et al. (2019) Junxian He, Daniel Spokoyny, Graham Neubig, and Taylor Berg-Kirkpatrick. 2019\.  Lagging inference networks and posterior collapse in variational autoencoders.  In _7th International Conference on Learning Representations, ICLR 2019_ , New Orleans, LA, USA.
  * Hewitt and Liang (2019) John Hewitt and Percy Liang. 2019.  Designing and interpreting probes with control tasks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 2733–2743, Hong Kong, China. Association for Computational Linguistics.
  * Higgins et al. (2018) Irina Higgins, David Amos, David Pfau, Sébastien Racanière, Loïc Matthey, Danilo J. Rezende, and Alexander Lerchner. 2018.  Towards a definition of disentangled representations.  _CoRR_ , abs/1812.02230.
  * Higgins et al. (2017) Irina Higgins, Loïc Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017.  beta-vae: Learning basic visual concepts with a constrained variational framework.  In _5th International Conference on Learning Representations, ICLR 2017, Conference Track Proceedings_ , Toulon, France.
  * Hurley and Rickard (2009) N. Hurley and S. Rickard. 2009.  Comparing measures of sparsity.  _IEEE Transactions on Information Theory_ , 55(10):4723–4741.
  * John et al. (2019) Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2019.  Disentangled representation learning for non-parallel text style transfer.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 424–434, Florence, Italy. Association for Computational Linguistics.
  * Kim and Mnih (2018) Hyunjik Kim and Andriy Mnih. 2018.  Disentangling by factorising.  In _Proceedings of the 35th International Conference on Machine Learning, ICML 2018_ , volume 80 of _Proceedings of Machine Learning Research_ , pages 2649–2658, Stockholmsmässan, Stockholm Sweden. PMLR.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, Conference Track Proceedings_ , San Diego, CA, USA.
  * Kingma and Welling (2014) Diederik P. Kingma and Max Welling. 2014.  Auto-encoding variational bayes.  In _2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings_.
  * Kumar et al. (2018) Abhishek Kumar, Prasanna Sattigeri, and Avinash Balakrishnan. 2018.  VARIATIONAL INFERENCE OF DISENTANGLED LATENT CONCEPTS FROM UNLABELED OBSERVATIONS.  In _International Conference on Learning Representations_.
  * Locatello et al. (2019) Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. 2019.  Challenging common assumptions in the unsupervised learning of disentangled representations.  In _Proceedings of the 36th International Conference on Machine Learning, ICML 2019_ , volume 97 of _Proceedings of Machine Learning Research_ , pages 4114–4124, Long Beach, California, USA. PMLR.
  * Marvin and Linzen (2018) Rebecca Marvin and Tal Linzen. 2018.  Targeted syntactic evaluation of language models.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1192–1202, Brussels, Belgium. Association for Computational Linguistics.
  * Mathieu et al. (2019) Emile Mathieu, Tom Rainforth, N Siddharth, and Yee Whye Teh. 2019.  Disentangling disentanglement in variational autoencoders.  In _Proceedings of the 36th International Conference on Machine Learning_ , volume 97 of _Proceedings of Machine Learning Research_ , pages 4402–4412. PMLR.
  * Mitchell and Beauchamp (1988) T. J. Mitchell and J. J. Beauchamp. 1988.  Bayesian variable selection in linear regression.  _Journal of the American Statistical Association_ , 83(404):1023–1032.
  * Montero et al. (2021) Milton Llera Montero, Casimir JH Ludwig, Rui Ponte Costa, Gaurav Malhotra, and Jeffrey Bowers. 2021.  The role of disentanglement in generalisation.  In _International Conference on Learning Representations_.
  * Pelsmaeker and Aziz (2020) Tom Pelsmaeker and Wilker Aziz. 2020.  Effective estimation of deep generative language models.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7220–7236, Online. Association for Computational Linguistics.
  * Pimentel et al. (2020) Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. 2020.  Information-theoretic probing for linguistic structure.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4609–4622, Online. Association for Computational Linguistics.
  * Prokhorov et al. (2020) Victor Prokhorov, Yingzhen Li, Ehsan Shareghi, and Nigel Collier. 2020.  Hierarchical sparse variational autoencoder for text encoding.  _arXiv preprint arXiv:2009.12421_.
  * Prokhorov et al. (2019) Victor Prokhorov, Ehsan Shareghi, Yingzhen Li, Mohammad Taher Pilehvar, and Nigel Collier. 2019.  On the importance of the Kullback-Leibler divergence term in variational autoencoders for text generation.  In _Proceedings of the 3rd Workshop on Neural Generation and Translation_ , pages 118–127, Hong Kong. Association for Computational Linguistics.
  * Razavi et al. (2019) Ali Razavi, Aäron van den Oord, Ben Poole, and Oriol Vinyals. 2019.  Preventing posterior collapse with delta-vaes.  In _7th International Conference on Learning Representations, ICLR 2019_ , New Orleans, LA, USA.
  * Ridgeway and Mozer (2018) Karl Ridgeway and Michael C Mozer. 2018.  Learning deep disentangled embeddings with the f-statistic loss.  In _Advances in Neural Information Processing Systems_ , volume 31, pages 185–194. Curran Associates, Inc.
  * Ross and Doshi-Velez (2021) Andrew Slavin Ross and Finale Doshi-Velez. 2021.  Benchmarks, algorithms, and metrics for hierarchical disentanglement.  _CoRR_ , abs/2102.05185.
  * Torroba Hennigen et al. (2020) Lucas Torroba Hennigen, Adina Williams, and Ryan Cotterell. 2020.  Intrinsic probing through dimension selection.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 197–216, Online. Association for Computational Linguistics.
  * Voita and Titov (2020) Elena Voita and Ivan Titov. 2020.  Information-theoretic probing with minimum description length.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 183–196, Online. Association for Computational Linguistics.
  * Yang et al. (2017) Zichao Yang, Zhiting Hu, Ruslan Salakhutdinov, and Taylor Berg-Kirkpatrick. 2017\.  Improved variational autoencoders for text modeling using dilated convolutions.  In _Proceedings of the 34th International Conference on Machine Learning, ICML 2017_ , volume 70 of _Proceedings of Machine Learning Research_ , pages 3881–3890, Sydney, NSW, Australia. PMLR.
  * Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  In _Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1_ , NIPS’15, page 649–657, Cambridge, MA, USA. MIT Press.

##  Appendix A Disentanglement Metrics Algorithms

To evaluate representations learned by a model on a dataset having the
attributes of Data Requirement, we further require a series of representation
space $\mathbb{R}_{ij}$, who has a bijection mapping with $\mathbb{S}_{ij}$.
Hence, when sampling representations which have the same value on one
generative factor, we only need to sample in one $\mathbb{R}_{ij}$.

Under these notations, we write the pseudo code of metrics in Algorithm 1-6.
For Algorithm 5 and 6, although we only use one criterion in the main paper,
we still provide the details for other criteria. We set $N=1000$ and $L=64$
for Algorithm 1 and 2, and $N=10000$ for Algorithm 3, 4, 5, and 6.

Algorithm 1 Metric of Higgins et al. (2017)

1:$\mathbb{D}=\varnothing$

2:for $f_{i}\in\mathbb{F}$ do

3: for $n=1,2,\ldots,N$ do

4: Sample $s_{n}$ from $\bigcup_{j}\mathbb{S}_{ij}$

5: Find the value $v_{ij}$ on $f_{i}$ for $s_{n}$

6: Sample $(\mathbf{z}_{1}^{(1)},\ldots,\mathbf{z}_{L}^{(1)})$ from
$\mathbb{R}_{ij}$

7: Sample $(\mathbf{z}_{1}^{(2)},\ldots,\mathbf{z}_{L}^{(2)})$ from
$\mathbb{R}_{ij}$

8:
$\mathbf{z}_{n}=\frac{1}{L}\sum_{l=1}^{L}|\mathbf{z}_{l}^{(1)}-\mathbf{z}_{l}^{(2)}|$

9: $\mathbb{D}=\\{(\mathbf{z}_{n},f_{i})\\}\bigcup\mathbb{D}$

10:Split $\mathbb{D}$ into training set $\mathbb{TR}$ and test set
$\mathbb{TE}$ with proportion (80%, 20%)

11:Train 10 MLPs with only input and output layer on $\mathbb{TR}$

12:Calculate the accuracy on $\mathbb{TE}$ for 10 models

13:Calculate the mean and variance of accuracy

Algorithm 2 Metric of Kim and Mnih (2018)

1:$\mathbb{D}=\varnothing$

2:for $d=1,2,\ldots,dim_{z}$ do

3: Calculate the standard deviation $\sigma_{d}$ of dimension $d$

4:for $f_{i}\in\mathbb{F}$ do

5: for $n=1,2,\ldots,N$ do

6: Sample $s_{n}$ from $\bigcup_{j}\mathbb{S}_{ij}$

7: Find the value $v_{ij}$ on $f_{i}$ for $s_{n}$

8: Sample $(\mathbf{z}_{1},\ldots,\mathbf{z}_{L})$ from $\mathbb{R}_{ij}$

9:
$d^{*}_{n}=\mathop{\arg\max}_{d}var(\frac{z_{1,d}}{\sigma_{d}},\ldots,\frac{z_{L,d}}{\sigma_{d}})$

10: $\mathbb{D}=\\{(d^{*}_{n},f_{i})\\}\bigcup\mathbb{D}$

11:Split $\mathbb{D}$ into training set $\mathbb{TR}$ and test set
$\mathbb{TE}$ with proportion (80%, 20%)

12:Train 10 majority vote classifiers on $\mathbb{TR}$

13:Calculate the accuracy on $\mathbb{TE}$ for 10 models

14:Calculate the mean and variance of accuracy

Algorithm 3 Metric of Kumar et al. (2018)

1:for $f_{i}\in\mathbb{F}$ do

2: for $v_{ij}\in\mathbb{V}_{i}$ do

3:
$p(v_{ij})=\frac{{Count}(\mathbb{S}_{ij})}{\sum_{j}{Count}(\mathbb{S}_{ij})}$

4: Sample $N_{j}=N\times p(v_{ij})$ representations $\mathbf{z}^{j}$ from
$\mathbb{R}_{ij}$

5: for $d=1,2,\ldots,dim_{z}$ do

6: $\mathbb{D}_{d}=\varnothing$

7: for $v_{ij}\in\mathbb{V}_{i}$ do

8: for $n=1,2,\ldots,N_{j}$ do

9: $\mathbb{D}_{id}=\\{(z_{n,d}^{j},v_{ij})\\}\bigcup\mathbb{D}_{id}$

10: Split $\mathbb{D}_{d}$ into training set $\mathbb{TR}_{d}$ and test set
$\mathbb{TE}_{d}$ with proportion (80%, 20%)

11: Train a linear SVM classifier on $\mathbb{TR}_{d}$

12: Record the accuracy ${acc}_{d}$ on $\mathbb{TE}_{id}$

13: $d^{*}=\mathop{\arg\max}_{d}{acc}_{d}$

14: ${SAP}_{i}={acc}_{d^{*}}-\max_{d\neq d^{*}}{acc}_{d}$

15:${score}={avg}({SAP}_{i})$

Algorithm 4 Metric of Chen et al. (2018)

1:for $d=1,2,\ldots,dim_{z}$ do

2: Divide values on dimension $d$ into 20 uniform bins $\mathbb{B}_{d}$

3: for $n=1,2,\ldots,20$ do

4:
$p(z_{d}\in\mathbb{B}_{d}^{n})=\frac{{Count}(\\{z_{d}\in\mathbb{B}_{d}^{n}\\})}{\sum_{n=1}^{20}{Count}(\\{z_{d}\in\mathbb{B}_{d}^{n}\\})}$

5: $H(z_{d})=-\sum_{n=1}^{20}p(z_{d}\in\mathbb{B}_{d}^{n})\log
p(z_{d}\in\mathbb{B}_{d}^{n})$

6:for $f_{i}\in\mathbb{F}$ do

7: for $v_{ij}\in\mathbb{V}_{i}$ do

8:
$p(v_{ij})=\frac{{Count}(\mathbb{S}_{ij})}{\sum_{j}{Count}(\mathbb{S}_{ij})}$

9: Sample $N_{j}=N\times p(v_{ij})$ representations $\mathbf{r}^{j}$ from
$\mathbb{R}_{ij}$

10: $H(f_{i})=-\sum_{j}p(v_{ij})\log p(v_{ij})$

11: for $d=1,2,\ldots,dim_{z}$ do

12: for $v_{ij}\in\mathbb{V}_{i}$ do

13: for $n=1,2,\ldots,20$ do

14:
$p(z_{d}\in\mathbb{B}_{d}^{n}|v_{ij})=\frac{{Count}(\\{r^{j}_{d}\in\mathbb{B}_{d}^{n}\\})}{\sum_{n=1}^{20}{Count}(\\{r^{j}_{d}\in\mathbb{B}_{d}^{n}\\})}$

15:
$H(z_{d}|f_{i})=-\sum_{j}p(v_{ij})\sum_{n=1}^{20}p(z_{d}\in\mathbb{B}_{d}^{n}|v_{ij})\log
p(z_{d}\in\mathbb{B}_{d}^{n}|v_{ij})$

16: $I(z_{d},f_{i})=H(z_{d})-H(z_{d}|f_{i})$

17: $d^{*}=\mathop{\arg\max}_{d}\frac{I(z_{d},f_{i})}{H(f_{i})}$

18: ${MIG}_{i}=\frac{I(z_{d^{*}},f_{i})}{H(f_{i})}-\max_{d\neq
d^{*}}\frac{I(z_{d},f_{i})}{H(f_{i})}$

19:${score}={avg}({MIG}_{i})$

Algorithm 5 Metric of Ridgeway and Mozer (2018)

Modularity:

1:Same steps as Algorithm 4 without step 17, 18 and 19

2:for $d=1,2,\ldots,dim_{z}$ do

3: $i^{*}=\mathop{\arg\max}_{i}I(z_{d},f_{i})$

4: $\theta_{d}=I(z_{d},f_{i^{*}})$

5: for $f_{i}\in\mathbb{F}$ do

6: if $i=i^{*}$ then

7: $t_{i}=\theta_{d}$

8: else

9: $t_{i}=0$

10:
$\delta_{d}=\frac{\sum_{i}(I(z_{d},f_{i})-t_{i})^{2}}{\theta_{d}^{2}({Count}(\mathbb{F})-1)}$

11:${score}={avg}(1-\delta_{d}$)

Explicitness:

1:for $f_{i}\in\mathbb{F}$ do

2: $\mathbb{D}_{i}=\varnothing$

3: for $v_{ij}\in\mathbb{V}_{i}$ do

4:
$p(v_{ij})=\frac{{Count}(\mathbb{S}_{ij})}{\sum_{j}{Count}(\mathbb{S}_{ij})}$

5: Sample $N_{j}=N\times p(v_{ij})$ representations $\mathbf{r}^{j}$ from
$\mathbb{R}_{ij}$

6: for $n=1,2,\ldots,N_{j}$ do

7: $\mathbb{D}_{i}=\\{(\mathbf{r}_{n}^{j},v_{ij})\\}\bigcup\mathbb{D}$

8: Split $\mathbb{D}_{i}$ into training set $\mathbb{TR}_{i}$ and test set
$\mathbb{TE}_{i}$ with proportion (80%, 20%)

9: Train an one-versus-rest logistic regress classifier on $\mathbb{TR}_{i}$

10: Record the ROC area-under-the-curve (AUC) ${auc}_{ij}$ on
$\mathbb{TR}_{i}$ for every $v_{ij}$

11:${score}={avg}({auc}_{ij})$

Algorithm 6 Metric of Eastwood and Williams (2018)

1:for $f_{i}\in\mathbb{F}$ do

2: $\mathbb{D}_{i}=\varnothing$

3: for $v_{ij}\in\mathbb{V}_{i}$ do

4:
$p(v_{ij})=\frac{{Count}(\mathbb{S}_{ij})}{\sum_{j}{Count}(\mathbb{S}_{ij})}$

5: Sample $N_{j}=N\times p(v_{ij})$ representations $\mathbf{z}^{j}$ from
$\mathbb{R}_{ij}$

6: for $n=1,2,\ldots,N_{j}$ do

7: $\mathbb{D}_{i}=\\{(\mathbf{z}_{n}^{j},v_{ij})\\}\bigcup\mathbb{D}$

8: Split $\mathbb{D}_{i}$ into training set $\mathbb{TR}_{i}$ and test set
$\mathbb{TE}_{i}$ with proportion (80%, 20%)

9: Train a random forest classifier on $\mathbb{TR}_{i}$

10: Informativeness score ${inf}_{i}$ is the accuracy on $\mathbb{TE}_{i}$

11: $r_{id}$ is the relative importance of dimension $d$ in predicting
$v_{ij}$, obtained from the random forest

12: for $d=1,2,\ldots,dim_{z}$ do

13: $P_{d}=\frac{r_{id}}{\sum_{d}r_{id}}$

14: $H=-\sum_{d}P_{d}\log_{dim_{z}}P_{d}$

15: $dis_{i}=1-H$

16:${score}_{disentanglement}={avg}({dis}_{i})$

17:${score}_{informativeness}={avg}({inf}_{i})$

18:for $d=1,2,\ldots,dim_{z}$ do

19: for $f_{i}\in\mathbb{F}$ do

20: $Q_{i}=\frac{r_{id}}{\sum_{i}r_{id}}$

21: $H=-\sum_{i}Q_{i}\log_{{Count}(\mathbb{F})}Q_{i}$

22: Completeness score $com_{d}=1-H$

23:${score}_{completeness}={avg}({com}_{d})$