# Quantum computing for data science

Barry C. Sanders  Institute for Quantum Science and Technology, University of
Calgary, Alberta T2N 1N4, Canada<EMAIL_ADDRESS>
###### Abstract

I provide a perspective on the development of quantum computing for data
science, including a dive into state-of-the-art for both hardware and
algorithms and the potential for quantum machine learning.

##  1 Introduction

Quantum information theory transforms the very foundations of information
theory and computing [1, 2]. The pre-quantum (known as ‘classical’) scientific
framework allows objective information to be labelled by integers such as bit
strings (e.g., representing text characters by 7-bit strings according to the
American Standard Code for Information Interchange, or ASCII), which is
foundational to information theory. Processing information can be executed
under the rules of Boolean logic, manifested as concatenations of one-bit
operations such as NOT and two-bit operations such as NAND.

Quantum information changes the information game entirely by allowing coherent
superpositions of informational states, which, following the quantum principle
of complementarity, can be thought of as being both corpuscular (particle-
like) and undular (wave-like). For example, a three-bit string 010 becomes
quantumly a label for a quantum state $\ket{010}$ (element of Hilbert space),
which could be manifested physically, say, by three electrons that are spin-
down, spin-up and spin-down, with a spin-down state labelled $\ket{0}$ and a
spin-up state labelled $\ket{1}$ in Dirac, or bra-ket, notation [3].
Superposing this three-electron state with its orthogonal complement would be
$\ket{101}$. For state normalisation implicit throughout this paper, the
superposition of these two states is $\ket{010}+\ket{101}$, which, in binary
representation, is a superposition of the numbers 2 and 5.

These superpositions of information states can be processed quantumly, i.e.,
in a way that preserves coherence. Ideally, this superposition state can be
transformed by an arbitrary unitary map (isometry on Hilbert space).
Realistically, open-system effects such as noise and loss can impede
performance, but almost-unitary maps (such as completely positive trace-
preserving maps [1] that are close to being unitary) can suffice for useful
quantum-information processing provided that the quantum version of error
correction is employed in a fault-tolerant way [4].

An early motivation for quantum computing was for simulating physics,
particularly for simulating quantum systems in a way that is natural for
quantum descriptions [5], i.e., by using quantum computing. Since that
original idea, remarkable quantum algorithms have arisen, where remarkable
refers to delivering superior performance compared to classical algorithms,
such as efficient computation meaning computational resources such as run-time
and number of bits or qubits, being quantum bits, scale no worse than
polynomially with respect to the size of the problem quantified by how many
bits are required for specifying the problem instance [1].

One example of quantum algorithms being advantageous is for efficiently
solving number factorization, whose hardness is subexponential (almost
exponential) using the best known classical algorithm [6]. Another example is
a provably quadratically faster algorithm for function inversion (hence,
unstructured search) [7]. Quantum annealing is speculated to enhance
optimisation methods in some cases [8]. Building quantum computers is rapidly
advancing, albeit small-scale and noisy, making quantum computing now viable
(in a limited way) and commercial [9, 10, 11].

Quantum computing is arguably a disruptive technology, i.e., a technology that
is both innovative and market-changing [12], if quantum computing eventually
delivers a game-changing application such as for data science [12, 13]. Some
big ‘ifs’ stand in the way of quantum computing being disruptive, though. My
view is that quantum computing faces two huge ‘ifs’, i.e., challenges: (1)
building a quantum computer and (2) making said quantum computer useful.

In a way, the latter is harder than the former. Scaling a quantum computer to
large size (many qubits) and large depth (able to pass through many cycles of
quantum logic gates, defined as unitary or near-unitary maps of quantum
information states) is technically extremely challenging but conceivable.
Figuring out, on the other hand, what to do with a quantum computer that is
truly useful and transformative is, at best, challenging because of a limit to
our imagination of what could be the next great quantum algorithm and, at
worst, hopeless if no new great quantum algorithm exists. Thus, quantum
computing is a wonderful, scary, risky adventure, which we need to navigate
with our eyes wide open.

##  2 Nexus between quantum computing and data science

Now I elaborate on the nexus between quantum computing and data science, as
any meaningful overlap between these two galvanising research fields is
exciting. Data science [14] concerns the full pipeline from capturing and
maintaining data to processing and analysing these data and, finally,
communicating or executing data-driven action [15]. Computing is a key part of
this pipeline, not only for exploiting quantum speedup [16] or superior
accuracy [17] but also for the possibility of storing information in
superposition in a quantum random access memory [18] as well as considering
cyberattacks on data storage and processing.

Secure data storage could involve quantum cryptography [19], and quantum-
secure use of servers could need to exploit methods such as blind quantum
computing [20] or quantum homomorphic encryption [21, 22]. The full impact of
quantum information processing on data science is not yet known and needs
extensive study. Here, we focus here strictly on the impact of quantum
computing per se on data science.

I mentioned a little about quantum computing being about superpositions of
information states and processing by unitary or near-unitary maps. As quantum
computing is rather subtle in its nature, let us now take a moment to
appreciate how it works. I would like to illustrate how quantum computing
works by considering a quantum simulation, executable on a quantum computer,
of Schrödinger’s famous example of a cat whose existential state can be in a
life-death superposition, so to speak [23]. Our purpose is not to philosophise
here but rather to recast Schrödinger’s cat paradox [24] as quantum
information processing thereby illustrating how quantum computing works.

Here, I cast the cat’s quantum state of being alive ($b=0$) or dead ($b=1$) as
being represented by a quantum state $\ket{b}$. Similarly, we can consider a
radioactive nucleus, that decays by emitting an alpha ($\alpha$) particle.
$\alpha$ decay can be regarded as the emission of an $\alpha$ particle,
equivalent to a He2+ ion, by quantum tunnelling out of the remaining nucleus
[25]. The undecayed nucleus can be regarded as being an electromagnetic
potential trap for the $\alpha$ particle, with the nuclear interior being
attractive and the nuclear boundary being repulsive. We represent the
undecayed nucleus by $\ket{0}$ and the decayed nucleus by $\ket{1}$. Left
alone the initial nuclear state $\ket{0}$ evolves into a superposition such as
$\ket{\pm}:=\ket{0}\pm\ket{1}$ over one half-life with choice of $\pm$
indicating the coherence phase between not having decayed and having decayed.

To make clear the connection with quantum information processing,
Schrödinger’s cat paradox is depicted in Fig. 2 as a cat in a box with poison
gas triggered by an $\alpha$-particle detection. The correlation between
$\alpha$ decay and the cat dying is shown in Fig. 2, with the resultant
entangled nuclear-cat state shown in Fig. 2 as well. We can write this
entangled state as $\ket{00}+\ket{11}$. This state is extremely important in
quantum information science and is often called a Bell state. Just as we have
resources like qubits and gates, this resource is sometimes called an ebit,
short for a pair of maximally entangled qubits.

Figure 1: A cat is placed in an opaque box along with a poison gas that is
released contingent on detecting the decay of an $\alpha$ particle. The decay
is modelled as the escape of an $\alpha$ particle, equivalently a He2+ ion,
from the nucleus, with this decay occurring by quantum tunnelling.

Figure 2: Above: the $\alpha$ particle is deteced, which causes the release of
a poison gas that kills the cat; in Dirac notation, the nuclear-cat entangled
state is a superposition of the nucleus being undecayed and the cat being
alive with the orthogonal state of the nucleus having decayed and the cat
having died.

We can think of Schrödinger’s cat paradox in terms of a quantum circuit. The
input is initialised to $\ket{00}$ for an undecayed nucleus and a living cat.
Then one waits for precisely one half-life to achieve $\ket{0}\mapsto\ket{+}$.
If decay were a unitary process for one degree of freedom (not actually the
case for nuclear decay but didactically convenient) then decay could be
described by a quantum logic gate known as the Hadamard gate (H), whose
unitary implies H$:\ket{1}\mapsto\ket{-}$ as well, which would mean that a
decayed nucleus implausibly returns to its undecayed state.

The killing of the cat depicted in Fig. 2 is effecting the quantum logic gate
known as CNOT, which stands for the controlled-not operation. The CNOT gate is
equivalent to the XOR gate: flip the second qubit, i.e., apply the quantum NOT
gate, if the first qubit is $\ket{1}$ but not if the first qubit is $\ket{0}$.
Mathematically, for any two qubits in the computational basis, written as
$\ket{b\;b^{\prime}}$ for any bits $b$ and $b^{\prime}$, the mapping is ${\rm
CNOT}:\ket{b\;b^{\prime}}\mapsto\ket{b\;b^{\prime}\oplus b}$. The CNOT gate
can be extended in an obvious way, as a NOT gate controlled by more than one
qubit, such as the controlled-controlled-NOT, or Toffoli, gate ${\rm
CCNOT}:\ket{b\;b^{\prime}\;b^{\prime\prime}}\mapsto\ket{b\;b^{\prime}\;b^{\prime}b^{\prime\prime}\oplus
b}$, which is valuable as a quantum version of repetition codes [26].

For Schrödinger’s cat paradox, waiting a half-life means the first qubit is a
superposition of $\ket{0}$ and $\ket{1}$, not specifically one of these two
states, and the output is then the Bell state. Bell-state preparation,
described by the procedure to manufacture the Schrödinger cat paradox, is a
workhorse of some quantum computers, such as for ion-trap quantum computing
[27].

The Hadamard gate plays a key role in quantum computing. An outer product of
Hadamard gates maps a qubit string by $\ket{00\cdots
0}\mapsto\ket{++\cdots+}$, which is a superposition of all strings of qubits
of given length. Some quantum algorithms commence this way, which is at the
heart of some references to quantum computing as parallelisation, but
parallelisation is only part of the quantum computing story [2].

##  3 Building a quantum computer

Of course we would not be performing quantum computing using cats (although
this terminology is used popularly for quantum computing [23], and the term
“two-component Schrödinger cat states” is used for some type of
superconducting quantum computing [28] although I prefer the term “entangled
coherent state” for such states [29]) and instead resort to realisable quantum
computers. I now list the current favourite platforms with examples of how the
qubits are manifested [9, 10], and I sidestep the growing and intriguing areas
of so-called continuous variable quantum computing [30, 31] and qudit quantum
computing [32].

Photonic quantum computing was an early candidate and remains a strong
contender. A single-rail qubit could be manifested as a superposition of no
photon and one photon in a particular spatiotemporal mode; in contrast a dual-
rail qubit is a single photon with the logical qubit inherent in the
polarisation or which-path or which-time ($\ket{{\rm early}}$ and $\ket{{\rm
late}}$, say) state of the photon. In magnetic resonance, the qubit can be the
nuclear spin state, which can be controlled through hyperfine interactions
with electronic qubits in the atom. Trapped ions can be superpositions of
atomic excited and ground states, and controlled quantum logic gates can be
achieved by exploiting collective motion of the ions leading to phononic
sidebands of the atomic energy states. Neutral atomic qubits are also
superpositions of ground and excited electronic states, with entangling two-
qubit gates achievable by controlled collisions between atoms. Solid-state
systems could be semiconductors, with the qubit inherent in the spin of a
trapped electron, and superconducting quantum computing can involve quantized
flux or superpositions of charge or using microwave photons in the resonator.
A hybrid semiconductor-superconductor system could lead to topologically
protected qubits.

Quantum computers can be realised according to different approach to
processing quantum information. The gate ‘model’ is about making a universal
primitive set of gates, such as H and CNOT discussed earlier, as well as
another gate called T, which maps $\ket{0}\mapsto\ket{0}$ and
$\ket{1}\mapsto{\rm e}^{{\rm i}\nicefrac{{\pi}}{{4}}}\ket{1}$. Any multi-qubit
unitary map can be decomposed efficiently into an efficiently scaling number
of computer cycles, with each cycle comprising these single-qubit gates,
including the identity gate, plus CNOT gates.

Measurement-based quantum computing, on the other hand, begins by entangling
all qubits together and then carving out a circuit by measuring unwanted
qubits in an appropriate basis. The remaining qubits are sequentially
measured, with measurement basis determined by previous measurement outcomes;
the final measurement outcomes provide the desired solution. Adiabatic quantum
computing is about slow, controlled, continuous-time evolution of a
Hamiltonian to realise approximately the target unitary map. Topological
quantum computing involves braiding anyonic worldlines and could be realised
via the fractional quantum Hall effect. This list describes universal quantum
computers; purpose-built quantum computers such as quantum annealing—adiabatic
or diabatic [33]—and boson sampling [34] enrich the set of quantum computers
in development.

Quantum computer technology is quite impressive today, with quantum annealing
reaching thousands of qubits and gate-based quantum computing having over a
hundred qubits. Quantum computing primacy has arguably arrived [35, 36].
Certainly, quantum computing technology has surpassed what I imagined would be
the status today, but much more is needed to make quantum computers genuinely
useful.

##  4 Applied quantum algorithms for data science

A plethora of quantum algorithms has been developed [37]. Although
factorisation is the most stunning of the quantum algorithms, near-term
algorithms are the current focus. Two especially enticing approaches to
exploiting noisy intermediate-scale quantum (NISQ) computers [38] involve
optimisation [39], which can use variational quantum algorithms [39, 40], and
quantum machine learning [41]. In both cases, a quantum advantage remains an
open question.

A business analysis of the potential for commercially viable quantum computing
commercially is in combinatorial optimisation [42]. This analysis identifies
three business verticals, which are niches that serve needs of a specific set
of customers. They identify the verticals of financial portfolio management,
computing lowest-energy configurations with applications to material design
and to pharmaceuticals, and predicting rare failures for advanced
manufacturing. The near-term gain would, one hopes, arise from exploiting
quantum-inspired algorithms [43]. Intriguingly, even without a quantum
advantage, a theoretical study suggests that a quantum economic advantage for
companies developing quantum computers can emerge even if quantum computers do
not offer a quantum computational advantage [44]. This argument for a quantum
computational advantage, even if quantum computers do not pay off, is based on
modeling a duopoly comprising a quantum computing company vs a classical
computing company. Their model shows a “quantum economic advantage”, even
without quantum computing advantage, partially through market creation.

Quantum machine learning aims to achieve quantum-enhanced machine learning,
such as enhancing unsupervised, supervised, generative or transfer learning.
Here, I summarise Wiebe’s list of key questions that a “quantum machine
learner” should ask [45]. Wiebe relates quantum-enhanced machine learning
specifically to certain potential advantages over using classical computing.
He identifies four sought-after quantum enhancements to machine learning,
namely, fewer computational steps for training or classifying, reducing sample
complexity, generating a new data model, and using quantum optimisation for
loss functions. As a second track of quantum machine learning research, Wiebe
summarises how a quantum device could be natural for classifying or extracting
features. To this end, he lists examples, namely, quantum Boltzmann machines
and quantum neural networks, quantum algorithms such as for principal
component analysis, and a quadratic time reduction for quantum-enhanced
function invertion applied to data-training.

Consultants are involved in prognosticating commercial applications of quantum
algorithms, and I share a little bit here of such endeavours. Gartner, Inc,
anticipates various quantum computing potential applications [46]. They regard
chemistry applications as first past the post, with such applications
requiring somewhere around one hundred to two hundred qubits. As the number of
qubits and quantum computational cycles increase, they forecast quantum
optimisation and then quantum machine learning as being viable, followed by
material science, with each of these application areas needing hundreds to
thousands of qubits. These forecasts are in line with the business forecast
discussed above [42]. Gartner’s analysis culminates with the expectation that,
eventually, quantum computers will sovle “unknown problems” requiring hundreds
of thousands of qubits.

A November 2020 TechRepublic report summarises six experts’ predictions for
2021 [47] beginning with IBM predicting it will achieve 127 qubits, which
indeed happened in 2021. The aforementioned Gartner, Inc, foresaw that cloud
providers would incorporate quantum capability, which is so. KnowBe4, a
company that focuses on security awareness training, predicted that quantum
computing would break traditional public-key cryptography; did not happen, and
I do not see this happening in the foreseeable future. Lux Research
foreshadowed advances in optimising quantum hardware to reduce resource needs
for running quantum algorithms. Finally, Forrester predicted that 2021 would
be a trough of disillusionment, which is the third of five phases of the
Gartner hype cycle [48, 49]. The hype cycle is a plot of maturity or
visibility of an emerging technology as a function of time, and these five
phases are the “technology trigger”, followed by the “peak of inflated
expectations”, then the “trough of disillusionment”, subsequently the “slope
of enlightenment” (waning interest with failure to deliver with survival
dependent on satisfying early adopters) and, finally, the “plateau of
productivity”.

##  5 Conclusions and outlook

We have reviewed the idea and advantages of quantum computing, the connection
with data science emphasising the computational side such as optimisation and
machine learning, and took an excursion into what consulting firms are saying.
Now we take stock of all this information to figure out where quantum
computing is today and how it affects data science and its applications and
verticals.

Clearly, quantum computers are a game changer for computing, at least at a
conceptual level. The idea that hard problems can be solved efficiently on a
quantum computer shows that quantum computing is more powerful, at least in
one type of application. That quantum computing is _provably_ quadratically
superior, by some measure, at inverting functions, also points to quantum
computing being superior in some way to classical computing.

On the other hand, quantum computers today are small and noisy, and, even if
they were large and coherent, their application to data science entails only
speculative advantages. Whether or not quantum computers are advantageous for
data science might only be resolved by building quantum computers, testing
algorithmic performance on such computers, and seeing whether an advantage has
been found or not; this empirical approach is already ongoing in the field.
Fortunately, quantum-inspired algorithms, which typically arise as ways to
challenge a purported quantum advantage by solving the same problem as well or
better on a classical computer, are giving beneficial computational results at
present that arguably would not be happening if quantum computing research
were not occurring.

Given the unknowns of quantum computing, including how much quantum computers
can scale up and whether we will have good algorithms that exploit the
potential of quantum computing for data science, an approach I call “quantum-
aware, quantum-ready, quantum-active” is prudent. This approach means that, at
this stage, we stakeholders in quantum computing, including data scientists,
need to be aware of when and how quantum computing could change the game. We
maintain awareness by testing how well existing quantum computers or
simulators thereof perform on problems of interest. If and when quantum
computing matters for the application at hand, it is time to become quantum-
ready, which means train up on the technology and prepare for transitioning to
quantum computing so that, when the time comes, we are ready to use it without
delay. Finally, in the optimistic scenario that quantum computing has become a
disruptive technology whose adoption is necessary, we have reached the
quantum-active stage where data scientists need to be fully engaged in using
quantum computing to solve problems. We need to remember that quantum
computing is a high-risk high-reward venture and treat it accordingly.

## Acknowledgments

This project has been funded by the Alberta Government and by NSERC. Thanks to
S. L. Sanders who prepared the figures over two decades ago; those figures
have been valuable for my presentations on Schrödinger’s cat ever since but
have not been published before.

## References

## References

  * [1] Nielsen M A and Chuang I L 2010 Quantum Computation and Quantum Information: 10th Anniversary Edition (Cambridge MA: Cambridge University Press)
  * [2] Aaronson S 2013 Quantum Computing since Democritus (Cambridge MA: Cambridge University Press)
  * [3] Dirac P A M 1939 Math. Proc. Cambridge Philos. Soc. 35 416–418
  * [4] Gottesman D 2020 Commun. ACM 64 105 ISSN 0001-0782
  * [5] Feynman R P 1982 Int. J. Theor. Phys. 21 467–488
  * [6] Shor P W 1994 Algorithms for quantum computation: discrete logarithms and factoring Proc. 35th Annual Symp. on Foundations of Computer Science pp 124–134
  * [7] Grover L K 1996 A fast quantum mechanical algorithm for database search Proceedings of the Twenty-Eighth Annual ACM Symposium on Theory of Computing STOC ’96 (New York, NY, USA: Association for Computing Machinery) p 212–219
  * [8] Heim B, Rønnow T F, Isakov S V and Troyer M 2015 Science 348 215–217
  * [9] Sanders B C 2017 How to Build a Quantum Computer (Bristol: IOP Publishing)
  * [10] Sanders B C 2020 Building a quantum computer (invited) 2020 ACM/IEEE International Workshop on System Level Interconnect Prediction (SLIP) pp 1–4
  * [11] MacQuarrie E, Simon C, Simmons S and Maine E 2020 Nat. Rev. Phys. 2 596–598
  * [12] Christensen C M 1997 The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail The Management of Innovation and Change (Boston: Harvard Business Review Press)
  * [13] Wu L, Wang D and Evans J A 2019 Nature 566 378–382
  * [14] Kotu V and Deshpande B 2019 Data Science: Concepts and Practice 2nd ed (Cambridge MA: Morgan Kaufmann)
  * [15] Muller M, Lange I, Wang D, Piorkowski D, Tsay J, Liao Q V, Dugan C and Erickson T 2019 How data science workers work with data: Discovery, capture, curation, design, creation CHI ’19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York: Association for Computing Machinery) pp 1–15
  * [16] Rønnow T F, Wang Z, Job J, Boixo S, Isakov S V, Wecker D, Martinis J M, Lidar D A and Troyer M 2014 Science 345 420–424
  * [17] Willsch D, Willsch M, De Raedt H and Michielsen K 2020 Computer Physics Communications 248 107006
  * [18] Blencowe M 2010 Nature 468 44–45
  * [19] Pirandola S, Andersen U L, Banchi L, Berta M, Bunandar D, Colbeck R, Englund D, Gehring T, Lupo C, Ottaviani C, Pereira J L, Razavi M, Shaari J S, Tomamichel M, Usenko V C, Vallone G, Villoresi P and Wallden P 2020 Adv. Opt. Photon. 12 1012–1236
  * [20] Fitzsimons J F 2017 npj Quantum Inf. 3 23
  * [21] Broadbent A and Jeffery S 2015 Quantum homomorphic encryption for circuits of low t-gate complexity Advances in Cryptology—CRYPTO 2015 ed Gennaro R and Robshaw M (Berlin: Springer) pp 609–629
  * [22] Dulek Y, Schaffner C and Speelman F 2016 Quantum homomorphic encryption for polynomial-sized circuits Advances in Cryptology—CRYPTO 2016 ed Robshaw M and Katz J (Berlin: Springer) pp 3–32
  * [23] Gribbin J 2014 Computing with Quantum Cats: From Colossus to Qubits (Prometheus) ISBN 9781616149215
  * [24] Schrödinger E 1935 Naturwissenschaften 23 807–812
  * [25] Gamow G 1928 Zeitschrift für Physik 51 204–212
  * [26] Bacon D 2006 Phys. Rev. A 73(1) 012340
  * [27] Mølmer K and Sørensen A 1999 Phys. Rev. Lett. 82(9) 1835–1838
  * [28] Mirrahimi M, Leghtas Z, Albert V V, Touzard S, Schoelkopf R J, Jiang L and Devoret M H 2014 New Journal of Physics 16 045014
  * [29] Sanders B C 2012 J. Phys. A: Math. Theor. 45 244002
  * [30] Lloyd S and Braunstein S L 1999 Phys. Rev. Lett. 82(8) 1784–1787
  * [31] Bartlett S D, Sanders B C, Braunstein S L and Nemoto K 2002 Phys. Rev. Lett. 88(9) 097904
  * [32] Wang Y, Hu Z, Sanders B C and Kais S 2020 Front. Phys. (Lausanne) 8 589504
  * [33] Crosson E J and Lidar D A 2021 Nat. Rev. Phys. 3 466–489
  * [34] Aaronson S and Arkhipov A 2013 Theory of Computing 9 143–252
  * [35] Dalzell A M, Harrow A W, Koh D E and La Placa R L 2020 Quantum 4 264
  * [36] Sanders B C 2021 Physics 14 147
  * [37] Jordan S 2021 Quantum algorithm zoo URL https://quantumalgorithmzoo.org
  * [38] Preskill J 2018 Quantum 2 79
  * [39] Moll N, Barkoutsos P, Bishop L S, Chow J M, Cross A, Egger D J, Filipp S, Fuhrer A, Gambetta J M, Ganzhorn M, Kandala A, Mezzacapo A, Müller P, Riess W, Salis G, Smolin J, Tavernelli I and Temme K 2018 Quantum Science and Technology 3 030503 URL https://doi.org/10.1088/2058-9565/aab822
  * [40] Cerezo M, Arrasmith A, Babbush R, Benjamin S C, Endo S, Fujii K, McClean J R, Mitarai K, Yuan X, Cincio L and Coles P J 2021 Nat. Rev. Phys. 3 625–644
  * [41] Schuld M, Sinayskiy I and Petruccione F 2015 Contemp. Phys. 56 172–185
  * [42] Bova F, Goldfarb A and Melko R G 2021 EPJ Quantum Technology 8 2
  * [43] Arrazola J M, Delgado A, Bardhan B R and Lloyd S 2020 Quantum 4 307
  * [44] Bova F, Goldfarb A and Melko R G 2022 Quantum economic advantage Working Paper 29724 National Bureau of Economic Research
  * [45] Wiebe N 2020 New J. Phys. 20 091001
  * [46] Gartner 2019 The CIO’s guide to quantum computing URL tinyurl.com/2s4dvjae
  * [47] Greig J 2020 6 experts share quantum computing predictions for 2021 URL tinyurl.com/y79vyds9
  * [48] Fenn J and Raskino M 2008 Mastering the Hype Cycle: How to Choose the Right Innovation at the Right Time (Gartner) (Boston: Harvard Business Review Press)
  * [49] Steinert M and Leifer L 2010 Scrutinizing Gartner’s hype cycle approach Technology Management for Global Economic Growth (PICMET), 2010 Proceedings of PICMET ’10 pp 1–13


# PrefGen: Preference Guided Image Generation with Relative Attributes

Alec Helbling  Christopher J. Rozell  Matthew O’Shaughnessy  Kion Fallah
Georgia Institute of Technology
Atlanta, GA 30332 USA
<EMAIL_ADDRESS>
###### Abstract

Deep generative models have the capacity to render high fidelity images of
content like human faces. Recently, there has been substantial progress in
conditionally generating images with specific quantitative attributes, like
the emotion conveyed by one’s face. These methods typically require a user to
explicitly quantify the desired intensity of a visual attribute. A limitation
of this method is that many attributes, like how “angry” a human face looks,
are difficult for a user to precisely quantify. However, a user would be able
to reliably say which of two faces seems “angrier”. Following this premise, we
develop the PrefGen system, which allows users to control the relative
attributes of generated images by presenting them with simple paired
comparison queries of the form “do you prefer image $a$ or image $b$?” Using
information from a sequence of query responses, we can estimate user
preferences over a set of image attributes and perform preference-guided image
editing and generation. Furthermore, to make preference localization feasible
and efficient, we apply an active query selection strategy. We demonstrate the
success of this approach using a StyleGAN2 generator on the task of human face
editing. Additionally, we demonstrate how our approach can be combined with
CLIP, allowing a user to edit the relative intensity of attributes specified
by text prompts. 111Code at https://github.com/helblazer811/PrefGen

##  1 Introduction

Deep generative models have demonstrated the ability to generate high-quality
images of content like human faces [23]. To expand their capabilities, there
is substantial interest in improving the controllability of these models,
providing the end user the ability to modify the continuous attributes of
output images [13, 33, 1, 22, 34]. Some of these techniques allow a user to
edit a set of visual attributes found through unsupervised learning [13, 33],
while others leverage supervised learning or pre-trained classifiers to form
mappings from attributes to the generative model latent space [1, 22, 34].
However, with all of these approaches, if a user wants to precisely modify the
intensity of a visual attribute, they must explicitly tune a set of
quantitative attributes. This user interface is ill-suited to the task of
modifying the relative attributes [24] of images, like how “sad” or “tired” a
person looks. Unlike properties such as “temperature,” people don’t share a
common notion of scale for attributes like “anger” [37], making it difficult
for them to explicitly quantify how “angry” a face looks. This limits the
utility of attribute-conditioned generative models to the class of attributes
that can be easily quantified.

Figure 1: Preference estimation of relative textual attributes. We estimate an
oracle’s preferences over the textual attributes “anger” and “red hair” using
PrefGen. Given a start image, PrefGen poses a sequence of pairwise comparisons
to determine an oracle’s preferred attributes in a target image. The top row
shows the generated estimate of an oracle’s preference after answering each
query. The second and third row represent the query prompted to the oracle,
with the images in the second row being preferred to those in the third. As
more queries are answered, the image generated from estimated preference
converges to the ground truth target in the final column. Figure 2: Proposed
system for preference guided image editing. After mapping an image to the
attribute space with $A(x)$, we present an oracle with a sequence of paired
comparison queries $Q_{t}=\\{q_{1}\dots q_{t}\\}$. Responses to the paired
queries are used to compute the posterior distribution over the attribute
space $p(a\mid Q_{t})$. We sample from this posterior distribution to produce
an estimate of the user’s ideal point $a^{*}$. We combine the ideal point
estimate with the latent vector of the input image $E(x_{i})=z_{i}$ to produce
a final vector in the generative model’s latent space $z_{f}\leftarrow
f(z_{i},a)$. Finally, we map our modified latent vector $z_{f}$ to image space
$G(z_{f})$ using our generator.

While people may not be able to explicitly specify how “angry” a face looks,
they can more easily say which of two faces looks “angrier” [17]. Motivated by
this observation, we develop the PrefGen system for controlling the relative
attributes [24] of images using the information from paired comparison queries
of the form “do you prefer image $x_{a}$ or image $x_{b}$?” By presenting a
user with a sequence of these queries, we can deduce their preferences over a
set of relative attributes without the need to explicitly quantify them. We
leverage the information from a sequence of these queries $Q$ to infer a
distribution of user preferences over a space of relative attributes
$\mathcal{A}$. We use a Bayesian framework for the attribute preferred by the
user, allowing us to actively select queries [32, 5, 14] and quantify
uncertainty. This allows us to efficiently search the space of attributes as
well as account for inconsistencies in a user’s responses, respectively.

Paired comparisons have long been regarded as a rich mode of information for
inferring user preferences over a set of items [17] as well as reasoning about
difficult-to-quantify psychological scales [37]. Additionally, paired
comparison queries can be passively extracted from widely available sources
like search engine mouse click data [27]. This provides the potential for the
proposed framework to control a generative model’s output using implicit
feedback from users, rather than requiring them to explicitly quantify their
preferences [13, 33, 1, 22, 34].

In our experiments, we demonstrate the success of our method with the
StyleGAN2 [16] architecture, a widely used Generative Adversarial Network
(GAN) [11] architecture. However, in principle, our method is sufficiently
general to be connected with any generative model with the ability to
conditionally generate images with different quantitative attributes. We apply
our methodology to the task of editing the attributes of human faces. We show
that PrefGen can be easily used in conjunction with existing approaches for
attribute-conditioned image generation. Specifically, we train supervised
mappings from attributes, like face age and pose, to latent vectors, as is
done in [34], and infer user preferences over those attributes using paired
comparisons. Recent works show that it is possible to modify the visual
attributes of images from textual information [25]. Building on this work, we
show that you can leverage joint vision-language models like CLIP [26] to edit
the relative intensity of visual attributes using the information from paired
comparisons.

The contributions of this work are:

  * •

The PrefGen system, which adds controllability to a pretrained generative
model by allowing users to edit the relative attributes of an output image by
answering a sequence of pairwise queries.

  * •

Empirical validation with two methods for building an attribute encoder, with
an extension to previous methods for active query selection that leverages a
continuous generative attribute space.

##  2 Background and Related Works

#### Relative Attributes

Many attributes have an inherent ordering, but are difficult for people to
explicitly quantify. For example, people can sort clothing based on how
“fashionable” they look and have an understanding of how “fancy” a dinner is,
but would likely struggle to consistently quantify these attributes. These are
called relative attributes [24] and there is a substantial body of work
attempting to model them. Much work focuses on learning to rank the relative
intensity of visual attributes [19, 41, 35]. A related line of work focuses on
learning metrics of similarity or similarity embeddings, where distances
between items in the embedding space correspond to notions of similarity [36,
2, 30]. The authors of WhittleSearch [18] have a similar motivation to us,
where they wish to search a finite set of images using relative attribute
feedback (paired comparison queries). However, our approach allows for
sampling from a continuous space of attributes using a generative model, and
not just a finite set of images.

Figure 3: Preference estimate update after a paired query. Presenting a user
with a paired query $q$ allows us to estimate a distribution over user
preference $p(a\mid q)$. After asking a query $q$, we can combine that
information with the set of previous queries $Q_{t}$ such that
$Q_{t+1}=Q_{t}\cup q$ and narrow down a region in attribute space that is
likely to align with user preference.

#### Controllable Generative Models

A difficult and open-ended problem in generative modeling is how to control
the attributes of generated content. Many generative models have the capacity
to generate high fidelity examples [11, 29], but require extra methodology to
control. There is a significant body of work focused on conditional sampling
of images with various continuous attributes using GANs [22, 1, 33, 25],
without modifying the weights of the underlying GAN. Some approaches deploy
supervised mappings with various underlying architectures, like normalizing
flows [1] and energy-based models [22]. Other methods attempt to infer
interpretable directions in StyleGAN’s latent space [33]. [34] train neural
network mappings from continuous attributes to StyleGAN latent space, and
fine-tune its representation so that it is disentangled (i.e. different latent
subspaces correspond to different interpretable attributes). All of these
methods require users to explicitly quantify continuous attributes. However,
many attributes are relative, and are difficult for user’s to explicitly
quantify. Motivated by this problem, we develop a method for user’s to control
the attributes of generated images by responding to paired comparison queries.
In our experiments, we demonstrate how our approach can be easily deployed on
top of existing conditional generation methods like [34] to conditionally
generate images with different continuous attributes using the information
from paired queries.

#### Text-driven Image Generation

There have been substantial recent advancements in guiding the outputs of
generative models using text [21, 28]. In the StyleGAN literature, there is a
body of work focusing on finding directions in the StyleGAN latent space that
correspond to different textual attributes [25, 1]. While supervised methods
for conditional generation of images with different attributes have impressive
capabilities, they require the use of large sets of annotated data or pre-
trained classifiers. This limits the user’s editing capabilities to a narrow
set of pre-defined attributes. In StyleCLIP [25], the authors show that it is
possible to edit StyleGAN generated images to match certain text prompts like
“an angry person” or “a cute cat.” They use the CLIP [26] joint text-image
embedding architecture to infer directions in StyleGAN latent space that
correspond to different textual attributes. Language provides a fixed lexicon
of adjectives and modifiers for describing the intensity of visual attributes.
This can make it challenging to precisely control the relative intensity of an
attribute with text alone. In our experiments, we deduce directions in CLIP
space using an approach similar to StyleCLIP, and demonstrate that it is
possible to use PrefGen to modify the intensity of relative textual
attributes.

#### Estimating Preferences with Paired Comparisons

Paired comparison queries of the form “do you prefer image $x_{a}$ or image
$x_{b}$” offer an elegant way to model a user’s preferences over relative
attributes [10, 3]. They unlock the ability to model intuitive concepts that
are difficult to explicitly describe or quantify [9, 37]. Additionally, paired
comparison queries can be passively collected from widely available data, like
search engine clicks [27, 4]. This means that it is possible to synthesize
user preference information from abundantly available sources and use
generative models to generate content that matches user preferences.

Our work closely draws on methods for efficiently searching pre-existing
similarity embeddings using the information from paired comparisons [5, 14, 8,
40]. A common simplifying assumption in this literature is to represent the
set of features or attributes that a user prefers by a single point, called
the ideal point $a^{*}\in\mathcal{A}$ [7]. We assume that given two points in
an embedding, a user would prefer the one closer to $a^{*}$. Our goal is to
estimate this ideal point using the information from paired queries. The
number of possible paired queries scales on the order of $O(N^{2})$ with the
number of images $N$, so it is typically infeasible and redundant to present a
user with all possible queries. The authors of [14] devise an approach to more
efficiently rank a fixed set of items by actively selecting paired queries.
Other approaches build on this methodology by attempting to search for the
ideal point over a continuous embedding [8, 40, 5].

[6] devise a probabilistic approach for estimating preferences and develop
methods for actively selecting queries based on information theory. We most
closely draw from the work of [5], which devises metrics for actively
selecting queries from a fixed set in a way that accounts for noise in their
responses. We apply their approach to the task of estimating user preferences
over a space of relative visual attributes, and build upon it by selecting
optimal queries from our continuous space instead of from a fixed set. To the
best of our knowledge, our work is the first that utilizes paired comparison
search with active querying to control the attributes of a generative model.

Perhaps most similar to our approach is the work of [12], where the authors
attempt to control a GAN given a set of paired comparison constraints.
However, they leverage a long short-term memory (LSTM) network to directly
predict a latent vector that generates an image that satisfies pairwise
constraints. We empirically show that this approach leads to inferior
performance over an analytic Bayesian framework [6, 5], potentially due to the
ill-posed problem of generalizing from training queries to a specific user’s
query responses. We adapt this approach as a baseline in Section 4.

##  3 Methods

Figure 4: Edited image over queries collected. Images generated to match
oracle preferences after N queries. Over time, the generated samples change
from the input image to more closely resemble the target image in the last
column.  Algorithm 1 Paired Comparison Based Image Generation

Input: Input Image $x_{i}$

Encode the input image to latent space $z_{i}=E(x_{i})$

Initialize query set $Q_{0}=\\{\\}$

for $t=1$ to $T$ do

Actively select a query $q=S(Q_{t})$

Prompt the oracle for response to $q$

Add the answered query to $Q_{t}=Q_{t-1}\cup q$

end for

Compute $\widehat{a}^{*}=\mathbb{E}[a\mid Q_{T}]$ using samples from MCMC.

Modify $z_{i}$ with our estimate $z_{f}=f(z_{i},\widehat{a}^{*})$

Generate the final edited image $x_{f}=G(z_{f})$

Output: $x_{f}$

Our goal is to allow an oracle to control the relative attributes of generated
images by asking simple paired comparison queries of the form “do you prefer
image $x_{a}$ or image $x_{b}$?” Using the information from a sequence of
these queries $Q_{t}=\\{q_{1},\dots,q_{t}\\}$ we want to estimate an oracle’s
preferences over a set of relative attributes $\mathcal{A}$. We can present an
unanswered query $q_{i}=(x_{a},x_{b})$ to an oracle $O(q_{i})$ which returns
an answered query $O(q)=(x_{p},x_{n})$. Thus,

$q_{i}=(x_{p},x_{n})\iff x_{p}\prec x_{n},$ (1)

where $x_{p}\prec x_{n}$ indicates that oracle “prefers” the image $x_{p}$ to
the image $x_{n}$. We assume that there exists some vector
$a^{*}\in\mathcal{A}$ called the target attribute which best represents an
oracle’s preferences. Our goal is to infer $a^{*}$ from the information in
queries $Q_{t}$.

Once we have an estimate $\widehat{a}^{*}$ of the oracle’s ideal point, we
need a way to generate images that are consistent with those preferences. We
do this by mapping the attribute to the generative model latent space
$\mathcal{Z}$. We represent this with an attribute mapping function, from a
latent vector $z_{i}$ and attribute $a$ to another latent vector
$z_{f}\leftarrow f(z_{i},a)$. We can both modify an initial latent vector
$z_{i}$, perhaps found using GAN Inversion $z_{i}=E(x_{i})$ [38], or generate
images using only the information from our attribute $z_{i}\sim p(z)$. Our
goal is to model $f(z_{i},a)$ so that $A(G(z_{f}))\approx a$ and the
identities of $G(z_{f})$ and $G(z_{i})$ are the same, where $A(x)$ is a
function that predicts the attributes of an image $x\in\mathcal{X}$. We
explore two key ways of modeling $A(\cdot)$ and $f(\cdot)$ in Section 3.3.

Figure 5: Qualitative trial run of PrefGen. We demonstrate two trial runs of
PrefGen in a 4D space formed by yaw, pitch, and age of human faces using a
supervised attribute mapping. See Figure 1 for figure details. We show each of
the queries presented to the oracle, and images generated by sampling from the
posterior after each query. Sample images qualitatively converge to the target
image as query responses are collected.

###  3.1 A Bayesian Model of User Preferences

We use a Bayesian framework to estimate the distribution $p(a\mid Q_{t})$ of
user preferences over attribute space $\mathcal{A}$ using the information from
the queries $Q_{t}$. We use a simplifying assumption, called the ideal point
model [7], that a user’s preference for a set of attributes $a\in\mathcal{A}$
corresponds to the distance between that vector $a$ and the target attributes
$a^{*}$. Let $A(x)$ be a (perhaps unknown) mapping from image space
$\mathcal{X}$ to attributes $\mathcal{A}$. We say a user “prefers” a positive
image $x_{p}$ to a negative image $x_{n}$ when the attributes $a_{p}=A(x_{p})$
of the positive image $x_{p}$ are closer to the target attributes $a^{*}$ than
the attributes $a_{n}=A(x_{n})$ of the negative image $x_{n}$:

$x_{p}\prec x_{n}\iff\|a^{*}-a_{p}\|<\|a^{*}-a_{n}\|.$ (2)

Following the assumption in Equation 2 and work from [5] we represent the
likelihood that $x_{p}\prec x_{n}$ as

$p(q\mid a)=\sigma\bigl{(}k_{q}(\|a-a_{n}\|^{2}-\|a-a_{p}\|^{2})\bigr{)},$ (3)

where $\sigma$ is the logistic function and $k_{q}$ is a scalar value that
accounts for the noisiness of oracle responses. Intuitively, this likelihood
produces a very high probability when a user prefers an image with attributes
$a_{p}$ that is much closer to the ideal point $a^{*}$ than the image with
attributes $a_{n}$. The noise constant $k_{q}$ can be thought of as a signal-
to-noise ratio that determines the expected amount of information contained in
each query. It is possible to for the oracle to accommodate for an expectation
of different amounts of noise in different queries by changing $k_{q}$ with
different queries [5].

By representing this problem probabilistically, we can accommodate various
sources of error, such as “noise” in user responses to paired queries. We can
combine the information from multiple paired queries through a recursive
application of Bayes rule,

$p(a\mid q_{1},...,q_{t})=\frac{p(a)\prod_{j=1}^{t}p(q_{j}\mid a)}{p(q_{t}\mid
q_{1},...,q_{t-1})},$ (4)

where our prior over the user preference $p(a)$ is a uniform distribution.
Finally, we use MCMC to sample from the posterior distribution $p(a\mid
Q_{t})$ where $Q_{t}=\\{q_{1},\dots,q_{t}\\}$ and take the expected value
$\mathbb{E}[a\mid Q_{t}]$ to form an estimate of the target attributes
$a^{*}$. Intuitively, this process can be thought of as a multidimensional
generalization of the binary search algorithm. This procedure is shown in
Algorithm 1 and Figure 3.

Figure 6: Comparing the quantitative performance of various approaches to
estimating user preferences from paired comparisons. We simulate an oracle’s
response to a sequence of queries for 20 trials, with 30 queries per trial. We
search a 2D space formed by the age and the yaw (left-right pose) of a face.
(Left) MSE in attribute space of preference estimate after $t$ queries and the
ground truth attribute $a^{*}$ for a given trial. (Right) Percentage of
attribute vectors from a batch of 1000 that are closer to the target
attributes than the estimate. According to both metrics, PrefGen is able to
efficiently estimate preferences with high precision. As expected, active
querying improves the efficiency of preference estimation as compared to
randomly querying, and the Closed Form approach, which leverages the
continuous generative model latent space, significantly outperforms the Best
of N approach.

###  3.2 Actively Query Selection for Efficient Preference Estimation

It is generally infeasible to present a user with every possible paired
comparison query. Given a fixed set of $N$ images, one can construct
$O(N^{2})$ paired queries from them. Furthermore, in our setting we have a
generative model capable of continuously sampling from attribute space, so we
have an infinite number of possible queries to choose from. So the problem
stands, how do we select a set of queries to best estimate the user’s
preferences? A naive solution, which we call random querying, is to simply
select a small, random subset of the many possible queries. However, many
queries are redundant given the information from previous queries that were
presented to a user.

We build on methods from active learning [32] to select queries that minimize
the amount of uncertainty in our posterior distribution $p(a\mid Q_{t})$. The
authors of [5] develop strategies for selecting queries that contain as much
new information as possible about the ideal point $a^{*}$ from a discrete set
of candidate queries. One such strategy is called mean-cut max-variance
(MCMV). Intuitively, the goal of MCMV is to select a query $q=(a_{p},a_{n})$
along an axis of high variance as determined by the posterior $p(a\mid
Q_{t})$, and that bisects the mean of the posterior $\mathbb{E}[a\mid Q_{t}]$.
We can represent the plane that bisects the attributes $a_{p}$ and $a_{n}$
with a normal vector $v=2(a_{p}-a_{n})$ and scalar offset
$\tau=|a_{p}|^{2}-|a_{n}|^{2}$. We can compute the variance of the posterior
distribution in the direction of the query computed by
$\sigma_{q}=v\Sigma_{a}v^{T}$ where $\Sigma_{a}$ is the covariance of samples
from our posterior. Cutting the mean means selecting a query $q=(a_{p},a_{n})$
such that the plane between the two query items parameterized by $v$ and
$\tau$ intersects with the mean of the distribution. This can be computed as
$\mu_{q}=(v^{T}\mathbb{E}[a\mid Q_{t}]-\tau)/|v|$. So the best query is the
one that minimizes $k_{q}\sigma_{q}-\lambda\mu_{q}$, where $k_{q}$ is our
aforementioned noise constant and $\lambda$, is a hyperparameter used to
balance the relative importance of the mean-cut and maximum-variance criteria.

In practice, we need to sample many potential queries from our attribute
space, measure the MCMV utility of each, and select the query with the highest
value. As we converge on an estimate of the oracle’s preferences, performance
starts to degrade since the best query is farther from the mean and axis of
maximum variance relative to the volume of the posterior (See Figure 6). Our
generative modeling framework grants us the ability to sample continuously
from attribute space (See Section 3.3). This enables us to directly select a
query that is mean-cut and max-variance, obviating the need to sample large
numbers of candidate queries. We simply select a query with points equidistant
to the mean that are along the axis of maximum variance of our posterior
$p(a\mid Q_{t})$. It is worth noting that this approach does not account for
models of noise that may interfere with the expected informativeness of
different queries. We call this approach active closed-form and the original
MCMV approach active best of N.

###  3.3 Controlling the Attributes of Generated Images

In this paper, we explore two key approaches for modeling the attribute
mapping function $f(z_{i},a)$. It is important to note that these two
approaches are just two of many possible ways one could construct a mapping
from attribute vectors to latent vectors (or directly images).

#### Supervised Attribute Mapping: Constructing Supervised Mappings from
Attributes to Latent Space

For our first method we train supervised mappings from attributes like face
pose and age to our generator’s latent space. For the implementation of this
approach, we build on top of GANControl [34], where the authors train
supervised mappings from attributes to a disentangled StyleGAN latent space.
This work shows that it is possible to modify quantitative attributes without
significantly changing the underlying identity of images.

#### CLIP Attribute Mapping: Modifying Relative Textual Attributes with
StyleCLIP

In StyleCLIP [25], the authors introduce a method to find a latent vector that
generates an image with a certain attribute, without altering the identity of
a given input latent vector. They describe a gradient based approach for
finding latent vectors that generate images with certain textual attributes.
Building off of StyleCLIP, we encode model relative attributes, allowing us to
interpolate between two text prompts: a neutral text prompt like “a person
with a neutral expression” and a target text prompt “a person with an angry
expression”. We can embed these prompts into CLIP [26] space using the CLIP
text encoder $\mathcal{T}(t)$ to produce a neutral and target vector $t_{n}$
and $t_{t}$ respectively. Images with cosine similarity close to $t_{n}$ will
have neutral expressions, and those closer to $t_{t}$ will better match the
target prompt. Let $\mathcal{V}(x)$ be the CLIP vector of an image
$x\in\mathcal{X}$. We can predict the intensity of an attribute by computing
the cosine similarity between $\mathcal{V}(x)$ and our attribute direction
$t_{a}=t_{t}-t_{n}$ (i.e.
$\frac{\langle\mathcal{V}(x),t_{a}\rangle}{\|\mathcal{V}(x)\|\|t_{a}\|}$). We
can then modify an image’s attribute by simply choosing some $\alpha\in[0,1]$
and minimizing the distance between our current predicted attribute intensity
and $\alpha$. We do this using gradient descent as is done in StyleCLIP with
additional penalty terms to ensure the identity of the image stays the same.
This approach allows us to quantitatively model the intensity of a relative
attribute, which is difficult to do by directly encoding text prompts as is
done in [28].

##  4 Experiments

The focus of our experiments is to evaluate the ability of our proposed method
to effectively estimate user preferences over a space of attributes from
paired comparison queries and conditionally generate images that match those
preferences. A core evaluation method that we use is to simulate user
responses to queries with a simple oracle. For each preference estimation
trial, we sample some arbitrary target attribute vector $a^{*}\in\mathcal{A}$
uniformly from $[0,1]^{d}$, which corresponds to the $d$ image attributes that
our user wants to be in samples images. We then sequentially pose several
paired comparison queries, each of the form $q=(a_{p},a_{n})$, such that they
satisfy Equation 2.

###  4.1 PrefGen with a Supervised Attribute Mapping

#### Implementation Details

One way to represent $f$ is to train a neural network with supervised learning
that maps attributes to the latent space of a generative model. We use the
methods from GANControl [34] to construct supervised mappings from the pose
(yaw, pitch, and roll) and the apparent age of a face to the latent space of a
StyleGAN2 [15] using the FFHQ Human Attributes dataset [16]. In our
quantitative experiments, for each method we conduct 20 trials of preference
estimation, each with 30 queries and $k_{q}=10$.

#### Quantitative Evaluation of PrefGen

Figure 7: Improving active selection. (Left) Preference estimation performance
with 30 query responses as the number of candidate queries increases using
Best of N. (Right) Even as the number of candidate queries increases, the Best
of N strategy has higher attribute estimation error compared to the proposed
Closed Form strategy.

To measure our ability to estimate a user’s target attributes $a^{*}$ both
precisely and efficiently, we use three key metrics: (1) the Mean Squared
Error (MSE) of our current estimate and the ground truth attribute in the
attribute space after each query response, (2) the percentage of attributes
closer than our estimate to the target attributes $a^{*}$, and (3) the
percentage of paired comparison constraints where the estimate of the target
attributes $a^{*}$ is closer to the positive than the negative. Figure 6 shows
that our approaches (both active and random) are able to converge on
preference estimates with low MSE to the target attributes and that are closer
than a large percentage of randomly sampled attribute vectors. Table 1 further
confirms these results, and shows that PrefGen can estimate a set of
attributes that satisfy a high percentage of constriants. Further results in
Appendix C show that this pattern is also replicated when searching over
higher dimensional attribute spaces.

Figure 8: Robustness of our method to query response noise. Preference
estimation under additive Gaussian noise. (Left) As noise is added,
performance gradually degrades, while still remaining in the top estimation
percentile. (Right) Uncertainty, as measured by the determinant of attribute
posterior covariance, increases with increased noise.

#### Comparison to a Baseline Method

We implement a baseline method from [12], which we call LSTM Baseline. In this
paper, the authors use an LSTM to estimate a latent vector that generates an
image which satisfies a sequence of paired comparison constraints. In our
approach, we separate preference estimation and mapping to the generative
model latent space. For the sake of comparison, we extend the LSTM approach to
estimate a low dimensional attribute vector that satisfies a given set of
constraints. Further details in the implementation of this baseline are show
in the Appendix B. As shown in Figure 6 and Table 1, our method with both
random and active querying can estimate the target attributes with higher
precision and efficiency.

| MSE to G.T. Attribute $\downarrow$ | Percentage Closer ($\%$) $\downarrow$ | Constraint Satisfied ($\%$) $\uparrow$
---|---|---|---
LSTM | 5.86e-2 $\pm$ 5.60e-2 | 13.330 $\pm$ 10.390 | 83.83 $\pm$ 10.290
Random | 1.89e-1 $\pm$ 1.64e-2 | 5.731 $\pm$ 5.209 | 97.17 $\pm$ 2.420
Active (Best of N) | 5.37e-4 $\pm$ 6.41e-4 | 0.145 $\pm$ 0.139 | 98.50 $\pm$ 2.467
Active (Closed Form) |  1.69e-5 $\pm$ 2.35e-5 |  0.003 $\pm$ 0.001 |  99.83 $\pm$ 0.007
Table 1:  Performance of Preference Estimation on Various Methods. We
quantitatively evaluate different preference estimation approaches after 30
queries on 20 experiment trials. MSE to Ground Truth measures the Mean Squared
Error of our final preference estimate in attribute space to the ground truth.
Percentage Closer ($\%$) shows the percentage of attribute vectors in
attribute space that are closer to the ground truth than the preference
estimate. Constraint Satisfied ($\%$) shows the percentage of collected
queries that are satisfied by our final preference estimation. For all
metrics, our approaches (Random, Active (Best of N) and Active (Closed Form))
outperform the LSTM baseline, and Active (Closed Form) outperforms all other
methods across each metric.

#### Qualitative Performance of PrefGen

Figure 4 shows how the image sampled for a given preference estimate changes
as more queries are presented to the oracle. After each query we sample a
latent vector $z_{f}=f(z_{i},\mathbb{E}[a\mid Q_{t}])$, with attributes
corresponding to the expected user preferences $\mathbb{E}[a\mid Q_{t}]$, and
generate the corresponding image $G(z_{f})$. Qualitatively, the attributes of
the preference samples converge to the attributes of the target images. Figure
5 shows two trial runs of our PrefGen procedure. Qualitatively, it is clear
that the preference estimate is converging on the target attributes using the
information provided by each paired comparison query.

#### Robustness to Query Response Noise

In most practical settings, various sources of noise can cause errors or
inconsistencies in an oracle’s responses to queries. Hence, it is desirable to
build a system that is robust to such noise. To evaluate PrefGen’s robustness
to query noise, we purposely add Gaussian noise to our query likelihood as
follows: $\sigma(k_{q}(||a-a_{n}||^{2}-||a-a_{p}||^{2}+n))$ where
$n\sim\mathcal{N}(0,\sigma_{n})$ and $\sigma_{n}$ is the degree of noise we
add. Figure 8 shows the percentage of attribute vectors from a large sample
(1000) that are closer to the target attributes than the preference estimate
after 30 queries for 20 trials. For each degree of noise, we tune $k_{q}$
using maximum likelihood estimation on a set of training triplets (see
Appendix B for more details). As the degree of noise increases, we note the
performance of preference estimation degrades, but more importantly, it does
not catastrophically fail. Additionally, we can quantify uncertainty in our
posterior $p(a\mid Q_{t})$ by computing the determinant of the covariance
matrix of samples drawn from $p(a\mid Q_{t})$, which shows that as we increase
the amount of noise there is an increase in the uncertainty of our posterior.

#### Effectiveness of Closed Form Active Querying

Section 3.2 discusses the importance of deploying a method which actively
selects queries informative of an oracle’s preference and effective for
estimation with fewer responses. Figure 6 clearly shows that actively
selecting queries both drastically improves the efficiency of our preference
estimation procedure and the final precision. Additionally, it is clear that
the proposed Closed Form active querying method outperforms the Best of N
approach. Figure 7 shows that Closed Form continues to outperform the Best of
N approach as one increases the number of candidate samples.

###  4.2 PrefGen with Textual Attributes

We demonstrate that it is possible to use PrefGen to edit relative attributes
described by text prompts. We find two directions in CLIP space, corresponding
to the attributes “anger” and “red hair”. For “anger” $t_{n}=\textit{``a
person with a neutral expression"}$ and $t_{p}=\textit{``a person with an
angry expression"}$ and for “red hair” we use $t_{n}=\textit{``a person with
dark hair"}$ and $t_{p}=\textit{``a person with red hair"}$. Figure 1 shows
PrefGen in this 2D space formed by these attributes. We are not just
interested in editing vectors within a GAN latent space, we also want to be
able to edit real world images. To this end, we deploy GAN Inversion
techniques, specifically use the Encoder4Editing [38], to embed images into
the latent space of our StyleGAN2 model, and edit those vectors.

##  5 Discussion

In this work, we propose PrefGen, a framework for controlling generative
models using the information from pairwise comparisons. We demonstrate the
success of this approach using StyleGAN2 on the task of human face editing.
Our method works in conjunction with both a supervised attribute mapping and
CLIP-based textual attribute mapping.

A potential limitation of our experimental approach is that we deploy a
simulated oracle in our experiments as a stand-in for a user’s responses to
queries instead of a user directly. However, we believe this is a reasonable
assumption as we specifically construct our attribute embeddings to correspond
to human interpretable features, and there is a substantial body of work
studying the utility of pairwise comparisons for modeling user preferences in
that setting [9, 18]. Another potential limitation is that we use linear
directions in CLIP space to model relative attributes, however, there is no
guarantee that these linear directions will encode the relative intensity of
textual features. A potential line of future work is to better leverage joint
text-image models like CLIP to model relative attributes.

Despite their potential benefits, generative models also raise diverse ethical
concerns that can be particularly acute when applied to images of people.
Potential harmful uses include the creation of disinformation [31] or
“deepfakes” [20]. We urge developers applying this technology to carefully
consider potential benefits and harms, and to pursue strategies to mitigate
harms (e.g., [39]) when appropriate.

##  6 Acknowledgements

We would like to thank the generous support and guidance of the members of the
Georgia Tech SIPLab. This work was partially supported by Georgia Tech’s
President’s Undergraduate Research Award (PURA), the NSF CAREER award
CCF-1350954, and the NSF Research Experiences for Undergraduates (REU)
program, and an ONR Grant. We would also like to thank Asa Harbin for helping
with the manuscript.

## References

  * [1] R. Abdal, P. Zhu, N. J. Mitra, and P. Wonka.  StyleFlow: Attribute-conditioned exploration of StyleGAN-generated images using conditional continuous normalizing flows.  ACM Transactions on Graphics, 40(3):1–21, may 2021.
  * [2] S. Agarwal, J. Wills, L. Cayton, G. Lanckriet, D. Kriegman, and S. Belongie.  Generalized non-metric multidimensional scaling.  In Artificial Intelligence and Statistics, pages 11–18. PMLR, 2007\.
  * [3] M. Aggarwal and A. Fallah Tehrani.  Modelling human decision behaviour with preference learning.  INFORMS Journal on Computing, 31(2):318–334, 2019.
  * [4] R. Atterer, M. Wnuk, and A. Schmidt.  Knowing the user’s every move: User activity tracking for website usability evaluation and implicit interaction.  In Proceedings of the 15th International Conference on World Wide Web, WWW ’06, page 203–212, New York, NY, USA, 2006. Association for Computing Machinery.
  * [5] G. Canal, A. Massimino, M. Davenport, and C. Rozell.  Active embedding search via noisy paired comparisons.  In International Conference on Machine Learning (ICML), Long Beach, CA, June 2019.
  * [6] W. Chu and Z. Ghahramani.  Preference learning with gaussian processes.  In Proceedings of the 22nd International Conference on Machine Learning, ICML ’05, page 137–144, New York, NY, USA, 2005. Association for Computing Machinery.
  * [7] C. H. Coombs.  Psychological scaling without a unit of measurement.  Psychological Review, 57(3):145–158, 1950.
  * [8] M. A. Davenport.  Lost without a compass: Nonmetric triangulation and landmark multidimensional scaling.  In 2013 5th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), pages 13–16. IEEE, 2013\.
  * [9] H. A. David.  The method of paired comparisons, volume 12.  London, 1963.
  * [10] J. Fürnkranz and E. Hüllermeier.  Preference learning and ranking by pairwise comparison.  In Preference learning, pages 65–82. Springer, 2010.
  * [11] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio.  Generative adversarial networks, 2014.
  * [12] E. Heim.  Constrained generative adversarial networks for interactive image generation, 2019.
  * [13] E. Härkönen, A. Hertzmann, J. Lehtinen, and S. Paris.  Ganspace: Discovering interpretable gan controls.  2020\.
  * [14] K. G. Jamieson and R. D. Nowak.  Active ranking using pairwise comparisons.  In Advances in Neural Information Processing Systems, 2011.
  * [15] T. Karras, S. Laine, and T. Aila.  A style-based generator architecture for generative adversarial networks, 2018.
  * [16] T. Karras, S. Laine, and T. Aila.  A style-based generator architecture for generative adversarial networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401–4410, 2019.
  * [17] M. G. Kendall and B. B. Smith.  On the method of paired comparisons.  Biometrika, 31(3/4):324–345, 1940.
  * [18] A. Kovashka, D. Parikh, and K. Grauman.  Whittlesearch: Image Search with Relative Attribute Feedback.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2012.
  * [19] T.-Y. Liu et al.  Learning to rank for information retrieval.  Foundations and Trends® in Information Retrieval, 3(3):225–331, 2009.
  * [20] M. Masood, M. Nawaz, K. M. Malik, A. Javed, and A. Irtaza.  Deepfakes generation and detection: State-of-the-art, open challenges, countermeasures, and way forward, 2021.
  * [21] A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen.  Glide: Towards photorealistic image generation and editing with text-guided diffusion models, 2021.
  * [22] W. Nie, A. Vahdat, and A. Anandkumar.  Controllable and compositional generation with latent-space energy-based models.  In NeurIPS, 2021.
  * [23] A. Oussidi and A. Elhassouny.  Deep generative models: Survey.  In 2018 International Conference on Intelligent Systems and Computer Vision (ISCV), pages 1–8, 2018.
  * [24] D. Parikh and K. Grauman.  Relative attributes.  In Proceedings of the 2011 International Conference on Computer Vision, ICCV ’11, page 503–510, USA, 2011. IEEE Computer Society.
  * [25] O. Patashnik, Z. Wu, E. Shechtman, D. Cohen-Or, and D. Lischinski.  Styleclip: Text-driven manipulation of stylegan imagery, 2021.
  * [26] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever.  Learning transferable visual models from natural language supervision, 2021.
  * [27] F. Radlinski and T. Joachims.  Active exploration for learning rankings from clickthrough data.  In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’07, page 570–579, New York, NY, USA, 2007. Association for Computing Machinery.
  * [28] A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, and I. Sutskever.  Zero-shot text-to-image generation, 2021.
  * [29] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer.  High-resolution image synthesis with latent diffusion models, 2021.
  * [30] F. Schroff, D. Kalenichenko, and J. Philbin.  Facenet: A unified embedding for face recognition and clustering.  2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2015.
  * [31] K. Sedova.  AI and the Future of Disinformation Campaigns: Part 2, a Threat Model.  Center for Security and Emerging Technology, 2021.
  * [32] B. Settles.  Active learning.  Synthesis lectures on artificial intelligence and machine learning, 6(1):1–114, 2012.
  * [33] Y. Shen, J. Gu, X. Tang, and B. Zhou.  Interpreting the latent space of gans for semantic face editing.  In CVPR, 2020.
  * [34] A. Shoshan, N. Bhonker, I. Kviatkovsky, and G. Medioni.  Gan-control: Explicitly controllable gans.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14083–14093, 2021.
  * [35] K. K. Singh and Y. J. Lee.  End-to-end localization and ranking for relative attributes.  In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI 14, pages 753–769. Springer, 2016.
  * [36] O. Tamuz, C. Liu, S. Belongie, O. Shamir, and A. T. Kalai.  Adaptively learning the crowd kernel.  In International Conference on Machine Learning (ICML), 2011.
  * [37] L. L. Thurstone.  A law of comparative judgement.  Psychological Review, 34:278–286, 1927.
  * [38] O. Tov, Y. Alaluf, Y. Nitzan, O. Patashnik, and D. Cohen-Or.  Designing an encoder for stylegan image manipulation, 2021.
  * [39] J. D. Weisz, M. Muller, J. He, and S. Houde.  Toward general design principles for generative ai applications, 2023\.
  * [40] A. Xu and M. Davenport.  Simultaneous preference and metric learning from paired comparisons.  Advances in Neural Information Processing Systems, 33, 2020.
  * [41] X. Yang, T. Zhang, C. Xu, S. Yan, M. S. Hossain, and A. Ghoneim.  Deep relative attributes.  IEEE Transactions on Multimedia, 18(9):1832–1842, 2016.

##  Appendix A Methods Details

##  Appendix B Experiments Details

###  B.1 Selecting an Optimal Noise Constant

The performance of preference estimation is sensitive to certain
hyperparameters. One such parameter is the noise constant $k_{q}$ which acts
as a signal-to-noise ratio in our Bayesian approach to estimating user
preferences. For some experiments we choose a constant $k_{q}=10$ which was
found to generally give strong performance. This is satisfactory when
comparing different approaches where the degree of noise in user responses to
queries remains fixed (e.g. the experiment showed in Figure 6). However, in
Figure 8 we show results from an experiment investigating the impact of
different degrees of noise on the preference estimation performance and
uncertainty quantification capabilities of our methodology. For this
experiment, it is necessary to tune a noise constant $k_{q}$ for each degree
of noise.

The way we tune $k_{q}$ for each degree of noise is to generate a small
dataset ($\approx 1000$ examples) of attribute triplets
$t=(a_{a},a_{p},a_{n})\in T$ that satisfy $|a_{a}-a_{p}|+n<|a_{a}-a_{n}|$
where $n\sim N(0,\sigma_{n})$. Here we are generating a dataset of triplets
with varying degrees of noise, parameterized by the standard deviation
$\sigma_{n}$. We want to select a noise constant $k_{q}$ for each degree of
noise that maximizes the likelihood

$\sum_{(a_{a},a_{p},a_{n})\in
T}\sigma(k_{q}(||a_{a}-a_{n}||^{2}-||a_{a}-a_{p}||^{2})).$ (5)

Intuitively, as the degree of noise in our dataset increases we want our model
to become less confident in its estimate of the user’s preferences. The way
this is encoded is by decreasing the noise constant $k_{q}$ so that the
$\sigma(k_{q}(||a_{a}-a_{n}||^{2}-||a_{a}-a_{p}||^{2}))$ goes closer to $0.5$
on average. This means that as we decrease $k_{q}$ we weight the impact of any
given query lower, leading to slower convergence, and higher posterior volume.

###  B.2 Preference Estimation Experimental Procedure

Given a generative model $G(z)$ (in our case a StyleGAN2 [16] model) we
simulate a user’s responses to paired queries and attempt to estimate a user’s
preferences over a set of attributes $\mathcal{A}$. Let $T$ be the number of
queries we present to a user. Let $a^{*}\in\mathcal{A}$ be the ground truth
attributes that describe a user’s preferences over a set of attributes
$\mathcal{A}$. Our goal is to approximate $a^{*}$.

For each trial we follow this procedure:

  1. 1.

First we generate a set of target attributes $a^{*}\in\mathcal{A}$ sampled
uniformly from $\mathcal{A}$.

  2. 2.

We randomly sample an initial latent vector $z_{i}\in\mathcal{Z}$ from our
StyleGAN2 latent space (actually $\mathcal{W}$ space of StyleGAN2, but we call
it $\mathcal{Z}$ for convenience).

  3. 3.

$Q_{0}=\\{\\}$

  4. 4.

For $T$ paired queries:

    1. (a)

Sample a paired query $q=(a_{a},a_{b})$ of attributes using the desired
sampling scheme (Active, Random, etc.).

    2. (b)

Generate the corresponding set of images for the query
$q_{x}=(G(f(z_{i},a_{a})),G(f(z_{i},a_{b})))$

    3. (c)

Answer the query using a simulated oracle.

    4. (d)

$Q_{t}=Q_{t-1}\cup q$

    5. (e)

Use the Stan modeling language 222https://mc-stan.org/ to sample from the
posterior $p(a\mid Q_{t})$ using MCMC.

    6. (f)

Compute the current posterior estimate $\mathbb{E}[a\mid Q_{t}]$ of the
oracle’s preferences.

    7. (g)

Compute the estimate latent vector $z_{t}=f(z_{i},\mathbb{E}[a\mid Q_{t}])$.

    8. (h)

Generate the estimate image $x_{t}=G(z_{t})$.

    9. (i)

Compute any desired evaluation metrics as a function of $z_{t},x_{t}$ and
$z_{i}$.

###  B.3 LSTM Baseline Implementation Details

In the core paper, we implemented a variation of [12], which we call the LSTM
Baseline. Here we describe the alterations that we made to their architecture
for the sake of fair comparison.

Figure 9: Comparing the quantitative performance of various approaches to
estimating user preferences from paired comparisons (4D Yaw, Pitch, Roll, and
Age). This figure is similar to Figure 6 from the core paper. We simulate an
oracle’s response to a sequence of queries for 20 trials, with 30 queries per
trial. We search a 4D space formed by the age and the pose (yaw, pitch, and
roll) of a face. (Left) MSE in attribute space of preference estimate after
$t$ queries and the ground truth attribute $a^{*}$ for a given trial. (Right)
Percentage of attribute vectors from a batch of 1000 that are closer to the
target attributes than the estimate. According to both metrics, PrefGen is
able to efficiently estimate preferences with high precision. As expected,
active querying improves the efficiency of preference estimation as compared
to randomly querying, and the Closed Form approach, which leverages the
continuous generative model latent space, significantly outperforms the Best
of N approach.

#### The Original Approach

In [12], the author wants to generate images from a GAN, given a set of
pairwise constraints. This motivation is very similar to ours, however, their
approach is substantially different. They use an LSTM architecture to estimate
a latent vector $z_{f}$ that satisfies a given set of constraints given an
input latent vector $z_{i}$.

The LSTM architecture is defined as follows. The architecture has three core
components. The first component is a read network $\phi(x)$ that maps image
constraints to a semantic space, this semantic space serves a similar function
to our attribute space $\mathcal{A}$. The second component is a process
network that combines all of the constraints into a single representation,
this serves a similar function to our Bayesian preference estimation
procedure. The third component is a write network which maps a final
representation to an image, which is what our GAN generator does.

Let $\\{c_{1},\dots,c_{n}\\}$ be a set of embedded constraints produced by the
read network. Each constraint $c_{i}=a_{p}\|a_{n}$ for a query
$q_{i}=(a_{p},a_{n})$, where $\|$ corresponds to vector concatenation. The
authors describe a process network that takes in these constraints and an
input latent vector $z$ and produce an output latent vector $z_{f}$ that can
be fed into a generator to produce an image that satisfies constraints. The
LSTM framework for the process network is described below:

$h_{t}=LSTM(z,h_{t-1}^{*})$ (6) $e_{i,t}=c_{i}\cdot b_{t}$ (7)
$a_{i,t}=\frac{exp(e_{i,t})}{\sum_{j}^{n}exp(e_{j,t})}$ (8)
$r_{t}=\sum_{i}^{n}a_{i,t}c_{i}$ (9) $h_{t}^{*}=h_{t}\|r_{t}$ (10)

As described in [12] an input $z$ and the hidden state from the previous
repetition are put through the LSTM. The next hidden state $h_{t}$ is then
combined with each constraint $c_{i}$ using a dot product to produce a scalar
value $e_{i,t}$ for each constraint. These values are then fed into a softmax
function to obtain the scalars $a_{i,t}$ which is used in a weighted sum. The
sum is the operation that combines the constraints. The final $r_{t}$ is
concatenated to $h_{t}$ and is used as the input in the next processing
iteration. This process is run for $p$ steps, to eventually produce
$h_{p}^{*}$ which is fed into a fully connected layer to produce $s$ which is
directly fed into the write network (generator).

The authors train this framework end-to-end using a triplet loss and an
adversarial training procedure.

#### Our Experimental Changes

In our paper, we do not need to train our system end-to-end. Instead, we
segregate the procedure into two key components. First, we estimate user
preferences over an attribute space using a Bayesian approach. Second, we
outline a general procedure for modifying latent vectors so that they generate
images that match those preferences. Our Bayesian preference estimation
procedure serves a very similar function to the process network in [12]. The
only key difference is that the process network in [12] produces a vector $s$
which is fed directly into the write network (generator). This requires the
LSTM to both model a user’s preferences given a set of constraints, and have
an understanding of the structure of the generator latent space. Our
preference estimation procedure is agnostic to any given latent representation
and simply needs to find a vector in a low-dimensional (compared to a GAN
latent space) attribute space that satisfies a set of constraints. For the
sake of comparison, we modify the LSTM framework so that it simply needs to
produce a low-dimensional attribute vector given a set of constraints. We can
then plug this into our attribute mapping functions $f(z_{i},a)$ to produce
latent vectors that satisfy constraints. This allows us to directly compare
the performance of our Bayesian preference estimation approach and the LSTM
approach.

Specific to this architecture, the only thing we change is to feed a zero
vector as input into the LSTM instead of a latent vector $z$, while still
giving the LSTM a set of constraints. We then treat the final output of the
LSTM as an attribute vector $a$.

Figure 10: PrefGen with CLIP. Here we show two example trials of PrefGen in a
2D space formed by the textual attributes “red hair” and “anger”.

##  Appendix C Further Results

###  C.1 Supervised Mapping: Face Pose and Age Preference Estimation

#### Qualitative: Face Pose and Age Preference Estimation

In Figure 5 we showed qualitative examples of two trial runs of our preference
estimation procedure. For each, we showed the queries that were presented to
the oracle, the initial image, target image, and samples of how our estimate
of the oracle’s preferences changed over time. Because of a lack of space, we
could not include more examples, but here we show several more. This is shown
in Figure 11.

Figure 11: More Pose and Age Preference Estimation Examples. Here are more
examples of the preference estimation procedure similar to Figure 5. We
demonstrate 6 trials of PrefGen in a 4D space formed by yaw, pitch, roll, and
age of a human face. We show each query that is presented to the oracle, and
the images generated by sampling from the posterior after each query. Sample
images tend to qualitatively converge on the target.

#### Quantitative: Face Pose and Age Preference Estimation

In the core paper, we showed various quantitative results which demonstrated
the effectiveness of our approach at estimating user preferences in a 2D
attribute space formed by the left-right orientation of a face and the age of
a face. However, it is important that our method generalizes to spaces with
more attributes. Here we show parallel results in a 4D space formed by face
age, yaw, pitch, and roll. These results are shown in Figure 9.

###  C.2 CLIP Based Mapping

We used CLIP to infer directions that correspond to different relative textual
attributes. In Figure 1 we showed a trial of PrefGen using an encoded image of
Tom Cruise. In Figure 10 we show two more example trials of the same features
(i.e. “red hair” and “anger”) with start images generated with random latent
vectors. It is also possible to manipulate different types of relative textual
attributes. Figure 12 shows the manipulation of several more relative
attributes.

Figure 12: Modeling Relative Attributes with CLIP. It is possible to transform
certain relative attributes of a face by inferring directions in CLIP space
corresponding to those attributes. We can then use a gradient-based approach
on the latent vector of an image until it produces images that embed to CLIP
vectors with a certain intensity along this inferred relative attribute
direction.