# Risk aversion and uniqueness of equilibrium: a polynomial approach

Andrea Loi  Andrea Loi, Dipartimento di Matematica e Informatica
Università di Cagliari, Italy<EMAIL_ADDRESS>and  Stefano Matta  Stefano
Matta, Dipartimento di Scienze economiche e Aziendali
Università di Cagliari, Italy<EMAIL_ADDRESS>
###### Abstract.

We study the connection between risk aversion, number of consumers and
uniqueness of equilibrium. We consider an economy with two goods and $c$
impatience types, where each type has additive separable preferences with HARA
Bernoulli utility function,
$u_{H}(x):=\frac{\gamma}{1-\gamma}\left(b+\frac{a}{\gamma}x\right)^{1-\gamma}$.
We show that if $\gamma\in\left(1,\frac{c}{c-1}\right]$, the equilibrium is
unique. Moreover, the methods used, involving Newton’s symmetric polynomials
and Descartes’ rule of signs, enable us to offer new sufficient conditions for
uniqueness in a closed-form expression highlighting the role played by
endowments, patience and specific HARA parameters. Finally, new necessary and
sufficient conditions in ensuring uniqueness are derived for the particular
case of CRRA Bernoulli utility functions with $\gamma=3$.

The first author was supported by INdAM. GNSAGA - Gruppo Nazionale per le
Strutture Algebriche, Geometriche e le loro Applicazioni. Both authors were
supported by STAGE - Funded by Fondazione di Sardegna.

Keywords: Uniqueness of equilibrium; Excess demand function; Risk aversion;
Polynomial approximation; Descartes’ rule of signs; Netwon’s symmetric
polynomials.

JEL Classification: C62, D51, D58.

## Acknowledgement

We would like to acknowledge Alexis Akira Toda for useful comments and
critical remarks.

##  1\. Introduction and statements of the main results

Uniqueness plays a crucial role in comparative statics and stability. Yet, as
highlighted by [7], it is rarely possibile to provide easy analytical
conditions that guarantee uniqueness in applied models. In a recent paper,
Geanakoplos and Walsh [3] have presented new sufficient conditions ensuring
uniqueness and stability of equilibrium in an economy with two goods, where
$c$ agents, $c\geqslant 2$, ordered according to a parameter
${\underline{\beta}}=(\beta_{1},\ldots,\beta_{c}$),
$\beta_{1}<\cdots<\beta_{c}$, representing patience, have identical endowments
and the same Bernoulli utility function displaying non-increasing absolute
risk aversion (DARA). The role played in that paper by the DARA assumption is
twofold: it ensures that income effects can be ordered across types and
determines a positive covariance between consumption and income effect, hence
bounding the market income effect.

For our purposes, their main results (see Proposition 2 and Proposition 5 in
[3] for details) can be summarized as follows.

######  Theorem GW.

Let $u$ be a Bernoulli utility function and let impatience type $i$’s
preferences be represented by

$u_{i}(x,y)=u(x)+\beta_{i}u(y),\ i=1,\dots,c.$

Denote by $(e_{i},f_{i})$ consumer $i$’s endowments of goods $x$ and $y$,
respectively.

The price is unique if:

  1. (1)

$u$ satisfies DARA and agents have identical endowments, i.e.
$(e_{i},f_{i})=(e_{j},f_{j})$ for all $i$ and $j$;

  2. (2)

$u$ satisfies CRRA and the following restrictions on patience and endowments
hold: $e_{i}\leqslant e_{j}$ and $f_{i}\geqslant f_{j}$, for any $i<j$.

They point out that the assumption of identical endowments, although used in
several papers, is rather restrictive and highlight that, under DARA
preferences, there is no evident condition which ensures uniqueness if the
assumption of identical endowments is dropped. They conjecture that there
shouldn’t be “too much heterogeneity” across agents to ensure uniqueness,
arguing that heterogeneity should involve a condition on the patience
parameter ${\underline{\beta}}$, endowments and the particular Bernoulli
utility function used.

In this paper we are mainly interested in exploring sufficient conditions on
the parameter $\gamma$ that guarantees uniqueness of equilibrium without
imposing any restriction on endowments, unlike Theorem GW. Moreover, the
methods used enable us to address the above issue of heterogeneity raised by
[3] and to obtain necessary and sufficient conditions for CRRA preferences.

More precisely, we consider an economy with two goods and $c$ impatience
types, where type $i$ has preferences represented by the utility function

$u_{i}(x,y)=u_{H}(x)+\beta_{i}u_{H}(y),$ (1)

where $u_{H}$ is HARA111We will not consider the well-known case $\gamma=1$,
where the Bernoulli utility is logarithmic., i.e.

$u_{H}(x):=\frac{\gamma}{1-\gamma}\left(b+\frac{a}{\gamma}x\right)^{1-\gamma},\
\gamma>0,\gamma\neq 1,a>0,b\geqslant 0.$ (2)

Notice that HARA is an important subclass of DARA preferences extensively used
in the literature, which also encompasses the CRRA case by setting $b=0$.

Our main result is the following theorem which establishes a connection
between uniqueness, $\gamma$ and the number $c$ of impatience types in the
economy.

######  Theorem 1.

Let $u_{H}$ be the Bernoulli utility function (2) and let utility of
impatience type $i$ be given by (1). If

$\gamma\in\left(1,\frac{c}{c-1}\right],$ (3)

then the economy has a unique equilibrium.

This result is also made interesting by the fact that “utility function
concavity parameters in the range of $1$ to $2$ are widely considered
plausible in the literature”, as Kaplow observes (see [6, Chapter 3] and
references therein).

For CRRA preferences this links the coefficient of relative risk aversion
$\gamma$ to the number of consumers and also answers the issue raised by [13],
i.e. whether a value of $\gamma$ in the interval $(1,2]$ is compatible with
multiple equilibria for CRRA preferences with $2$ consumers. Moreover, it
provides a generalization for any number of consumers and arbitrary endowments
allocations.

In fact, it is known (see [11] and references therein; see also [5] and [12])
that for $C^{2}$ separable preferences $\sum_{l}u_{l}(x)$, where each $u_{l}$
is a monotonic and concave, relative risk aversion less or equal to $1$
implies uniqueness. In particular, under CRRA preferences, i.e. when $b=0$, if
$\gamma$ belongs to $(0,1)$, we have uniqueness. Toda and Walsh [13] show for
CRRA preferences the existence of multiple equilibria in an economy with two
goods and two consumers when $\gamma>2$. Whether multiplicity is possibile or
not for $1<\gamma\leqslant 2$ was left an open question. An immediate
consequence of Theorem 1, see Corollary 7, rules out this possibility.
Moreover, it highlights the interplay between relative risk aversion and the
number of consumers. An equivalent result also holds for HARA preferences (see
Corollary 6).

As far as the heterogeneity issue raised by [3] is concerned, our contribution
is the following theorem.

######  Theorem 2.

In the HARA case with two impatience types and for $\gamma$ sufficiently close
to $3$, we have at most three equilibria. Moreover, if we assume that

$\beta_{1}<\beta_{2},\ e_{1}\leqslant e_{2},\ f_{1}\geqslant f_{2},$ (4)

and

$b\geqslant\frac{a}{3}\left(\frac{\beta_{2}}{\beta_{1}}\right)^{\frac{2}{3}}\left(e_{2}+f_{1}\right)$
(5)

are satisfied, then the equilibrium is unique.

Condition (5) confirms what has been claimed by [3]. It represents a closed-
form expression that embodies what [3] argue about for arbitrary DARA
preferences, highlighting the role played by endowments, patience, and
specific Bernoulli utility parameters in ensuring uniqueness. In particular,
the parameters involved $a,b,\gamma$ affect risk tolerance, i.e. the inverse
of absolute risk aversion. Following [3]’s insight, this closed-form
expression ensures the effect of positive covariance across types. Moreover,
it allows for more heterogeneity in allocations than might be expected.

Since we believe that the approach used is as interesting as the results, we
provide an intuition here, leaving the details in Section 2. As usual, from
the maximization problem, one obtains the excess demand function $Z$,
expressed as an implicit function of the price raised to a positive function
depending on the parameter $\gamma$. The strategy is to turn $Z$, by algebraic
manipulations, into a polynomial $P$. Since $\gamma$ can be a real number, we
use the density of $\mathbb{Q}$ in $\mathbb{R}$ to approximate $\gamma$. This
strategy “generically” works for regular values for a transversality argument.
Finally, we write $P$ in terms of Newton’s symmetric polynomials and apply
Descartes’ rule of signs, which states that the number of positive roots of a
real polynomial, arranged in ascending or descending powers, cannot exceed the
number of sign variations in consecutive coefficients (see, e.g., [1]).

It is remarkable how this simple method can be very useful and powerful. In
fact, it allows to study the number of equilibria without ad hoc restrictions
on the set of parameters and, moreover, it can provide sufficient and
necessary conditions for uniqueness as we show in Theorem 10, which
establishes a connection between the number of equilibria, their type (regular
and critical), endowments and utility weights, thus complementing [13]’s
analysis of a CRRA economy with relative risk aversion $\gamma=3$.

The literature on uniqueness is vast. For a survey, the reader is primarily
referred to [7, 11], and references therein. As we have already pointed out, a
feature of Theorem 1 is that we do not impose restrictions on endowments. From
this point of view, it is related to the strand of the literature which
provides sufficient conditions that guarantee uniqueness globally, i.e. for
every possible allocation of resources among consumers. Two recent papers that
belong to this line of research are [9, 10], where uniqueness is globally
characterized through geometric properties of the equilibrium manifold. For a
different approach, which provides sufficient conditions on offer curves in a
two-commodity, two-agent exchange economy, see [4]. We finally refer the
reader to [8], and references therein, for a survey on how to solve economic
equilibria described as solutions to systems of polynomial equations.

We believe that the uniqueness issue, and the methods used in this paper, can
deserve further attention and research. In particular, we think about
extending Theorem 2 to an arbitrary number of consumers without imposing
restrictions on $\gamma$. Moreover, it could be interesting to apply these
methods to different Bernoulli utility functions, in order to achieve new
sufficient conditions on uniqueness.

This paper is organized as follows. In Section 2 we describe the economic
setting for HARA preferences, the methods used and we prove Theorem 1. Section
3 applies our approach to the particular case of $c=2$ consumer types
(Corollary 6 and 7) and provides new sufficient (Theorem 2) and necessary and
sufficient conditions for uniqueness (Theorem 10), for HARA and CRRA
preferences, respectively.

##  2\. HARA Preferences

We consider a pure exchange economy with two goods and an arbitrary (finite)
number ($c$) of impatience types, where type $i$ has preferences represented
by (1).

Consumer $i$’s endowments is denoted by $(e_{i},f_{i})$ . We have
$\sum_{i=1}^{c}e_{i}=r_{x}(c)$ and $\sum_{i=1}^{c}f_{i}=r_{y}(c)$, where
$(r_{x}(c),r_{y}(c))$ is the total resources vector.

Under the budget constraint $pe_{i}+f_{i}\leqslant px+y$, consumer $i$’s
maximization problem, for $i=1,\ldots,c$, leads to the aggregate excess demand
function for good $x$:

$Z\left(\underline{e},\underline{f},p,\varepsilon,\underline{\sigma},a,b,c\right):=\sum_{i=1}^{c}\frac{b-bp^{\varepsilon}\sigma_{i}+a\varepsilon\left(pe_{i}+f_{i}\right)}{a\varepsilon\left(p+\sigma_{i}p^{\varepsilon}\right)}-r_{x}(c),$
(6)

where we set

$\varepsilon:=\frac{1}{\gamma},\ \sigma_{i}:=\beta_{i}^{\varepsilon},\
i=1,\dots,c$

and we denote

$\underline{e}=\left(e_{1},\dots,e_{c}\right),\underline{f}=\left(f_{1},\dots,f_{c}\right),\underline{\sigma}=\left(\sigma_{1},\dots,\sigma_{c}\right).$

In order to make algebraic manipulations of expression (6) easier (see
Proposition 3 below), we will exploit the presence of symmetric polynomials
within (6) via Newton’s identities. The interested reader may refer to [2],
although this paper is self-contained.

For each integer $t$ such that $1\leqslant t\leqslant c$ and $i=1,\dots,c$ set

$\mathfrak{s}\left(t,c\right):=\sum_{i_{1}<\cdots<i_{t}}^{c}\sigma_{i_{1}\dots
i_{t}},\ i_{l}=1,\dots,c,\ l=1,\dots,t,$ (7)

where

$\sigma_{i_{1}\dots i_{t}}=\sigma_{i_{1}}\cdots\sigma_{i_{t}}.$

Moreover, set

$\mathfrak{s}(c,c,j)=0$ (8)

and for $1\leqslant t<c$

$\mathfrak{s}(t,c,i):=\sum_{i_{1}(i)<\cdots<i_{t}(i)}^{c}\sigma_{i_{1}(i)\dots
i_{t}(i)},\ i_{l}(i)=1,\dots\hat{i},\dots c,\ l=1,\dots,t,$ (9)

where $\hat{i}$ means that the index $i$ is omitted.

Further, set

$\mathfrak{s}(0,c)=\mathfrak{s}(0,c,i)=1$ (10)

and

$\mathfrak{s}(c,c-1)=\mathfrak{s}(c,c-1,i)=0.$ (11)

Then it is not hard to see that the following equalities hold true:

$\mathfrak{s}(t,c)=\mathfrak{s}(t,c-1)+\sigma_{c}\mathfrak{s}(t-1,c-1),$ (12)
$\mathfrak{s}(t,c,i)=\mathfrak{s}(t,c-1,i)+\sigma_{c}\mathfrak{s}(t-1,c-1,i),$
(13) $\mathfrak{s}(t,c,c)=\mathfrak{s}(t,c-1),$ (14)

for all $c\geqslant 2$, $1\leqslant t\leqslant c$ and $i=1,\dots,c$.

######  Proposition 3.

The zeros set of the aggregate excess demand function (6) equals that of the
following function:

$\begin{split}z(\underline{e},\underline{f},p,\varepsilon,\underline{\sigma},a,b,c):=-\sigma_{1\dots
c}\left(r_{x}(c)+\frac{cb}{a\varepsilon}\right)p^{c(\varepsilon-1)+1}+\\\
-\sum_{t=1}^{c-1}\left[\xi(\underline{e},\underline{\sigma},c,t)+u(\underline{\sigma},b,c,t)\right]p^{t(\varepsilon-1)+1}\\\
+\sum_{t=1}^{c-1}v(\underline{f},\underline{\sigma},b,c,t)p^{t(\varepsilon-1)}+r_{y}(c)+\frac{cb}{a\varepsilon}\end{split}$
(15)

where

$\xi(\underline{e},\underline{\sigma},c,t):=\left[r_{x}(c)\mathfrak{s}(t,c)-\sum_{i=1}^{c}e_{i}\mathfrak{s}(t,c,i)\right],$
(16)
$u(\underline{\sigma},a,b,c,t):=\frac{b}{a\varepsilon}\sum_{i=1}^{c}\sigma_{i}\mathfrak{s}(t-1,c,i),$
(17)
$v(\underline{f},\underline{\sigma},a,b,c,t):=\sum_{i=1}^{c}\left(f_{i}+\frac{b}{a\varepsilon}\right)\mathfrak{s}(t,c,i).$
(18)

[Proof]At an equilibrium price $p$ the function (6)

$-pr_{x}(c)+\sum_{i=1}^{c}\frac{pe_{i}+f_{i}+\frac{b}{a\varepsilon}-\frac{b}{a\varepsilon}\sigma_{i}p^{\varepsilon}}{1+\sigma_{i}p^{\varepsilon-1}},$

or equivalently

$-pr_{x}(c)\prod_{i=1}^{c}\left(1+\sigma_{i}p^{\varepsilon-1}\right)+\sum_{i=1}^{c}\left[\left(pe_{i}+f_{i}+\frac{b}{a\varepsilon}-\frac{b}{a\varepsilon}\sigma_{i}p^{\varepsilon}\right)\prod_{\hat{i}=1}^{c}\left(1+\sigma_{\hat{i}}p^{\varepsilon-1}\right)\right],$

where $\hat{i}$ means that the index $i$ is omitted, vanishes.

By Equations (7) and (9), we can write the products above as follows

$\prod_{i=1}^{c}(1+\sigma_{i}p^{\varepsilon-1})=1+\sum_{t=1}^{c-1}\mathfrak{s}(t,c)p^{t(\varepsilon-1)}+\sigma_{1\ldots
c}p^{c(\varepsilon-1)}$

$\prod_{\hat{i}=1}^{c}(1+\sigma_{\hat{i}}p^{\varepsilon-1})=1+\sum_{t=1}^{c-1}\mathfrak{s}(t,c,i)p^{t(\varepsilon-1)}$

and rewrite the expression accordingly:

$-pr_{x}(c)\left[1+\sum_{t=1}^{c-1}\mathfrak{s}(t,c)p^{t(\varepsilon-1)}+\sigma_{1\ldots
c}p^{c(\varepsilon-1)}\right]+\sum_{i=1}^{c}\left(pe_{i}+f_{i}+\frac{b}{a\varepsilon}-\frac{b}{a\varepsilon}\sigma_{i}p^{\varepsilon}\right)\left[1+\sum_{t=1}^{c-1}\mathfrak{s}(t,c,i)p^{t(\varepsilon-1)}\right].$

By expanding and rearranging, we immediately get

$\displaystyle-r_{x}(c)\sigma_{1\ldots
c}p^{c(\varepsilon-1)+1}-\sum_{t=1}^{c-1}\left[r_{x}(c)\mathfrak{s}(t,c)-\sum_{i=1}^{c}e_{i}\mathfrak{s}(t,c,i)\right]p^{t(\varepsilon-1)+1}+\sum_{t=1}^{c-1}\sum_{i=1}^{c}(f_{i}+\frac{b}{a\varepsilon})\mathfrak{s}(t,c,i)p^{t(\varepsilon-1)}$
$\displaystyle-\frac{b}{a\varepsilon}(\sum_{i=1}^{c}\sigma_{i})p^{\varepsilon}-\frac{b}{a\varepsilon}\sum_{t=1}^{c-1}\sum_{i=1}^{c}\sigma_{i}\mathfrak{s}(t,c,i)p^{t(\varepsilon-1)+\varepsilon}+r_{y}(c)+\frac{cb}{a\varepsilon}.$

Notice now that by the change of index $u:=t+1$, one gets

$\displaystyle-\frac{b}{a\varepsilon}\sum_{t=1}^{c-1}\sum_{i=1}^{c}\sigma_{i}\mathfrak{s}(t,c,i)p^{t(\varepsilon-1)+\varepsilon}=-\frac{b}{a\varepsilon}\sum_{t=1}^{c-1}\sum_{i=1}^{c}\sigma_{i}\mathfrak{s}(t,c,i)p^{(t+1)(\varepsilon-1)+1}$
$\displaystyle=-\frac{b}{a\varepsilon}\sum_{u=2}^{c}\sum_{i=1}^{c}\sigma_{i}\mathfrak{s}(u-1,c,i)p^{u(\varepsilon-1)+1}=\frac{b}{a\varepsilon}\sum_{i=1}^{c}\sigma_{i}\mathfrak{s}(0,c,i)p^{(\varepsilon-1)+1}$
$\displaystyle-\frac{b}{a\varepsilon}\sum_{t=1}^{c-1}\sum_{i=1}^{c}\sigma_{i}\mathfrak{s}(t-1,c,i)p^{t(\varepsilon-1)+1}-\frac{b}{a\varepsilon}\sum_{i=1}^{c}\sigma_{i}\mathfrak{s}(c-1,c,i)p^{c(\varepsilon-1)+1}$
$\displaystyle=\frac{b}{a\varepsilon}(\sum_{i=1}^{c}\sigma_{i})p^{\varepsilon}-\frac{b}{a\varepsilon}\sum_{t=1}^{c-1}\sum_{i=1}^{c}\sigma_{i}\mathfrak{s}(t-1,c,i)p^{t(\varepsilon-1)+1}-\frac{cb}{a\varepsilon}\sigma_{1\ldots
c}p^{c(\varepsilon-1)+1},$

where in the last equality we use (10) and
$\sum_{i=1}^{c}\sigma_{i}\mathfrak{s}(c-1,c,i)=c\sigma_{1\ldots c}$, where
$\sigma_{1\ldots c}=\sigma_{1}\cdots\sigma_{c}$.

By inserting this last equality into the previous expression one gets

$\displaystyle-\sigma_{1\dots
c}\left(r_{x}(c)+\frac{cb}{a\varepsilon}\right)p^{c(\varepsilon-1)+1}-\sum_{t=1}^{c-1}\left[r_{x}(c)\mathfrak{s}(t,c)-\sum_{i=1}^{c}e_{i}\mathfrak{s}(t,c,i)+\frac{b}{a\varepsilon}\sum_{i=1}^{c}\sigma_{i}\mathfrak{s}(t-1,c,i)\right]p^{t(\varepsilon-1)+1}$
$\displaystyle+\sum_{t=1}^{c-1}\sum_{i=1}^{c}\left(f_{i}+\frac{b}{a\varepsilon}\right)\mathfrak{s}(t,c,i)p^{t(\varepsilon-1)}+r_{y}(c)+\frac{cb}{a\varepsilon},$

and the proposition follows.

The following two lemmata are needed to prove Theorem 1.

######  Lemma 4.

Set

$F(t,c):=r_{x}(c)\mathfrak{s}(t,c)-\sum_{i=1}^{c}e_{i}\mathfrak{s}(t,c,i).$
(19)

Then $F(t,c)>0$ for each integer $t$ such that $1\leqslant t\leqslant c-1$.

[Proof]We work by induction on $c\geqslant 2$ for all $t$ such that
$1\leqslant t\leqslant c-1$. The base on the induction is immediate:

$\displaystyle F(1,2)$
$\displaystyle=(e_{1}+e_{2})\sigma(1,2)-e_{1}\sigma(1,2,1)-e_{2}\sigma(1,2,2)$
$\displaystyle=(e_{1}+e_{2})(\sigma_{1}+\sigma_{2})-e_{1}\sigma_{2}-e_{2}\sigma_{1}$
$\displaystyle=e_{1}\sigma_{1}+e_{2}\sigma_{2}>0$ $\displaystyle F(2,2)$
$\displaystyle=(e_{1}+e_{2})\sigma(2,2)-e_{1}\sigma(2,2,1)-e_{2}\sigma(2,2,2)=(e_{1}+e_{2})\sigma_{1}\sigma_{2}>0.$

Assume now, by the induction assumption, that

$F(t,c-1)>0$

for each integer $1\leqslant t\leqslant c-2$. By (12), (13) and (14) equation
(19) reads as

$F(t,c)=r_{x}(c-1)\mathfrak{s}(t,c-1)+\sigma_{c}r_{x}(c)\mathfrak{s}(t-1,c-1)-\sum_{j=1}^{c-1}e_{j}\left[\mathfrak{s}(t,c-1,j)+\sigma_{c}\mathfrak{s}(t-1,c-1,j)\right],$

which can be rewritten as

$F(t,c)=F(t,c-1)+\sigma_{c}F(t-1,c-1)+\sigma_{c}e_{c}\mathfrak{s}(t-1,c-1),$

which is strictly positive by the induction assumption.

The next lemma can only be applied to regular equilibra, being robust to
sufficiently small perturbations. Let us denote by
$N(\underline{e},\underline{f},\varepsilon,\underline{\sigma},a,b,c)$ the
(finite) cardinality of the set of regular equilibria of the aggregate excess
demand (6), or equivalently (15), in the Hara case.

######  Lemma 5.

For every $\varepsilon_{0}\in(0,1)$, there exist two natural numbers $m,\,n$,
$0<m<n$, where $\frac{m}{n}$ is sufficiently close to $\varepsilon_{0}$, such
that

$N(\underline{e},\underline{f},\varepsilon_{0},\underline{\sigma},a,b,c)\leqslant
N(\underline{e},\underline{f},\frac{m}{n},\underline{\sigma},a,b,c),$

for all $\underline{e},\underline{f},\underline{\sigma},a,b,c$.

[Proof]

For fixed $(\underline{e},\underline{f},\underline{\sigma},a,b,c)$, the
aggregate excess demand function (15) continuously depends on the price $p$
and the parameter $\varepsilon$. Notice that if $p_{0}$ is a regular
equilibrium of the function of one variable
$z\left(\underline{e},\underline{f},p,\varepsilon_{0},\underline{\sigma},a,b,c\right)$,
then its graph transversally intersects the $p$ axis in the $p-z$ plane. The
same property holds for a small perturbation of $\varepsilon_{0}$. Hence the
result follows by the density of $\mathbb{Q}$ in $\mathbb{R}$.

This lemma points out that the number of regular equilibra cannot decrease
after the perturbation. The following figure provides an insight into the
previous lemma. The black curve represents the aggregate excess demand curve
for a given $\varepsilon$. The perturbation induced in the original curve by
replacing $\varepsilon$ with $\frac{m}{n}$ is represented by the red curve. In
this case the number of regular equilibria is such that

$3=N(\underline{e},\underline{f},\varepsilon,\underline{\sigma},a,b,c)\leqslant
N(\underline{e},\underline{f},\frac{m}{n},\underline{\sigma},a,b,c)=5.$

Observe that if the perturbation had “opposite direction”, the number of
regular equilibria would remain unchanged.

$p$$z(p)$ Figure 1. Aggregate excess demand after replacing $\varepsilon$ with
$\frac{m}{n}$.

We are now in position to prove Theorem 1.

[Proof of Theorem 1] By inserting $\varepsilon=\frac{m}{n}$, rational number,
as in Lemma 5 into (15) and denoting $q=p^{\frac{1}{n}}$, we deduce that

$\begin{split}z(\underline{e},\underline{f},q,\frac{m}{n},\underline{\sigma},a,b,c)&:=-\sigma_{1\dots
c}(r_{x}(c)+\frac{cb}{a\varepsilon})q^{c(m-n)+n}\\\
&-\sum_{t=1}^{c-1}\left[\xi(\underline{e},\underline{\sigma},c,t)+u(\underline{\sigma},a,b,c,t)\right]q^{t(m-n)+n}\\\
&+\sum_{t=1}^{c-1}v(\underline{f},\underline{\sigma},a,b,c,t)q^{t(m-n)}+r_{y}+\frac{cb}{a\varepsilon}\end{split}$
(20)

We claim that for $\frac{m}{n}\in\left[\frac{c-1}{c},1\right)$ and for
$\underline{e},\underline{f},\underline{\sigma},a,b,c$ arbitrarly chosen,
there exists a unique $q_{0}>0$ which is a zero of the function

$z(\underline{e},\underline{f},q,\frac{m}{n},\underline{\sigma},a,b,c),$ (21)

i.e.
$z(\underline{e},\underline{f},q_{0},\frac{m}{n},\underline{\sigma},a,b,c)=0$.
Then, by applying Lemma 5, we would deduce that $p_{0}=q_{0}^{n}$ is the only
positive solution of (15) for $\varepsilon\in[\frac{c-1}{c},1)$ , i.e. we have
unicity of price equilibrium in the HARA economy under the assumption (3),
thus arriving at a conclusion of the proof of the theorem.

In order to prove the claim, fix
$\underline{e},\underline{f},\underline{\sigma},a,b,c$ and set

$\mu_{c}:=\sigma_{1\dots c}(r_{x}(c)+\frac{cb}{a\varepsilon}),\
\mu_{t}:=\xi_{t}+u_{t},\ v_{t},\ v_{0}=r_{y}+\frac{cb}{a\varepsilon},$

where for $t=1,\dots c-1$ we define

$\xi_{t}:=\xi(\underline{e},\underline{\sigma},c,t),\
u_{t}:=u(\underline{\sigma},a,b,c,t),\
v_{t}:=v(\underline{f},\underline{\sigma},a,b,c,t).$

By multiplying equation (21) by the monomial $q^{(c-1)(n-m)}$ ($n>m$) and
using (20), one sees that there exists a positive zero of (21) if and only if
the following polynomial in the variable $q$ has a unique positive root:

$\begin{split}P(q)&:=v_{c-1}+v_{c-2}q^{n-m}+\dots+v_{1}q^{(c-2)(n-m)}+v_{0}q^{(c-1)(n-m)}\\\
&-\mu_{c}q^{m}-\mu_{c-1}q^{n}-\dots-\mu_{1}q^{(c-2)(n-m)+n}\end{split}$ (22)

Notice now that by the very definition of the symbols involved, $\mu_{c}$,
$\nu_{0}$, $u_{t}$ and $v_{t}$ are strictly positive real numbers for all
$t=1,\dots,c-1$. By applying Lemma 4, we also deduce that $\xi_{t}$ is
strictly positive and hence $\mu_{t}>0$ for all $t=1,\dots,c-1$. Since the
assumption $\frac{m}{n}\in[\frac{c-1}{c},1)$ is equivalent to
$(c-1)(n-m)\leqslant m$, it follows that the monomials appearing in $P(q)$ are
written in increasing order. Thus, by Descartes’ rule of sign, the polynomial
$P(q)$ has a unique positive root and the theorem follows.

##  3\. The case of two consumers

In this section we the study the number of equilibria in the case of $c=2$
consumers. Two immediate implications of Theorem 1 are the following
corollaries.

######  Corollary 6.

In the HARA case with two goods, if $\gamma$ belongs to the interval $(1,2]$,
then the equilibrium price is unique.

######  Corollary 7.

In the CRRA case with two goods, if the relative risk aversion $\gamma$
belongs to the interval $(1,2]$, then the equilibrium price is unique.

This last corollary answers the question left open by Toda and Walsh [13,
Remark 1] of whether multiplicity is possibile or not for $\gamma\leqslant 2$
(see Subsection 3.1 below for a deeper analysis of Toda-Walsh example).

In the sequel, we are interested to analyze the number of equilibria in the
HARA case when $\gamma=3$.

By the proof of Theorem 1, we have reduced our investigation to the analysis
of the zeros of the polynomial (22) when $c=2$, $m=3$, $n=1$
($\varepsilon=\frac{1}{3}$). Namely, by (16), (17) and (18), one has

$P(q)=A(\underline{e},\underline{\sigma},a,b)q^{3}+B(\underline{f},\underline{\sigma},a,b)q^{2}+C(\underline{e},\underline{\sigma},a,b)q+D(\underline{f},\underline{\sigma},a,b),$
(23)

where

$\displaystyle A(\underline{e},\underline{\sigma},a,b):=$
$\displaystyle-\mu_{1}=-(e_{1}\sigma_{1}+e_{2}\sigma_{2})-\frac{3b}{a}(\sigma_{1}+\sigma_{2}),$
(24) $\displaystyle B(\underline{f},\underline{\sigma},a,b):=$ $\displaystyle
v_{0}=f_{1}+f_{2}+\frac{6b}{a},$ $\displaystyle
C(\underline{e},\underline{\sigma},a,b):=$
$\displaystyle-\mu_{2}=-(e_{1}+e_{2})\sigma_{1}\sigma_{2}-\frac{6b}{a}\sigma_{1}\sigma_{2},$
$\displaystyle D(\underline{f},\underline{\sigma},a,b):=$ $\displaystyle
v_{1}=f_{1}\sigma_{2}+f_{2}\sigma_{1}+\frac{3b}{a}(\sigma_{1}+\sigma_{2}).$

In order to prove Theorem 2, we need a simple algebraic lemma.

######  Lemma 8.

Assume that the polynomial $P(x)=Ax^{3}+Bx^{2}+Cx+D$ has three sign changes
and $ABCD\neq 0$. If

$AD-BC<0,$ (25)

then $P(x)$ has a unique positive root.

[Proof]The discriminant of $P(x)$ is given by:

$\Delta=B^{2}C^{2}-4AC^{3}-4B^{3}D-27A^{2}D^{2}+18ABCD.$ (26)

The assumption (25) gives $A^{2}D^{2}>B^{2}C^{2}$ and $A^{2}D^{2}>ABCD$. Then
$\Delta<0$. Hence the polynomial has a unique root which is positive by the
Decartes’ rule of signs (or by the mean value theorem).  [Proof of Theorem 2]
By Lemma 5, one can assume that $\gamma=\frac{1}{\varepsilon}=3$. The
boundedness of the number of equilibria is immediate, since the polynomial
(23) has degree three. In order to prove the second part of the theorem, by
Lemma 8 we need to verify that (4) and (5) imply

$A(\underline{e},\underline{\sigma},a,b)D(\underline{f},\underline{\sigma},a,b)-B(\underline{f},\underline{\sigma},a,b)C(\underline{e},\underline{\sigma},a,b)<0$

where
$A(\underline{e},\underline{\sigma},a,b),B(\underline{f},\underline{\sigma},a,b),C(\underline{e},\underline{\sigma},a,b),D(\underline{f},\underline{\sigma},a,b)$
are given by (24) above. A long but straightforward computation yields

$\begin{split}A(\underline{e},\underline{\sigma},a,b)D(\underline{f},\underline{\sigma},a,b)-B(\underline{f},\underline{\sigma},a,b)C(\underline{e},\underline{\sigma},a,b)&=\\\
&=(\sigma_{2}-\sigma_{1})(e_{1}f_{2}\sigma_{1}-e_{2}f_{1}\sigma_{2})\\\
&+E(\underline{e},\underline{f},\underline{\sigma},a,b),\end{split}$ (27)

where

$E(\underline{e},\underline{f},\underline{\sigma},a,b):=-\frac{9b^{2}}{a^{2}}\left(\sigma_{1}-\sigma_{2}\right)^{2}+\frac{3b}{a}\left[\left(e_{1}+e_{2}+f_{1}+f_{2}\right)\sigma_{1}\sigma_{2}-(e_{1}+f_{2})\sigma_{1}^{2}-(e_{2}+f_{1})\sigma_{2}^{2}\right].$

Notice that by (4), the first summand of (27), namely
$(\sigma_{2}-\sigma_{1})(e_{1}f_{2}\sigma_{1}-e_{2}f_{1}\sigma_{2})$, is
strictly less than zero:

$(\sigma_{2}-\sigma_{1})(e_{1}f_{2}\sigma_{1}-e_{2}f_{1}\sigma_{2})\leqslant(\sigma_{2}-\sigma_{1})f_{1}(e_{1}\sigma_{1}-e_{2}\sigma_{2})<(\sigma_{2}-\sigma_{1})f_{1}\sigma_{2}(e_{1}-e_{2})\leqslant
0.$

Moreover, $E(\underline{e},\underline{f},\underline{\sigma},a,b)\leqslant 0$
if and only if

$b\geqslant\frac{a}{3}\frac{\left[(e_{1}+e_{2}+f_{1}+f_{2})\sigma_{1}\sigma_{2}-(e_{1}+f_{2})\sigma_{1}^{2}-(e_{2}+f_{1})\sigma_{2}^{2}\right]}{\left(\sigma_{1}-\sigma_{2}\right)^{2}}.$

Again by (4) one can find an upper bound of the right hand side of this
inequality, namely:

$\displaystyle\frac{a}{3}\frac{\left[(e_{1}+e_{2}+f_{1}+f_{2})\sigma_{1}\sigma_{2}-(e_{1}+f_{2})\sigma_{1}^{2}-(e_{2}+f_{1})\sigma_{2}^{2}\right]}{\left(\sigma_{1}-\sigma_{2}\right)^{2}}$
$\displaystyle<\frac{a}{3}\frac{\left[(e_{1}+e_{2}+f_{1}+f_{2})\sigma_{1}\sigma_{2}\right]}{\left(\sigma_{1}-\sigma_{2}\right)^{2}}$
$\displaystyle<\frac{a}{3}(\frac{\sigma_{2}}{\sigma_{1}})^{2}(e_{2}+f_{1}).$

Thus, if (4) and
$b\geqslant\frac{a}{3}(\frac{\sigma_{2}}{\sigma_{1}})^{2}(e_{2}+f_{1})$ hold
true, we get that (27) is strictly less than zero and by Lemma 8 there is
uniqueness of equilibria. Hence the proof of the theorem follows, since
$\sigma_{i}=\beta_{i}^{\frac{1}{3}}$, $i=1,2$.

######  Remark 9.

Conditions (4) are exactly those of [3, Proposition 5], which holds for CRRA
(homotethic) preferences. In fact, assuming only conditions (4) and CRRA
preferences, one can follow the same argument of the proof of Theorem 2 and
achieve Proposition 5 in a different way. Hence Theorem 2 encompasses
Proposition 5 as a special case, when $\gamma=3$ (see Remark 11).222We thank
again Alexis Akira Toda for suggesting us the analysis of the general case,
arbitrary $\gamma$ with two consumers, which will be the subject of our future
research. Condition (5) enables us to extend the result to HARA preferences
and appraise the connection in ensuring uniqueness between endowments,
patience, and Bernoulli utility parameters affecting the concavity of the
Bernoulli utility function.

###  3.1. The Toda-Walsh example with CRRA preferences

In this last part of the paper, we consider the example of multiple
equilibria, provided by Toda and Walsh [13], of an economy with two goods and
two consumers under the assumption of CRRA preferences and symmetric
endowments. Observe that this CRRA example can be obtained from HARA (see (2))
by setting $b=0$ and

$\beta_{1}=(\frac{1-\alpha}{\alpha})^{\gamma},\
\beta_{2}=(\frac{\alpha}{1-\alpha})^{\gamma},\ 0<\alpha<1.$ (28)

More precisely, preferences are given by

$u_{1}(x_{1},x_{2})=\frac{1}{1-\gamma}(\alpha^{\gamma}x_{1}^{1-\gamma}+(1-\alpha)^{\gamma}x_{2}^{1-\gamma})$

$u_{2}(x_{1},x_{2})=\frac{1}{1-\gamma}((1-\alpha)^{\gamma}x_{1}^{1-\gamma}+\alpha^{\gamma}x_{2}^{1-\gamma})$

and consumers’ endowments are symmetric:

$(e_{1},f_{1})=(e,1-e)\ (e_{2},f_{2})=(1-e,e),\ 0<e<1.$ (29)

In this setting we obtain the following result.

######  Theorem 10.

In the CRRA symmetric case with $\gamma=3$ the following facts hold true:

  1. (1)

there is uniqueness of equilibria if and only if $(\alpha-3e)(2\alpha-1)>0$;

  2. (2)

there is critical equilibria if and only if $\alpha=3e$, $\alpha=\frac{1}{2}$;

  3. (3)

there are three equilibria if and only if $(\alpha-3e)(2\alpha-1)<0$.

[Proof]In order to study the equilibria prices, we have to analyze the roots
of (23) in this particular case. Substituting (28) and (29) into (24), we get

$P(q)=-\delta(\alpha,e)q^{3}+q^{2}-q+\delta(\alpha,e),$

where

$\delta(\alpha,e):=\frac{\alpha^{2}-(2\alpha-1)e}{\alpha-\alpha^{2}},\
0<\alpha,e<1.$

The discriminant of this polynomial (cfr. (26)) is

$\Delta=-(3\delta(\alpha,e)-1)^{3}(\delta(\alpha,e)+1).$

Thus the result follows by noticing that $\Delta\geqslant 0(<0)$ iff and only
if $\delta(\alpha,e)\leqslant\frac{1}{3}(>\frac{1}{3})$, i.e.
$(\alpha-3e)(2\alpha-1)\leqslant 0(>0)$.

Theorem 10 complements [13, Proposition 1] by providing necessary and
sufficient conditions for uniqueness in the CRRA economy with relative risk
aversion $\gamma=3$. It establishes a connection between the number of
equilibria, their type (regular and critical), endowments and utility weights.

We finally notice that for $\alpha=\frac{1}{7}$ and $e=\frac{1}{49}$, one has
$\delta(\alpha,e)=\frac{2}{7}$ and the polynomial $P(q)$ becomes
$-\frac{2}{7}q^{3}+q^{2}-q+\frac{2}{7}$ which has three solutions,
$\\{\frac{1}{2},1,2\\}$, which correspond, since $p=q^{3}$, to
$\\{\frac{1}{8},1,8\\}$ in accordance with [13]’s example.

######  Remark 11.

Theorem 10 has shown how this polynomial approach makes it easier to deal with
the particular case $\gamma=3$. We believe that this method deserves future
investigation since it can be used to tackle the general case of $c$ consumers
under CRRA and HARA preferences and arbitrary $\gamma$.

## References

  * Anderson et al [1998] Anderson, B., Jackson, J., and Sitharam, M., 1998, Descartes’ Rule of Signs Revisited, The American Mathematical Monthly, 105, 5, 447-451.
  * Edwards, H. M., [1984] Edwards, Harold M., 1984, Galois Theory. Springer-Verlag, Cambridge, New York.
  * Geanakoplos and Walsh [2018] Geanakoplos, J., Walsh, K. J., 2018, Uniqueness and stability of equilibrium in economies with two goods, Journal of Economic Theory, 174, 261-272.
  * Gimemez [2019] Gimémez, E. L., 2019, Offer curves and Uniqueness of competitive equilibrium, Ecobas Working Paper, http://ecobas.webs.uvigo.es/wk/2019-02_Gimenez_Offer-curves-and-uniqueness-of-competitive-equilibrium-ECOBAS.pdf
  * Hens and Loeffler [1995] Hens, T., Loeffler, A., 1995, Gross substitution in financial markets, Econ. Lett. 49(1), 39- 43.
  * Kaplow L., [2008] Kaplow, Louis, 2008, The Theory of Taxation and Public Economics. Princeton University Press.
  * Kehoe [1998] Kehoe, Timothy J., 1998, Uniqueness and stability. In: Kirman, Alan (Ed.), Elements of General Equilibrium Analysis. Wiley–Blackwell, pp. 38–87 (Chapter 3).
  * Kubler et al. [2014] Kubler, F., Renner P., and Schmedders, K., 2014, Computing All Solutions to Polynomial Equations in Economics. In: Schmedders, K. and Judd, K. L., (Eds), Handbook of Computational Economics, volume 3, pp. 599-652 (Chapter 11).
  * Loi and Matta [2018] Loi, A., Matta, S., 2018 Curvature and uniqueness of equilibrium, Journal of Mathematical Economics 74 , 62-67.
  * Loi and Matta [2021] Loi, A., Matta, S, Minimal entropy and uniqueness of price equilibria in a pure exchange economy, 2021, arXiv:2102.09827 [econ.TH].
  * Mas-Colell, Andreu, [1991] Mas-Colell, Andreu, 1991, On the uniqueness of equilibrium once again. In: Barnett, William A., Cornet, Bernard, D’Aspermont, Claude, Gabszewicz, Jean, Mas-Colell, Andreu (Eds.), Equilibrium Theory and Applications. Cambridge University Press, pp. 275–296 (Chapter 12).
  * Mas-Colell, A. et al., [1995] Mas-Colell, A., Whinston, Michael D., Green, J. R., 1995, Microeconomic Theory. Oxford University Press.
  * Toda and Walsh [2017] Toda, A. A., Walsh, K. J., 2017, Edgeworth box economies with multiple equilibria, Econ. Theory Bull., 5, 65-80.


# Language Grounded QFormer for Efficient Vision Language Understanding

Moulik Choraria∗,1 Nitesh Sekhar 2 Yue Wu2 Xu Zhang2 Prateek Singhal2 Lav R.
Varshney2 1: University of Illinois at Urbana Champaign, 2: Amazon*: The first
author performed the work while at Amazon, correspondence to
<EMAIL_ADDRESS>
###### Abstract

Large-scale pretraining and instruction tuning have been successful for
training general-purpose language models with broad competencies. However,
extending to general-purpose vision-language models is challenging due to the
distributional diversity in visual inputs. A recent line of work explores
vision-language instruction tuning, taking inspiration from the Query
Transformer (QFormer) approach proposed in BLIP-2 models for bridging frozen
modalities. However, these approaches rely heavily on large-scale multi-modal
pretraining for representation learning before eventual finetuning, incurring
a huge computational overhead, poor scaling, and limited accessibility. To
that end, we propose a more efficient method for QFormer-based vision-language
alignment and demonstrate the effectiveness of our strategy compared to
existing baselines in improving the efficiency of vision-language pretraining.

##  I Introduction

Vision-language models (VLMs) are a rapidly emerging topic of research due to
their ability to harness multi-modal inputs for simultaneous visual and
textual comprehension for diverse applications. Following the success of
general purpose large language models [1, 2], efforts are underway towards
analogous general purpose VLMs with few-shot learning capabilities. As such,
methods relying on instruction finetuning [3, 4] are appealing due to their
ability to exploit natural language understanding; one particular line of work
that combines it with multi-modal fusion of language and vision models has
received tremendous attention [5, 6]. Specifically, [5] relies on combining
frozen unimodal representations via a multi-stage pretrained QFormer, followed
by multi-task instruction finetuning, yielding state-of-the-art zero-shot
performance on a variety of tasks [6].

The success of [5] heavily relies on the large-scale two-stage pretraining on
the QFormer. However, despite the undeniable benefits, this pretraining
presents its own challenges. The sheer volume of data required for effective
pretraining requires significant computational resources, making it expensive,
time-consuming, and inaccessible to the larger research community [7, 8].
Additionally, curating diverse and unbiased multi-modal datasets requires
substantial efforts to ensure VLMs do not inadvertently propagate harmful
biases in the training data [9]. Hence, there is a pressing need to develop
new data- and compute-efficient strategies for training VLMs without the
expensive pretraining stage.

We seek to reduce the training burden for VLMs. Specifically, we seek to
improve the pipeline in [5, 6] by proposing an efficient alternative pipeline
for QFormer-based LLM conditioning for text generation. Under controlled
settings, we demonstrate the effectiveness of our approach in both speeding up
vision-language representation learning as well as improving performance, thus
paving the way for efficient pretraining with reduced data and compute burden
for general purpose vision-language understanding.

##  II Proposed Method

###  II-A Problem Formulation

In this paper, we focus on the problem of training a general-propose VLM, from
the lens of instruction tuning. Specifically, given an image $i$ and a text
instruction/prompt $p$, the model should output an appropriate response
corresponding to the inputs. Some example tasks include visual question
answering where $p$ is a question based on the image, or image captioning
where $p$ refers to the instruction of captioning the image. To exploit the
unimodal representations from large pretrained language and vision models, one
line of work explores bridging the two via a smaller model (like a QFormer for
BLIP models [5]) for effective vision-language learning. This approach,
augmented with instruction finetuning, can yield state-of-the-art multi-task
and zero shot abilities, as in InstructBLIP [6]. Going forward, we focus on
the aforementioned BLIP family of models.

###  II-B Revisiting InstructBLIP

InstructBLIP is one of the pioneering works on vision-language instruction
tuning, utilizing the QFormer approach as a trainable bridge between frozen
vision and language models. For ease of exposition, we consider LLMs with
encoder-decoder architectures [10, 2]. Consequently, the full InstructBLIP
framework can be interpreted as four different modules.
Image Encoder $\\{v(\cdot)\\}$: This module extracts visual features from
image $i$.
QFormer $\\{Q(\cdot,\cdot,\cdot)\\}$: The QFormer takes in a set of learnable
query tokens ($\bm{t}_{q}$), the prompt $p$, and the encoder vision features
$v(I)$ and outputs visually informed query tokens $\bm{t}_{qv}$.
LLM Encoder/Decoder $\\{l_{e}(\cdot,\cdot)$, $l_{d}(\cdot)\\}$: The encoder
takes the prompt $p$ and QFormer output tokens
$\bm{t}_{qv}=Q(\bm{t}_{v},v(i),p)$ as input and outputs a latent encoding,
which is then used by the decoder to generate the desired text output.

Concisely, the InstructBLIP output can be represented as:

$\omega_{insB}=l_{d}(l_{e}(p,Q(\bm{t}_{v},v(i),p)).$ (1)

Within the InstructBLIP framework, the image encoder $v(\cdot)$, the LLM
encoder $l_{e}(\cdot)$ and the LLM decoder $l_{d}(\cdot)$ are frozen during
training, with only the query tokens and the QFormer being trainable. Within
the QFormer, the query tokens are allowed to interact with the prompt text
tokens as well as each other via self-attention layers, and with the image
features via alternate cross-attention blocks. The goal is to output a set of
visually informed query tokens that can elicit an appropriate response from
the downstream LLM encoder and decoder.

The QFormer [5] requires two heavy pretraining stages: Stage 1 comprises
QFormer representation learning via image-text matching (ITM), image-text
contrastive learning (ITC), and image-to-QFormer text generation (ITG) using
the image encoder. Herein, the goal of the ITC and ITM is to align text
features and query tokens in the QFormer latent space, whereas ITG trains the
QFormer language modeling head for text generation by training the query
tokens to extract visual features. Then, in Stage 2, these QFormer
representations (“queries”) are used as inputs to the frozen LLM for end-to-
end image to text generation.

###  II-C Grounding the QFormer

Our motivation stems from noting that the QFormer output tokens essentially
aim to model a text distribution, as can be seen from the fact that firstly,
it is trained to minimize language modeling ITG loss and secondly, is used to
prompt a frozen LLM that expects text inputs. It is accepted wisdom in machine
learning that data often lies on a lower-dimensional manifold, embedded in a
higher-dimension input space [11]. Therefore, it is often more efficient to
model distributions in a lower-dimensional space [12, 13].

This observation is key to our method, that is, instead of directly prompting
the LLM, we enforce the QFormer output tokens to model a visually informed
encoder latent representation, which is then fed to the LLM decoder for text
image-to-text generation. To aid this process, drawing inspiration from
classical notions of language grounding in cognitive science [14, 15], we then
propose the idea of language/LLM grounding by augmenting the QFormer learning
with prompts from the LLM encoder, which act as a reference for modeling this
latent representation efficiently. More precisely, using previous notation and
$[\cdot,\cdot]$ to denote the concatenation operator, the output of our
framework with the grounded QFormer can be expressed as:

$\omega_{gQ}=l_{d}(\big{[}Q([\bm{t}_{v},l_{e}(p)],v(i),p),l_{e}(p)\big{]}).$
(2)

Note the two key differences from the standard QFormer (1): first, instead of
feeding QFormer outputs as ‘text’ inputs to frozen LLM, we use the QFormer
queries, along with the encoded prompts, to model encoder latents and
condition decoder generation. Second, the encoded prompts are also used to aid
QFormer query and representation learning, which we term as language
grounding. For better elucidation, we provide a visual comparison in Fig. 1.

It is imperative to note that this grounding directly informs the QFormer of
the encoder latent space, thus aiding it in learning the encoder
representations. However, conditioning only the decoder leads to a natural
tradeoff: the potential ease of modeling may be offset by the reduced model
capacity to incorporate said conditioning, since the encoder is no longer
involved in interpreting the QFormer queries. Our experiments show that
benefits due to the former significantly outweigh the latter. Additionally, we
find that splitting the LLM yields two other benefits: first, it reduces
memory requirements for end-to-end training, as the encoder embeddings can now
be precomputed. Second, inference/text generation is sped up significantly
since decoding requires multiple forward passes through the entire LLM,
whereas we only rely on the decoder.

(a) Standard QFormer

(b) Grounded QFormer (Ours)

Figure 1: Illustration of our framework for vision language fusion with the
language grounded QFormer (right), distinguishing it from the standard design
(left) as proposed in InstructBLIP [6].

##  III Experiments

Here, we provide empirical evidence to support our hypothesis on more
efficient training. Due to compute limitations, we restrict ourselves to
smaller LLMs and datasets for our experiments. Specifically, we consider the
FlanT5-base [2] as our frozen LLM and the Eva-clip-g/14 [16], a CLIP-based
vision transformer, for the frozen image encoder. Thus, it is important to
point out that we do not intend to compete against the state-of-art
InstructBLIP models, which would require billion parameter LLMs and large
scale pretraining. Rather, in our small but controlled setup, we aim to
compare the efficiency of the default QFormer pipeline against ours. Finally,
for both frameworks, the QFormer uses the default architecture and
initialization as in [5, 6], without any prior pretraining.
Tasks For this study, we primarily focus on two tasks: Captioning and Visual
Question Answering (VQA). For captioning we consider the COCO captions dataset
[17] and for visual question answering we consider VQAv2 [18] for training and
OKVQA (Outside Knowledge VQA) [19] for zero-shot evaluation. To specify the
task, we fix a set of text prompts for each task during training and randomly
sample one prompt from that set for each iteration.

###  III-A Single Task Evaluation

We begin evaluating our grounded QFormer against the default one, without any
pretraining, to establish baselines on captioning and question answering. Both
models are trained under multiple hyperparameter configurations until training
loss saturation, which takes about 50 epochs, and the respective best
validation scores are reported. Our results in Table I indicate that our
method significantly improves the learning capacity of the pipeline, even
without any pretraining.

Model | Captioning (BLEU-4) | VQA (Acc.) (%)
---|---|---
QFormer | 0.238 | 57.72
Grounded (Ours) | $\bm{0.364}$ | $\bm{63.25}$

TABLE I: Performance comparison for single task models; each model is only
trained on one task (either Captioning or VQA) and evaluated on the
corresponding validation set.

###  III-B Multi-task Training

A commonly sought advantage of VLMs is their ability to simultaneously perform
different tasks well. To achieve such a model, pretraining followed by multi-
task training/finetuning is effective [5, 6]. To emulate this, we train multi-
task models to perform both Captioning and VQA, by first using image
captioning as the pre-training objective (20 epochs), and then performing
multi-task instruction finetuning for captioning and VQA (15 combined epochs).
The choice of our datasets, Coco Captions and VQAv2—which share same train-
val-test image split—allows us to prevent train-test information leak for a
fair evaluation. Note that this procedure also mimics pretraining with
captioning, followed by an instruction finetuning procedure for training BLIP
models. Results are given in Table II. We find that for the designated scale
pretraining followed by instruction finetuning task, our method achieves
significant improvements over the standard QFormer. Additionally, we see that
our method beats the single task baseline for VQAv2 (Table I), harnessing the
benefits of multi-task training and the synergy between captioning and VQA
tasks.

Model | Pretrain Captioning (BLEU-4) | Final Captioning (BLEU-4) | VQA (Accuracy) (%)
---|---|---|---
QFormer | 0.231 | 0.209 | 55.4
Grounded (Ours) | $\bm{0.357}$ | $\bm{0.362}$ | $\bm{66.8}$

TABLE II: Performance comparison for pretraining, followed by multi-task
instruction finetuning.

###  III-C Zero-shot evaluation

Instruction finetuned VLMs can demontrate remarkable zero-shot abilities and
in the same vein, we evaluate the models on OKVQA, with results in Table III.
For comparison, we report the zero-shot OKVQA performance for BLIP-2 models
[5] (InstructBLIP includes OKVQA during training, so comparison unavailable).
Surprisingly, we find that our method demonstrates comparable zero-shot
performance, despite utilizing two orders of magnitude less pretraining and
much smaller LLMs.

Model | OKVQA (Acc.) (%) | LLM
---|---|---
QFormer (Baseline) | 28.83 | FlanT5base $\sim$ 240M
Grounded (Ours) | $\bm{38.96}$ | FlanT5base $\sim$ 240M
BLIP-2 OPT | $36.4$ | OPT $\sim$ 6.7B
BLIP-2 FlanT5xl | 40.7 | FlanT5xl $\sim$ 3B

TABLE III: Zero-shot performance comparison on OKVQA.

###  III-D Pretraining Efficiency

Next, we compare the pretraining efficiency of both pipelines. Specifically,
we evaluate the best validation score achieved for the captioning task within
a fixed number of epochs, across different hyperparameter and learning rate
configurations. Across all runs, we report the highest achieved performance
within a fixed number of epochs. As shown in Fig. 2, we find that our method
achieves significantly faster learning as well as notably better peak
performance.

Figure 2: Pretraining comparison, showcasing our framework significantly
improves pretraining efficiency.

.

###  III-E Language Grounding Ablation

Finally, we study the effect of language grounding i.e. feeding the encoded
prompts to the QFormer. With the multi-task setup, we first pretrain two
models, one with grounding and one without, to roughly the same captioning
score, noting that the model without grounding takes an additional five epochs
to reach similar performance. However, subsequent multi-task instruction
finetuning reveals more pronounced advantages. Specifically, while the
captioning score remains similar, we find that language grounding
significantly speeds up VQA learning, with the grounded model maintaining a
$5$–$6$% performance lead in the earlier epochs and hitting peak performance
much faster (see Fig. 3). This jives with our intuition that grounding aids
the QFormer in learning the appropriate encoder latent representations for
prompting the decoder, while noting that this speed-up is more significant for
VQA, which incidentally generates more informative input prompts i.e. image-
based questions in this case.

Figure 3: Language Grounding can significantly speed up multi-task learning,
especially when the prompts are more informative of the task as in VQA.

.

##  IV Discussion & Future Work

In this work, we presented an alternate pipeline for QFormer-based vision-
language fusion. Specifically, we used the QFormer to condition generation in
the LLM latent space, augmented with language grounding to improve this
conditioning. Experiments demonstrate that our method significantly improves
both performance and learning efficiency, when compared to the standard
QFormer pipeline. However, there remains a performance gap to state-of-the-art
models, arising from the limited LLM capacity and pretraining scale. Indeed,
in our preliminary experiments with larger FlanT5 models, we observed
performance dips (a slight one for our method; a drastic one for the baseline)
when pretrained on COCO captions ($\sim$ 500k samples), indicating need for
scaling up pretraining, potentially to InstructBLIP levels ($\sim$ 130m
samples). A second limitation lies in the fact that extending our framework to
decoder-only models may not be trivial, as the choice of decoder layer for a)
extracting prompt representations to ground the QFormer and b) feeding QFormer
queries and conditioning generation is less obvious, and becomes a tunable
hyperparameter in itself. Nevertheless, our results are compelling enough to
merit further exploration by scaling up: a) pretraining, b) scope of multi-
tasks, and c) larger LLMs with better generative capabilities.

## References

  * [1] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei,  “Language models are few-shot learners,”  CoRR, vol. abs/2005.14165, 2020.
  * [2] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei,  “Scaling instruction-finetuned language models,” 2022.
  * [3] Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le,  “Finetuned language models are zero-shot learners,” 2022.
  * [4] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou,  “Chain-of-thought prompting elicits reasoning in large language models,” 2023.
  * [5] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi,  “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” 2023.
  * [6] Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi,  “Instructblip: Towards general-purpose vision-language models with instruction tuning,” 2023.
  * [7] Emma Strubell, Ananya Ganesh, and Andrew McCallum,  “Energy and policy considerations for deep learning in NLP,”  CoRR, vol. abs/1906.02243, 2019.
  * [8] Julian Togelius and Georgios N. Yannakakis,  “Choose your weapon: Survival strategies for depressed ai academics,” 2023.
  * [9] Yi Zhang, Junyang Wang, and Jitao Sang,  “Counterfactually measuring and eliminating social bias in vision-language pre-training models,”  in Proceedings of the 30th ACM International Conference on Multimedia, New York, NY, USA, 2022, MM ’22, p. 4996–5004, Association for Computing Machinery.
  * [10] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova,  “Bert: Pre-training of deep bidirectional transformers for language understanding,” 2019.
  * [11] Ian Goodfellow, Yoshua Bengio, and Aaron Courville,  Deep Learning,  MIT Press, 2016.
  * [12] Diederik P Kingma and Max Welling,  “Auto-encoding variational bayes,” 2022.
  * [13] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio,  “Generative adversarial networks,” 2014.
  * [14] Deb K. Roy and Alex P. Pentland,  “Learning words from sights and sounds: A computational model,”  Cognitive Science, vol. 26, no. 1, pp. 113–146, 2002.
  * [15] Deb K. Roy,  “Learning visually grounded words and syntax for a scene description task,”  Computer Speech & Language, vol. 16, no. 3, pp. 353–385, 2002,  Spoken Language Generation.
  * [16] Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao,  “Eva-clip: Improved training techniques for clip at scale,” 2023.
  * [17] Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollar, and C. Lawrence Zitnick,  “Microsoft coco captions: Data collection and evaluation server,” 2015.
  * [18] Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh,  “Making the v in vqa matter: Elevating the role of image understanding in visual question answering,” 2017.
  * [19] Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi,  “Ok-vqa: A visual question answering benchmark requiring external knowledge,” 2019.