# Signature of half-metallicity in $\text{BiFeO}_{\text{3}}$

Soumyasree Jena<EMAIL_ADDRESS>Sanchari Bhattacharya  Department of
Physics and Astronomy, National Institute of Technology, Rourkela, Odisha,
India, 769008  Sanjoy Datta<EMAIL_ADDRESS>Department of Physics and
Astronomy, National Institute of Technology, Rourkela, Odisha, India, 769008
Center for Nanomaterials, National Institute of Technology, Rourkela, Odisha,
India, 769008

(August 28, 2024)

###### Abstract

$\text{BiFeO}_{\text{3}}$ has drawn a great attention over last several
decades due to its promising multiferroic character. In the ground state the
bulk $\text{BiFeO}_{\text{3}}$ is found to be in the rhombohedral phase.
However, it has been possible to stabilize $\text{BiFeO}_{\text{3}}$ with
tetragonal structure. The importance of tetragonal phase is due to its much
larger value of the electric polarization and the possible stabilization of
ferromagnetism as in the rhombohedral phase. Furthermore, the tetragonal
structure of $\text{BiFeO}_{\text{3}}$ has been reported with different $c/a$
ratio, opening up the possibility of a much richer set of electronic phases.
In this work, we have used density functional theory based first-principle
method to study the ferromagnetic phase of the tetragonal
$\text{BiFeO}_{\text{3}}$ structure as a function of the $c/a$ ratio. We have
found that as the $c/a$ ratio decreases from $1.264$ to $1.016$, the
tetragonal $\text{BiFeO}_{\text{3}}$ evolve from a ferromagnetic semiconductor
to a ferromagnetic metal, while passing through a _half-metallic_ phase. This
evolution of the electronic properties becomes even more interesting when
viewed with respect to the volume of each structure. The most stable half-
metallic phase initially counter-intuitively evolve to the magnetic-
semiconducting phase with a reduction in the volume, and after further
reduction in the volume it finally becomes a metal. So far, this type of metal
to insulator transition on compression was known to exist only in alkali
metals, especially in Lithium, in heavy alkaline earth metals, and in some
binary compound.

##  I Introduction

The discovery of the unusual electronic property, commonly known as half-
metallic ferromagnetism, in Mn-based Heusler alloys groot  had started an
intense research to find the existence of it in other materials both
theoretically as well as experimentally, which continues unabated till today.
In a half-metal, the Fermi energy is populated by electrons with only one type
of spin character, either up or down. This unique character of the charge
carriers of a half-metal makes it one of the most suitable candidate for the
spintronic devices Zutic ; Coey . Half-metallic ferromagnetic phase are mostly
found in Heusler alloys, and in some zinc-blende structures, colossal-
magnetoresistance materials, transition metal dioxides, double pervoskites and
sulfides Katsnelson . More recently, existence of half-metallicity has been
predicted in some Fe based perovskites such as,$(\rm{BaFeO_{3}})$ Zhi  and
$(\rm{CaFeO_{3}})$ Run . While the half-metallicity has been found in bulk
$(\rm{BaFeO_{3}})$ at ambient pressure, it is found in the surface states of
$(\rm{CaFeO_{3}})$.

Figure 1: The T-BFO cell-structure: (a) The unit-cell (UC), (b) The
$\text{FeO}_{\text{6}}$ octahedron with one axial (O1) and two equatorial
oxygen (O2 & O3),(c) The displacement of axial oxygen along {001} plane after
relaxation.

Figure 2: The T-BFO supercell:(a) The $\text{FeO}_{\text{6}}$ octahedra with
angles $\theta_{1}$ & $\theta_{2}$, (b) G-type anti-ferromagnetic (G-AFM) spin
imposition in $2\times 2\times 2$ supercell (sc), (c) Ferromagnetic (FM)
magnetic supercell (sc), where the ‘$\uparrow$’ and ‘$\downarrow$’ correspond
to spin-up and spin-down respectively.

In a parallel development, the search for a suitable material with the
possibility of external electrical control had been quite intense for last
couple of decades and multiferroic materials, especially Bismuth Ferrite
($(\rm{BiFeO_{3}})$), has emerged as one of the most promising candidate in
this endeavor. The coexistence of coupled magnetic order and electric
polarization has catapulted $\rm{BiFeO_{3}}$ (BFO) as one of the leading
contenders in this race. BFO has been found in various forms having different
crystal symmetries Ravindran ; oswaldo . In its ground state, bulk BFO has R3c
symmetry with a rhombohedral perovskite structure. It has a large electric
polarization along the $[111]$ direction of the pseudocubic unit cell along
with a G-type anti-ferromagnetic ordering of $\rm{Fe}^{3+}$ spins. The anti-
ferromagnetic ordering is superimposed with a long-wavelength cycloidal
modulation Sosnowska-82 ; Sosnowska-92 ; R.Przenioslo ; J.Herrero . BFO
started receiving intense attention ever since the possibility of weak
ferromagnetism (FM) induced by the Dzyaloshinski-Moriya type interaction was
predicted Kadomtseva-JETP2004 ; Spaldin-PRB2005  in this R3c structure.
However, the cycloidal modulation forces the spins to cant which induces a
spin-density-wave instead of the uniform FM state. A crucial step to obtain
the uniform FM phase is to suppress the cycloidal modulation of the spins.
There are evidences that this modulation can be suppressed in thin films
Sosnowska-92 . A large magnetic field can achieve this in the bulk BFO as well
Kadomtseva-JETP2004 ; Wardecki-JPSJ2008 . In 2003, magnetic moments as large
as $1\mu_{B}/\rm{Fe}$ wang  was reported in heteroepitaxially constrained
$\textrm{BiFeO}_{3}$ thin film while others Bellaiche ; Eerenstein-Science2005
; Forget-APL2005  claim to find much smaller or even non-existent FM moments
in this system. However, there are further experimental evidences in favor of
the weak FM in BFO films Kiryukhin-PRL2011 .

Although, the existence of large ferromagnetic moment in the rhombohedral BFO
(R-BFO) is still a debated issue, there are growing experimental evidences
that BFO can host FM moments Ramirez-ActaMat17  and it could be quite large
Anil-Kumar-EPL19  under certain conditions. Interestingly, the
heteroepitaxially constrained BFO structure belongs to $P4mm$ space group with
a body centered tetragonal unit cell. In a subsequent first-principle based
calculation, Ederer and Spaldin eder-spladin-prl  predicted the thin-film
tetragonal (T) BFO structure with G-type anti-ferromagnetic (G-AFM) ordering
to have much larger ferroelectric polarization as ($\approx 150\mu
C/\textrm{cm}^{2}$) compared to the R phase. Furthermore, using strain induced
calculation, they have studied all the types of anti-ferromagnetic ordering in
T-BFO eder-spladin-strain . This prediction of larger polarization has been
confirmed experimentally Zhang-Ramesh-PRL , although the measurement of the
polarization has not been done on pure T-BFO phase, but rather on the mixed
phase of T-BFO and R-BFO.

Figure 3: The Band structure (BS) with spin-up (SU) and spin-down (SD)
components corresponding to the FM phase in (a) structure-I, (b) structure-II,
and (c) structure-IV respectively.

The tetragonal structure is metastable Ravindran , with the possibility of a
wide range of $c/a$ ratio wang ; Ricinschi ; Gp-Srivastava ; Sando ; oswaldo .
Major efforts have been devoted to understand and to enhance the electric
polarization in the T-BFO structure Ricinschi Zhang-Ramesh-PRL eder-spladin-
prl . However, the issue of ferromagnetism has not been explored in pure T-BFO
structure as extensively as in the case of R-BFO phase, although, there are
density-functional theory (DFT) based first principle calculations on doped
T-BFO systemsMn-doping ; Ni-doping . The non-uniqueness of the structural
parameters of the T-BFO phase leaves the possibilities of exotic phases wide
open. In this work, we study the ferromagnetic phase of the T-BFO structure
using DFT based first principle calculations. We have systematically studied
the four different T-BFO structures (I,II,III and IV), having different $c/a$
ratios and which have already been reported experimentally wang ; Ricinschi .
The $c/a$ ratio decreases monotonically from structure-I to structure-IV. We
have found that in the presence of the uniform FM order, the structure-II is
energetically the most stable out of all the four structures. Interestingly,
this structure also hosts half-metallicity, while structure-I is a magnetic
semiconductor and the other two structures have metallic character.
Additionally, we have found that the electrical properties in this system
evolves counter-intuitively, i.e., there is a transition from the half-
metallic phase of structure-II to the semiconducting phase of structure-I
along with a significant decrease in the volume. Half-metallicity has been
predicted in doped T-BFO Ni-doping  system and in some BFO-based
heterostructures yin ; nan ; wei . However, to the best of our knowledge, this
is the first time the half-metallicity is being reported in the pure T-BFO
structure.
The paper is organized as follows: we have first discussed the computational
details in Sec.II, followed by the structural, electronic and magnetic
properties in Sec. III.1, III.2, III.3 respectively. In Sec. III.4, we have
discussed the role of the molecular orbitals, charge density and
hybridization. Finally, we conclude our results in Sec. IV.

##  II Computational Details

To systematically explore the effect of the uniform ferromagnetic state in the
T-BFO structure , we have performed DFT based first-principle calculations by
the pseudopotential method as implemented in the Quantum Espresso (QE) package
qe . The pseudopentials with Projected Augmented Wave (PAW) paw  type basis
sets that have been used in our calculation consists of 15 valence electrons
$(5d^{10}6s^{2}6p^{3})$ for bismuth (Bi), 16 valence electrons
$(3s^{2}3p^{6}3d^{6}4s^{2})$ for iron (Fe) and 6 valence electrons
$(2s^{2}2p^{4})$ for oxygen (O). We have used $GGA+U$ method with Perdew-
Burke-Ernzerhof (PBE) pbe  type exchange correlation functional. An energy
cut-off of 80 Ry has been used for the subsequent calculations. The Hubbard
$U$ parameter has been taken into account for the Fe-$3d$ orbital and its
strength has been set to $4.5\leavevmode\nobreak\ \rm{eV}$. With this choice
of the Hubbard $U$, the theoretical band gap has been found to be reasonably
close to the experimentally reported value in T-BFO chen ; Cameliu-expt-bg .
${\text{BiFeO}_{\text{3}}}$ crystallizes in a tetragonal phase with space
group $P\it{4mm}$ (99) with different possible $c/a$ ratios. Since it is well
known that in the ground state the magnetic moments are ordered in G-type
anti-ferromagnetic arrangement, we have used a $2\times 2\times 2$ supercell
(sc) composed of 8 formula units to compare the energies of the FM and G-AFM
phases. A $6\times 6\times 6$ Monkhorst-Pack k-point grids Monkhorst-pack  has
been used for the self-consistent calculation. The Methfessel and Paxton type
smearing is used with a width of 0.01 Ry Methfessel-paxton . Ionic relaxation
has been performed with the convergence threshold of the inter-atomic forces
being less than $10^{-3}$ Ry/bohr. We have used collinear spin-polarized
calculation to study the FM phase.

Figure 4: The total density of states (TDOS) corresponding to the FM phase in
(a) structure-I, (b) structure-II, and (c) structure-IV respectively.

Figure 5: The local density of states (LDOS) corresponding to the FM phase in
(a) structure-I, (b) structure-II, and (c) structure-IV respectively.

##  III Results and Discussion:

###  III.1 Structural properties

In this work, we have used the experimentally reported parameters of the T-BFO
unit-cell ($P\it{4mm}$ space-group and $C_{4v}$ point group) wang ; Ricinschi
. The highest $c/a$ ratio is 1.264 (structure- I) Ricinschi  and for the rest
of the structures it is 1.233 (structure-II), 1.049 (structure-III) Gp-
Srivastava  and 1.016(structure-IV) wang . The lattice parameter ‘a’ of these
structures are 3.67, 3.77, 3.88 and 3.93 Å respectively. The ionic relaxations
have been performed for all the structures while keeping the lattice constant
$a$ and the $c/a$ ratio fixed before studying the electronic and magnetic
properties. The relaxations have been carried out after imposing the magnetic
ordering.

In Fig. 1(a), we have shown the unit-cell of the T-BFO. The
$\text{FeO}_{\text{6}}$ octahedra has been shown in the Fig. 1(b), wherein,
the axial oxygen has been named as ‘O1’ while the two equatorial oxygen have
been designated as ‘O2’ and ‘O3’ respectively. The lattice plane perpendicular
to the [001] direction has been pointed out in Fig. 1(c). In Fig. 2(a), the
supercell has been introduced with the oxygen octahedra. The G-AFM and the FM
type of spin-orderings, that have been used in our study, have been shown in
Fig. 2(b) and (c) respectively.

In Appendix A, we have presented relative energies of these four structures
with G-AFM and FM type magnetic ordering. We have found that the G-AFM
ordering is energetically more favorable than the FM phase irrespective of the
$c/a$ ratio. However, the energy follows a similar pattern in the presence of
each type of magnetic ordering, that is, structure-II is energetically most
stable for a given type of magnetic ordering, followed by structure-I,
structure-III and structure-IV.

Table 1:  Top row: The angle Fe-$\textrm{O}_{e}$-Fe (in unit degree, °), where $\textrm{O}_{e}$ is the equatorial oxygen O2/O3. Second row: the bond-length Fe-O1 (in unit Å) Third row: Fe-$\textrm{O}_{e}$ bond-length (in unit Å). Bottom row: the volume of the unit-cell BFO (in unit Å3 ) Structures |  | str-I | str-II | str-III | str-IV
---|---|---|---|---|---
c/a ratio |  | 1.264 | 1.233 | 1.049 | 1.016
Fe-$\textrm{O}_{e}$-Fe |  | 146.59 | 144.01 | 163.31 | 165.04
O1-Fe |  | 1.85 | 1.83 | 1.89 | 1.88
Fe-$\textrm{O}_{e}$ |  | 1.92 | 1.98 | 1.96 | 1.98
Volume |  | 62.48 | 66.07 | 61.27 | 61.67

Figure 6: The top panel ((a), (b), (c)) corresponds to the FM phase in
Bi-$6p_{x},6p_{y},6p_{z}$. The middle panel ((d), (e), (f)) corresponds to
Fe-$3d-{e_{g}},t_{2g}$ and the lower panel ((g), (h), (i)) corresponds to O1,
O2, O3-$2p_{x},2p_{y},2p_{z}$.

Since our primary interest is in the nature of the FM phase, in the following
sections we present the results of T-BFO with only FM ordering. To understand
the relation between the energy of a given structure and its $c/a$ ratio, we
have studied the Fe-$\textrm{O}_{e}$-Fe angle, and the O1-Fe and
Fe-$\textrm{O}_{e}$ bond length of each structure after performing the ionic
relaxation. Here $\textrm{O}_{e}$ represent the equatorial oxygen atoms. These
results are presented in Table LABEL:Table:Fe-O-Fe-bond. The direct
correlation between the energy and the Fe-$\textrm{O}_{e}$-Fe angle is quite
apparent. Similar to the energy, from structure-I to structure-II,
Fe-$\textrm{O}_{e}$-Fe angle initially decreases and then it increases
monotonically. The bond length between ‘Fe’ and the two type of oxygen, i.e.,
the axial oxygen O1 and the two equatorial oxygen $\textrm{O}_{e}$ have also
been measured. The O1-Fe bond length decreases slightly from the structure-I
to the structure-II and then increases in structure-III. This bond length
remains almost unchanged in structure-IV, compared to structure-III. The trend
is almost identical with the pattern followed by the energy of each structure,
that is, similar to the energy, this bond length is the lowest in the case of
structure-II. Interestingly, the structure-II shows the half-metallic
character. However, a similar correlation is not observed between the
Fe-$\textrm{O}_{e}$ bond length and the energy. While the Fe-O2 and Fe-O3 bond
lengths are observed to be the same, the lowest Fe-$\textrm{O}_{e}$ bond
length is found in the case of structure-I. This bond length increases in
structure-II and then remains more or less constant till structure-IV. In the
later sections, we are going to explore in detail the relation between the
$c/a$ ratio of each structure and its electronic properties.

###  III.2 Electronic properties

For a systematic understanding of the electronic properties, we have computed
the band structure, total density of states (TDOS), local density of states
(LDOS) and the projected density of states (PDOS) for each structure. In Fig.
3, we have presented the spin-resolved band structure of structure-I, II and
IV. The fundamental nature of the band structure of structure-III is identical
with the structure-IV, and it has been studied in Appendix B. From the Fig.
3(a), it is clear that for the highest $c/a$ ratio $1.264$, i.e., in
structure-I, no states are available at the Fermi energy from either of the
spin channel. However, the band gap is not the same for the two spin channels.
The band gap corresponding to the spin-up channel is estimated to be $\simeq
0.53\leavevmode\nobreak\ \rm{eV}$, while it is $\simeq
1.61\leavevmode\nobreak\ \rm{eV}$ for the spin-down channel. The band gaps
have been calculated at the zone center. This shows that structure-I is a
magnetic semiconductor similar to the R3c phase of BFO rbfo . From Fig. 3(c),
it is obvious that structure-IV is a metal as both the spin channels
contribute states at the Fermi level. A similar behavior is observed in
structure-III (Appendix B).

Figure 7: HOMO and LUMO of structure-I with (a) and (b) spin-up, and (c) and
(d) spin-down respectively.

Figure 8: (a) and (b): HOMO and LUMO of structure-I with spin-up, and (c)
Molecular orbital for spin-down.

However, in between these two limits of the $c/a$ ratio, in the intermediate
structure with $c/a=1.233$, i.e., the structure-II surprisingly has a non-zero
contribution at the Fermi energy only from the spin down channel. This makes
the structure-II behave like a half-metal. In this case a band-gap of $\simeq
0.89\leavevmode\nobreak\ \rm{eV}$ is observed in the spin-up channel. The band
structure is presented in Fig. 3(b). A closer look at the systematic change in
the electronic properties with respect to the volume of each structure reveals
a fascinating phenomena, which has not been reported so far in a pure compound
to the best of our knowledge. A simple analysis shows that the volume of the
structure-II is the highest, while the structure-I has a lower volume than the
structure-II. Structure-III and IV occupy almost the same volume and is less
than that of the structure-I. This indicates that the change from the
semiconducting state of the structure-I to the half-metallic phase of the
structure-II is quite counter-intuitive, since the volume effectively
increases from the structure-I to the structure-II. It is generally expected
that under pressure (with reduction of volume), the insulator/semiconductor
turns into a metal. Interestingly, our results indicate an opposite trend to
this general expectation. Furthermore, the semiconducting state of the
structure-II becomes a metal in the structure-III and remains a metal in
structure-IV. This change is in tune with the general expectation of the
insulator/semiconductor to metal transition under pressure, as the volume of
structure-III/IV is lower than that of the structure-II. Interestingly, this
kind of metallic to insulating/semiconducting and back to metallic transition
has been observed in pure Li, which becomes a semiconductor at
$80\leavevmode\nobreak\ \rm{GPa}$ Shimizu  and then it becomes a metal once
again at $120\leavevmode\nobreak\ \rm{GPa}$ Matsuoka . Experimental evidences
of this kind of counter-intuitive metal to insulator transition under high
pressure have also been reported earlier in Na Ma-Sodium , FCC-Ca Dunn-FCC-Ca
, Stager-FCC-Ca , and Ni McMahan-Ni , and in the binary compound
$\rm{CLi}_{4}$ Jin-Chen-CLi4 .

The total density of states (TDOS) corresponding to the above mentioned band
structures have been shown in Fig. 4. In Fig. 4(a), the TDOS has been
presented for the structure-I, and it is clear that a band gap exists for both
the spin components, although they are unequal. It is also clear that the band
gap is smaller for the spin-up component. The TDOS of structure-II has been
presented in Fig. 4(b). While there are no states at the Fermi energy from the
spin-up channel, there exists a small but finite number of states from the
spin-down component, which is the typical signature of a half-metal. It is
clear from the Fig. 4(c), that both the spin components contribute to the
conductivity in the case of structure-IV, although the contribution is more
from the spin-up component.

In order to examine the orbital contribution of different atoms to the
electronic properties in the T-BFO structures, we have computed the local
density-of-states (LDOS). These results are presented in Fig. 5. In Fig. 5(a),
the results for the structure-I has been presented. It is clear that there are
no states at the Fermi energy and there exists a band gap for both the spin
channels. However, as the $c/a$ ratio is lowered from structure-I to
structure-II, one can observe in Fig. 5(b) that a finite amount of electronic
states appear at the Fermi energy. Interestingly, only the spin-down channel
of the Bi-$6p$ orbital appears to make the contribution to the TDOS at the
Fermi energy. In the molecular orbital analysis (Sec. III.4), we have found
contributions from the Fe-$3d$ and O-$2p$ orbitals. In Fig. 5(c), we have
presented the results for the structure-IV, which is found to be a metal from
its band structure. However, there are some interesting features to observe in
the charge carriers and its spin type. ‘Bi’, more specifically Bi-$6p$
orbital, contributes electron carriers from both the spin channels, while
Fe-$3d$ and O-$2p$ orbitals contribute hole carriers to the system only from
the spin-up channel.

It is important to note that, like the ferromagnetic rhombohedral phaserbfo ,
in Fig. 5, we can also observe an indication of strong hybridization between
the Fe-$3d$ and O-$2p$ orbitals in both the valence and the conduction bands.
Furthermore, a hybridization can also be observed between Bi-$6p$ and O-$2p$
in the conduction band. The hybridization between Fe-$3d$ and O-$2p$ is mainly
found in the spin-up channel of the valence band and in the spin-down channel
of the conduction band close to the Fermi energy. It is also important to note
that the Bi-$6s$ orbital contribution is found far below the Fermi level at
around $-1.5eV$, which indicates that these Bi-$6s$ lone pair electrons are
localized in this system. However, Bi-$6s$ orbital takes part in the
hybridization process. In Sec. III.4, we have discussed the type of
hybridization and the role of specific atomic orbitals in more details.

Figure 9: The charge density (CD) distributions at $\\{001\\}$-plane for (a)
structure-I, (b) structure-II, and (c) structure-IV respectively.

To further understand the contribution of the individual orbitals, we have
calculated the projected density of states (PDOS), and the results have been
presented in Fig. 6. From Figs. 6(a),(d), and (g), it is clear that there is a
band gap in both the spin channels. It is also clear that in this structure
the band just above the Fermi energy has contributions from Bi-$6p_{z}$,
Fe-$3d_{z^{2}}$ and O1-$2p_{z}$ orbitals. Interestingly, all of these orbitals
belong to the spin-down channel. On the other hand, the band just below the
Fermi energy has contributions only from Fe-$3d_{x^{2}-y^{2}}$, O2-$2p_{y}$,
and O3-$2p_{x}$. These orbitals belong to the spin-up channel. It is
interesting to note there is no contribution from the Bi atom in this band.
Surprisingly, as the $c/a$ ratio is lowered in the structure-II, the spin-down
channel of Bi-$6p_{z}$ moves below the Fermi level. This is very evident from
Fig. 6(b). The spin-down channels of the Fe-$3d_{z^{2}}$ and O1-$2p_{z}$
orbitals also move below the Fermi energy. However, they have vanishingly
small but finite contributions to the PDOS. Their presence has been detected
by the molecular orbital study, which has been discussed in Sec. III.4.
Interestingly, the spin-up channels of the orbitals, which were the closest to
the Fermi energy in the structure-I are pushed further down in energy in the
structure-II, giving rise to the _half-metallic_ character to the T-BFO. As
the $c/a$ ratio is lowered further in the structure-IV, both the spin channels
of the Bi-$6p_{z}$ orbital move further below the Fermi energy, while only the
spin-up component of the $e_{g}$ band of Fe along with the O1-$2p_{z}$,
O2-$2p_{y}$, and O3-$2p_{x}$ orbitals crosses over the Fermi level, thereby
making the structure-IV a ferromagnetic metal. Furthermore, it is also clear
from the Figs. 6(c), (f) and (i), that ‘Bi’ contributes only electron carriers
of both types of spin to the system, while the other orbitals mentioned above
contribute only hole carriers, that too from the spin-up channel, to the
system. It is also important to note that the Bi-$6p_{z}$ orbital consistently
moves towards the Fermi level from above and then crosses over the Fermi
energy as the $c/a$ ratio is decreased gradually. On the other hand, other
orbitals systematically moves closer to the Fermi energy from below and
finally crosses over with the decrease in the volume from the structure-I to
the structure-IV.

Table 2: Estimation of the total magnetic moment (TM) and the individual magnetic moments (in $\mu_{b}$) contributed by ‘Bi’, ‘Fe’ and ‘O’ corresponding to the G-type anti-ferromagnetic (G-AFM) phase. Structures | str-I | str-II | str-III | str-IV
---|---|---|---|---
c/a ratio | 1.264 | 1.233 | 1.049 | 1.016
TM | 0.0000 | 0.0000 | 0.0000 | 0.0000
Bi | $\mp 0.0000$ | $\mp 0.0000$ | $\mp 0.0000$ | $\mp 0.0000$
Fe | $\mp 3.5996$ | $\mp 3.6031$ | $\mp 3.6152$ | $\mp 3.6127$
O | $\mp 0.1823$ | $\mp 0.2195$ | $\mp 0.1417$ | $\mp 0.1343$
Table 3: Estimation of the total magnetic moment (TM) and the individual magnetic moments (in $\mu_{b}$) contributed by ‘Bi’, ‘Fe’ and ‘O’ corresponding to the ferromagnetic (FM) phase. Structures | str-I | str-II | str-III | str-IV
---|---|---|---|---
c/a ratio | 1.264 | 1.233 | 1.049 | 1.016
TM | 40.000 | 40.000 | 39.950 | 39.830
Bi | 0.0146 | 0.0117 | 0.0071 | 0.0063
Fe | 3.7300 | 3.6900 | 3.8300 | 3.8200
O | 0.2240 | 0.2584 | 0.2453 | 0.2450

###  III.3 Magnetic properties

In this section, we discuss the magnetic properties of the T-BFO structure and
its dependence on the $c/a$ ratio. In the FM phase, the total magnetic moment
(TM) is found to be around $40\leavevmode\nobreak\ \mu_{b}$ for all of the
four structures, which also agrees well with the results of the ferromagnetic
R3c phase of BFO rbfo . Contribution of different atoms to the magnetization
in the FM phase are listed in the Table LABEL:Table:FM-contribution. To
understand the crucial differences in the contribution towards magnetization
from different atoms compared with the anti-ferromagnetic phase, we have
presented a similar analysis for the G-AFM phases in the Table
LABEL:Table:G-AFM-contribution. The major difference that we have found is
that in the case of G-AFM phases ‘Bi’ doesn’t contribute anything to the total
magnetic moment for all the four structures, while in the case of FM phases it
has a finite contribution to the total magnetic moment. This behavior is
similar compared to the R3c structure rbfo . In the T-BFO, the contribution
decreases monotonically with the $c/a$ ratio from structure-I to structure-IV.
As expectedly, Fe’s contribution towards the total magnetic moment has been
found to be the largest among all the atoms in both the FM and G-AFM phases.
Furthermore, the Fe’s magnetic moment has been found to be larger in the FM
phase, and this trend is also similar to the rhombohedral phase rbfo .
However, in the T-BFO phase Fe’s magnetic moment has been observed to be lower
compared to the rhombohedral phase.

A similar scenario is found in the case of ‘O’ atom as well, which agrees with
the R3c structure, i.e., it contributes much more towards the total
magnetization in the FM phase as compared to the G-AFM phase rbfo . However,
in comparison to the rhombohedral structure, the magnetic moment of the ‘O’
atom is significantly higher in the G-AFM phase of the T-BFO structure, while
they are comparable in the FM phase. In this phase, in the case of structure-
IV and structure-III, the TM is less than $40\leavevmode\nobreak\ \mu_{b}$,
while in the structure-II it is exactly $40\leavevmode\nobreak\ \mu_{b}$. This
is consistent with our conclusions based on the band structure and the TDOS
results presented in Fig. 3 and Fig. 4. It is well known pickett  that in the
case of a half-metallic phase, the magnetic moment should be an integer
multiple of $\mu_{b}$. Interestingly, the structure-I, which is found to be a
magnetic semiconductor, also has a magnetic moment of $40\leavevmode\nobreak\
\mu_{b}$. This numerical value is identical with the strength of the magnetic
moment obtained in the magnetic semiconducting state of the rhombohedral BFO
structure rbfo . In the case of structure-I, we also would like to point out
that the Bi’s contribution to the total magnetic moment is higher in the
magnetic semi-conducting phase of the T-BFO structure compare to the R3c
structure rbfo .

###  III.4 Molecular orbital, Charge density and Hybridization:

For further understanding about the orbital contributions, we have
investigated the nature of the molecular orbitals in the vicinity of Fermi
energy. In Fig. 7, we have presented the highest occupied molecular orbital
(HOMO) and the lowest unoccupied molecular orbital (LUMO) for both the spin
channels corresponding to the structure-I. From Fig. 7(a) it is clear that the
HOMO corresponding to the spin-up channel has contribution from the
Fe-$3d_{x^{2}-y^{2}}$ orbital and the $2p$ orbitals of the equatorial oxygen,
more specifically from the O2-$2p_{y}$ and O3-$2p_{x}$ orbitals. This is
consistent with the PDOS results presented in Figs. 6(a), (d), and (g). In the
LUMO, i.e., in Fig. 7(b) contributions of Bi-$6p_{z}$ and a tiny contribution
from the equatorial oxygens can be seen. In the spin-down channel,
contributions of the Fe-3$d_{xy}$ orbital and the equatorial oxygens are found
in the HOMO, shown in Fig. 7(c), while the Fe-3$d_{z^{2}}$ orbital and the
O1-2$p_{z}$ contribute to the LUMO, plotted in Fig. 7(d). In the case of
structure-II, for the spin-up channel, the composition of the HOMO and the
LUMO shown in Figs. 8(a) and (b) respectively is identical with the same of
the structure-I. In the case of spin-down channel which shows the conducting
property, we have plotted the molecular orbital which is shown in Fig. 8(c).
In this case, the contributions from Bi-$6p_{z}$, Fe-3$d_{z^{2}}$ and the
O1-$2p_{z}$ are very prominent and retain their own character while the
contributions from equatorial oxygens and other Fe-3$d$ orbitals i.e.,
Fe-3$d_{x^{2}-y^{2}}$ and Fe-3$d$-$t_{2g}$ orbitals have lost their own
characters due to strong hybridization.

Furthermore, the electronic charge density denotes the nature of the bonding
among the different atoms. The charge density calculations along the
$\\{001\\}$ plane gives the details of the sharing of the chemical/ionic
bonding between ‘Bi’, ‘Fe’ and ‘O’ ions. From the charge density plots of Fig.
9, we find that in the case of structure-I and structure-II the charges are
localized which shows the charge sharing is less between ‘Bi’, ‘Fe’ and ‘O’.
But in the lowest $c/a$ ratio, i.e., in case of the structure-IV, the charges
are delocalized, which indicates greater charge sharing between ‘Bi’, ‘Fe’ and
‘O’. This gives the confirmation of the ferromagnetic metallic phase which we
obtained earlier in the case of structure-IV. In general, the amount of charge
sharing between the ions, i.e., the anion and the cation decide the type of
bonding that exists in the system whether it is ionic or covalent. In this
case, we find greater charge difference between ‘Bi’ and ‘O’. This type of
charge difference has also been found between ‘Fe’ and ‘O’. Hence we can
conclude that the ‘Bi’ and ‘Fe’ cations create ionic bond with electronegative
oxygen. The perovskite $\rm{BiFeO_{3}}$ can be viewed as a structure with a
central metal (either $\rm{Bi}$ or $\rm{Fe}$) surrounded by electron rich
ligands, i.e, $\rm{O}$. Here, in this case, we find that ‘Fe’ plays the role
of principal central metal atom as it has more contribution than ‘Bi’ to the
hybridization, which we find from L${\ddot{o}}$wdin charges. In this study,
modified $sp^{2}d^{2}$ hybridization has been found in all the cases which we
have discussed in Appendix C. In any such inorganic cluster, there is a direct
electron donation from the ligand to the metal atom. It may happen that the
metal atom may give back some electrons to the ligand LUMO. In the case of
structure-II, the electron bridging between O-Bi-O and O-Fe-O makes the hybrid
orbitals, which might play an important role in the half-metallicity.

##  IV Conclusion:

In conclusion, in this work we have used DFT based first-principle calculation
to explore the evolution of the ferromagnetic phase of the tetragonal
$\text{BiFeO}_{\text{3}}$ structure as a function of the $c/a$ ratio. We have
found that the ferromagnetic TBFO structure harbors rich set of electronic
phases, including the _half-metallic_ phase, incidentally which is also the
most stable electronic phase among all the possible ferromagnetic phases.
Additionally, we have also found that the electronic phase shows a half-metal
to a insulator, and finally to a metal transition on compression, which is
contrary to the commonly anticipated behavior of electronic phases under
pressure. We have found that Bi-$6p_{z}$ orbital plays a very crucial role in
determining the degree of metallicity in the ferromagnetic TBO structure. The
structure with the highest $c/a$ ratio was found to be a magnetic-
semiconductor. With the reduction of the $c/a$ ratio, initially only one spin
channel of the Bi-$6p_{z}$ orbital crosses the Fermi energy from above,
providing the half-metallic character to BFO, and finally both the spin
channels cross the Fermi energy on further reduction of the $c/a$ ratio. From
our molecular orbital analyses, we have found contribution of the Fe-$3d$ and
O-$2p$ orbitals towards the half-metallicity in this system. Since, typically
fabricating stable structures with half-metallicity experimentally is a
challenging task, our findings are going to be quite useful, especially from
the point of view of spintronic applications, as the tetragonal BFO structures
are fabricated quite routinely.

##  Appendix A Energy vs c/a ratio

The relative energy is calculated with respect to the G-AFM energy possessed
by the structure-II,

$\textrm{Relative
Energy}=\frac{\left(E^{structure-x}_{G-AFM/FM}-E^{{}^{structure-
II}}_{G-AFM}\right)}{E_{G-AFM}}\times 100$

where, $E^{structure-x}_{G-AFM/FM}$ is the energy of any structure with either
G-AFM or FM ordering.

The energy of each structure has been represented relative to the
energetically most favorable structure, that is the structure-II with G-AFM
ordering. It is clear from Fig. 10, that the G-AFM type magnetically ordered
phase is lower in energy than the corresponding FM phase irrespective of the
$c/a$ ratio. Interestingly, the energy of the FM phase also follows a similar
pattern to that of the G-AFM phase, that is, the structure-II is the most
stable one among all the possible structures with FM ordering. The lowest
energies found in the structure-II with the G-AFM ordering and FM ordering are
$-10883.8911\leavevmode\nobreak\ Ry$ and $-10883.7638\leavevmode\nobreak\ Ry$
respectively. The energy difference for the most stable G-AFM and the FM phase
is $0.1273\leavevmode\nobreak\ Ry$ which belongs to the structure-II. The next
higher energy differences have been found to be $0.1976\leavevmode\nobreak\
Ry$, $0.2288\leavevmode\nobreak\ Ry$ and $0.2391\leavevmode\nobreak\ Ry$ for
the cases of structure-I, structure-IV and structure-III respectively.

Figure 10: c/a ratio versus relative energy(in %), where the relative energy
of a structure is measured with respect to the most stable G-AFM state, that
is structure II.

Figure 11: The electronic properties of structure-III: (a) Band structure
(BS), (b) Total density of states (TDOS), (c) Local density of states (LDOS)

Figure 12: The projected density of states (PDOS) corresponding to the FM
phase: (a) Bi-$6p_{x},6p_{y},6p_{z}$, (b) Fe-$e_{g},t_{2g}$, (c) O1-$2p_{z}$,
O2-$2p_{y}$, O3-$2p_{x}$ in structure-III. Here, the $\uparrow$ and
$\downarrow$ belong to spin-up and spin-down component respectively.

##  Appendix B Results of structure-III

After increasing the $c/a$ ratio from 1.016 to 1.049, we have found that
structure-III also shows purely metallic behavior, which is not very away from
the properties of the structure-IV. Here, the band structure, TDOS, LDOS &
PDOS have been studied like all other structures. The metallicity in the band
structure, i.e., Fig.11(a) is clearly visible, which agrees with the
corresponding TDOS (Fig.11(b)). In the LDOS we found that, like structure-IV,
Bi-$6p$, Fe-$3d$ and O-$2p$ have contributions near the Fermi energy. Bi-$6p$
contributes electron carriers from both the spin channels, while Fe-$3d$ and
O-$2p$ orbitals contribute hole carriers to the system from the spin up
channel only which is also in the case of structure-IV. In PDOS, we find that
the Bi-$6p_{z}$ orbital contributes in both the spin-channels, whereas the
$e_{g}$ band of ‘Fe’ contributes along with the O1-$2p_{z}$, O2-$2p_{y}$, and
O3-$2p_{x}$ orbitals. These bands cross over the Fermi level for the spin-up
channel only, thereby making the structure-III to be a ferromagnetic metal.

##  Appendix C L${\ddot{o}}$wdin charges

From L${\ddot{o}}$wdin charges analysis, it is found that the hybridization of
the principal central metal ion ‘Fe’ are $sp^{2.420}d^{2.370}$ (structure-I),
$sp^{2.415}d^{2.366}$ (structure-II) and $sp^{2.414}d^{2.346}$ (structure-IV)
respectively. So, the hybridization present in T-BFO can be called as the
modified $s{p^{2}}{d^{2}}$. Also, we have found that the Bi-$5d$ does not have
any contribution in the hybridization. The extra population of electrons in
Fe-$3p$ & Fe-$3d$ may be the outcome of the back-bonding, which has been
created due to the interplay between Bi-$6s$ and Bi-$6p$.

## ACKNOWLEDGEMENTS

S. J. and S. D. would like to thank Prof. P. Sen for his generosity to access
the high performance computing cluster facility at H.R.I, Prayagraj, India,
where some of the initial calculations were performed. The authors are
grateful to Prof. A. P. Chattopadhyay, University of Kalyani, West Bengal,
India for very useful discussions. The authors would like to thank the
computing facility provided by the National Institute of Technology, Rourkela
and the computational facility provided by the Science and Engineering
Research Board, Department of Science and Technology, India (Grant No:
EMR/2015/001227).

## References

  * (1) R. A. De Groot, F. M. Mueller, P. G. Van Engen and K. H. J. Buschow, Physical Review Letters 50, 2024 (1983).
  * (2) I. Žutić, J. Fabian, and S. Das Sarma, Reviews of modern physics 76, 323 (2004).
  * (3) J. M. D. Coey, C. L. Chien, Mrs Bulletin 28, 720 (2003).
  * (4) M. I. Katsnelson, V. Yu. Irkhin, L. Chioncel, A. I. Lichtenstein, and R. A. de Groot, Reviews of modern physics 80, 315 (2008).
  * (5) Z. Li, T. Iitaka and T. Tohyama, Physical Review B 86, 094422 (2012).
  * (6) Run-Wu Zhang, Da-Shuai Ma, Jian-Min Zhang and Y. Yao, Physical Review B 103, 195115 (2021).
  * (7) P. Ravindran, R. Vidya, A. Kjekshus, H. Fjellväg, and O. Eriksson, Physical Review B 74 224412 (2006).
  * (8) O. Dièguez, O. E. González-Vázquez, J. C. Wojdeł and J. Íñiguez, Physical Review B 83, 094105 (2011).
  * (9) I. Sosnowska, T. P. Neumaier, and E. Steichele, Journal of Physics C: Solid State Physics 15, 4835 (1982).
  * (10) I. Sosnowska, M. Loewenhaupt, W. I. F. David, and R. M. Ibberson, Physica B 180, 117 (1992).
  * (11) R. Przeniosło, A. Palewicz, M. Regulski, I. Sosnowska, R. M. Ibberson, and K. S. Knight, Journal of Physics: Condensed Matter 18, 2069 (2006).
  * (12) J. Herrero-Albillos, G. Catalan, J. A. Rodriguez-Velamazan, M. Viret, D. Colson, and J. F. Scott, Journal of Physics: Condensed Matter 22 256001 (2010).
  * (13) A. M. Kadomtseva, A. K. Zvezdin, Y. F. Popov, A. P. Pyatakov, and G. P. Vorobév, Journal of Experimental and Theoretical Physics Letters 79, 571 (2004).
  * (14) C. Ederer, and N.A. Spaldin, Physical Review B 71, 060401 (2005).
  * (15) D. Wardecki, R. Przenioslo, I. Sosnowska, Y. Skourski, & M. Loewenhaupt, Journal of the Physical Society of Japan 77, 103709 (2008).
  * (16) J. B. N. J. Wang, J. B. Neaton,H. Zheng, V. Nagarajan, S. B. Ogale, B Liu, D. Viehland, V. Vaithyanathan, D. G. Schlom, U. V. Waghmare and N. A. Spaldin, Science 299, 1719 (2003).
  * (17) D. Albrecht, S. Lisenkov, W. Ren, D. Rahmedov, I. A. Kornev, L. Bellaiche, Physical Review B 81, 140401 (2010).
  * (18) W. Eerenstein, F. D. Morrison, J. Dho, M. G. Blamire, J. F. Scott, N. D. Mathur, J. Wang, A. Scholl, H. Zheng, S. B. Ogale, and D. Viehland, Science 307, 1203 (2005).
  * (19) H. Béa, M. Bibes, A. Barthélémy, K. Bouzehouane, E. Jacquet, A. Khodan, J. P. Contour, S. Fusil, F. Wyczisk, A. Forget, D. Lebeugle, D. Colson, and M. Viret, Applied Physics Letters 87, 072508 (2005).
  * (20) M. Ramazanoglu, M. Laver, W. Ratcliff II, S. M. Watson, W. C. Chen, A. Jackson, K. Kothapalli, S. Lee, S. W. Cheong, and V. Kiryukhin, Physical review letters 107, 207206 (2011).
  * (21) M. C. Ramírez-Camacho, C. F. Sánchez-Valdés, J. J. Gervacio-Arciniega, R. Font, C. Ostos, D. Bueno-Baques, M. Curiel, J. L. Sánchez-Llamazares, J. M. Siqueiros and O. Raymond-Herrera, Acta Materialia 128, 451 (2017).
  * (22) A. Sil, D. S. Negi, M. H. Naik, M. Jain, R. Datta, R. Ranjan, and P. A. Kumar, EPL (Europhysics Letters) 126 57003 (2019).
  * (23) C. Ederer and N. A. Spaldin, Physical review letters 95, 257601 (2005).
  * (24) C. Ederer and N. A. Spaldin, Physical Review B 81, 054109 (2010).
  * (25) J. X. Zhang, Q. He, M. Trassin, W. Luo, D. Yi, M. D. Rossell, P. Yu, L. You, C. H. Wang, C. Y. Kuo, and J. T. Heron, Physical Review Letters 107 147602 (2011).
  * (26) H. M. Tütüncü and G. P. Srivastava, Physical Review B 78, 235209 (2008).
  * (27) D. Ricinschi, K. Y. Yun and M. Okuyama, Journal of Physics: Condensed Matter 18, L97 (2006).
  * (28) D. Sando, B. Xu, L. Bellaiche, and V. Nagarajan, Applied Physics Reviews 3, 011106 (2016).
  * (29) Q. J. Wang, Q. H. Tan and Y. K. Liu, Computational Materials Science 105, 1 (2015).
  * (30) P. I. Rajan, S. Mahalakshmi and S. Chandra, Computational Materials Science 130, 84 (2017).
  * (31) N. Feng, W. Mi, X. Wang, Y. Cheng, and U. Schwingenschlögl, ACS applied materials & interfaces 7, 10612 (2015).
  * (32) W. Sun, W. Wang, D. Chen, G. Zhang, Z. Cheng and Y. Wang, Journal of Materials Chemistry C 7 463 (2019).
  * (33) L. Yin, Q. Zhang, W. Mi and X. Wang, Journal of Applied Physics 120, 165303 (2016).
  * (34) P. Giannozzi, S. Baroni, N. Bonini, M. Calandra, R. Car, C. Cavazzoni, D. Ceresoli, G. L. Chiarotti, M. Cococcioni, I. Dabo, and A. Dal Corso, Journal of physics: Condensed matter, 21, 395502 (2009).
  * (35) P. E. Blöchl, Physical Review B 50, 17953 (1994).
  * (36) J. P. Perdew, K. Burke and M. Ernzerhof, Physical review letters 77, 3865 (1996).
  * (37) P. Chen, N. J. Podraza, X. S. Xu, A. Melville, E. Vlahos, V. Gopalan, R. Ramesh, D. G. Schlom, and J. L.Musfeldt, Applied Physics Letters 96, 131907 (2010).
  * (38) C. Himcinschi, A. Bhatnagar, A. Talkenberger, M. Barchuk, D. R. Zahn, D. Rafaja, J. Kortus and M. Alexe, Applied Physics Letters 106, 012908 (2015).
  * (39) H. J. Monkhorst and J. D. Pack, Physical review B 13, 5188 (1976).
  * (40) M. P. A. T. Methfessel and A. T. Paxton, Physical Review B 40, 3616 (1989).
  * (41) H. M. H’Linh, E. M. Aguilar, J. A. García, J. R. Ariño, L. Mestres, P. Alemany, D.H. Galván, J. S. Beltrones and O. R. Herrera, Computational Materials Science 164, 66 (2019).
  * (42) T. Matsuoka and K. Shimizu, Nature 458, 186 (2009).
  * (43) T. Matsuoka, M. Sakata, Y. Nakamoto, K. Takahama, K. Ichimaru, K. Mukai, K. Ohta, N. Hirao, Y. Ohishi, and K. Shimizu, Physical Review B 89, 144103 (2014).
  * (44) Y. Ma, M. Eremets, A. P. Oganov, Y. Xie, I. Trojan, S. Medvedev, A. O. Lyakhov, M. Valle, and V. Prakapenka, Nature 458, 182 (2009).
  * (45) K. J. Dunn and F. P. Bundy, Physical Review B 24, 1643 (1981).
  * (46) R. A. Stager and H. G. Drickamer, Physical Review 131, 2524 (1963).
  * (47) A. K. McMahan and R. C. Albers, Physical Review Letters 49, 1198 (1982).
  * (48) X. Jin, X. J. Chen, T. Cui, H. K. Mao, H. Zhang, Q. Zhuang, K. Bao, D. Zhou, B. Liu, Q. Zhou and Z. He, Proceedings of the National Academy of Sciences, 113, 2366 (2016).
  * (49) W. E. Pickett, Physical review letters 77, 3185 (1996).


# Photometric Objects Around Cosmic Webs (PAC) Delineated in a Spectroscopic
survey. II. Morphology, Color and Size Dependences of the Stellar-halo Mass
Relation for Massive Galaxies

Kun Xu Department of Astronomy, School of Physics and Astronomy, Shanghai Jiao
Tong University, Shanghai, 200240, People’s Republic of China  Yipeng Jing
Department of Astronomy, School of Physics and Astronomy, Shanghai Jiao Tong
University, Shanghai, 200240, People’s Republic of China  Tsung-Dao Lee
Institute, and Shanghai Key Laboratory for Particle Physics and Cosmology,
Shanghai Jiao Tong University, Shanghai, 200240, People’s Republic of China

(Received XXX XX, XXXX; Revised XXX XX, XXXX; Accepted XXX XX, XXXX)

###### Abstract

In this paper, we report a robust measurement of the morphology, color and
galaxy size dependences of the stellar-halo mass relation (SHMR) at the high
mass end ($10^{11.3}{\rm M_{\odot}}<M_{\star}<10^{11.7}{\rm M_{\odot}}$) at
redshift $z_{s}\sim 0.6$111Throughout the paper, we use $z_{s}$ for redshift,
$z$ for the z-band magnitude.. Applying our method, Photometric objects Around
Cosmic webs (PAC), developed in a previous work to CMASS and HSC-SSP
observations, we measure the excess surface density
($\bar{n}_{2}w_{p}(r_{p})$) of satellites around massive central galaxies with
different morphologies indicated by Sérsic index $n$. We find that more
compact (larger $n$) central galaxies are surrounded by more satellites. With
the abundance matching method, we estimate halo mass for the central galaxies,
and find that halo mass is increased monotonically with $n$, solid evidence
for a morphology dependence of SHMR. Specifically, our results show that the
most compact galaxies ($n>6$) have the halo mass around 5.5 times larger than
the disk galaxies ($n<2$). Similarly, using the effective radius $R_{e}$ and
the rest-frame $u-r$ color, we find that red (large) galaxies reside in halos
that are in average $2.6$ ($2.3$) times more massive than those hosting blue
(small) galaxies.

Galaxy formation(595) — Galaxy dark matter halos (1880)

††journal: ApJ

##  1 Introduction

The stellar-halo mass relation (SHMR) is one of the most fundamental relations
in galaxy-halo connection (see Wechsler & Tinker (2018) for a review), in
which larger dark matter halos host more massive galaxies with a relatively
tight scatter. SHMR has been measured with different data and methods in the
past decade (Zheng et al., 2007; Guo et al., 2010; Wang & Jing, 2010; Yang et
al., 2012; Moster et al., 2013; Behroozi et al., 2019), and it reflects the
sophisticated regulation of complex physical processes in galaxy formation
such as gas accretion and feedback processes at different mass scales.

The SHMR is a relation between the halo mass and galactic stellar mass, which
is assumed to be independent of other properties of halos or galaxies in early
studies. However, recent studies (e.g., Cooper et al. (2010); Zentner et al.
(2014); Zu et al. (2021a, b)) have paid more attention on whether galaxies
with different properties have different SHMRs, which is also related to so-
called galaxy assembly bias that causes the scatter in the SHMR. Till now,
there is no consensus about the dependence on other galaxy properties. For
example, some studies found that red central galaxies reside in more massive
halos than blue ones (Cooper et al., 2010; Wang et al., 2013; Zentner et al.,
2014; Rodríguez-Puebla et al., 2015; Hearin et al., 2015; Mandelbaum et al.,
2016), while others found an opposite relation that blue galaxies have more
massive host halos (Tinker et al., 2013; Moster et al., 2018; Guo et al.,
2019). Using HI rotation curves, Posti et al. (2019) showed that local massive
spiral galaxies are hosted by smaller dark matter halos, implying a morphology
dependence of SHMR. Moreover, this dependence does not show up in some of
current cosmological hydrodynamical simulations (Marasco et al., 2020), while
it does exist in some others (e.g., Cui et al. (2021)). The galaxy size
dependence of SHMR has also been investigated in some studies (Charlton et
al., 2017; Desmond et al., 2017; Somerville et al., 2018) but a clear answer
to this question does not exist yet. In summary, it is still under hot debate
both in observations and in theories if there exist secondary parameters for
the SHMR other than the stellar mass.

In this paper, using the method named Photometric objects Around Cosmic webs
(PAC) developed in the first paper of this series (Xu et al., 2021), we
measure the dependences of SHMR on morphology, galaxy size and color for
massive galaxies, and report robust detections of both dependences, which
shows unambiguously that large, red or more compact galaxies are located in
more massive halos.

We adopt the cosmology with $\Omega_{m}=0.268$, $\Omega_{\Lambda}=0.732$ and
$H_{0}=71{\rm\ km/s/Mpc}$ through out the paper.

##  2 Data and method

In Xu et al. (2021, hearafter Paper I), we developed a method for estimating
the projected density distribution $\bar{n}_{2}w_{p}(r_{p})$ of photometric
objects around spectroscopic objects in a spectroscopic survey, where
$\bar{n}_{2}$ is the mean number density of the photometric objects and
$w_{p}$ is the projected cross-correlation function (PCCF). This quantity
describes the distribution of Photometric sources with certain physical
properties (e.g. luminosity, mass, color etc) Around Cosmic webs (PAC) traced
by the spectroscopic objects. Since the halo mass is highly correlated with
the satellite distribution (Wang et al., 2021), PAC is very powerful for
studying SHMR. For details of the method, we refer the readers to Paper I.

Figure 1: Rest-frame $u-r$ color vs. morphology diagram (left) and $R_{e}$ vs.
morphology diagram (right) of the CMASS sample. Figure 2: Normalized
distribution function of CMASS galaxies with different morphologies (left),
colors and sizes (right) as a function of stellar mass.

As in Paper I, we use the HSC-SSP PDR2 wide field photometric catalog (Aihara
et al., 2019) as the photometric sample. To obtain more accurate physical
properties, we choose sources in the footprints observed with all five bands
($grizy$) to ensure that there are enough bands for SED. Sources around bright
objects are masked using {grizy}_mask_pdr2_bright_objectcenter flag provided
by the HSC collaboration (Coupon et al., 2018). And we use the
{grizy}_extendedness_value flag to exclude stars in the sample. Finally, there
are around $2\times 10^{8}$ galaxies in our photometric sample. We can also
construct a random point catalog ($100/arcmin^{2}$) with the same selection
criteria from the HSC database for PAC analysis. The effective area calculated
from the random point number is $501\ deg^{2}$.

We use the CMASS sample in the Baryon Oscillation Spectroscopic Survey (BOSS;
Ahn et al. (2012); Bolton et al. (2012)) that consists of massive galaxies
with $i<19.9\ mag$ as spectroscopic sample (population 1). We first select
galaxies in the redshift range of $0.5<z_{s}<0.7$, and then cross match it
with the HSC photometric sample we constructed above and The DESI Legacy
Imaging Surveys DR9 catalog (Dey et al., 2019). After that, we get the
magnitudes in seven bands $grizyW1W2$ for each CMASS galaxy in the footprint
of HSC. Then we calculate the physical properties (e.g. stellar mass and rest-
frame colors) for these galaxies using the spectral energy distribution (SED)
fitting code CIGALE (Boquien et al., 2019). The details of SED have been
discribed in Paper I. As noticed by previous studies (Maraston et al., 2013;
Leauthaud et al., 2016; Guo et al., 2018), the CMASS sample is complete to
stellar mass $M_{*}\approx 10^{11.3}{\rm M_{\odot}}$. Therefore, we adopt a
stellar mass cut at $10^{11.3}{\rm M_{\odot}}$ in this study. Moreover, we
only consider central galaxies in the spectroscopic sample, so we select
galaxies which do not have more massive neighbors within the projected
distance of $1Mpc\ h^{-1}$. Finally, there are $8028$ massive ($>10^{11.3}{\rm
M_{\odot}}$) central galaxies left in the spectroscopic sample.

Figure 3: Measurements of $\bar{n}_{2}w_{p}(r_{p})$. Each panel shows the
results for the same photometric mass bin and for CMASS subsamples divided
according to their morphologies (top), colors (middle) and sizes (bottom).
Only three morphology bins are shown for to better illustrate the results.

To study the morphology and size of galaxies, we fit the HSC $z$ band images
with the Sérsic profile (Sérsic, 1963) for all the galaxies in our
spectroscopic sample. The radial luminosity distribution of a galaxy is
described by the Sérsic form:

$I(R)=I_{e}\exp\bigg{\\{}-b_{n}\bigg{[}\bigg{(}\frac{R}{R_{e}}\bigg{)}^{1/n}-1\bigg{]}\bigg{\\}},$
(1)

where $b_{n}$ satisfies:

$\gamma(2n;b_{n})=\frac{1}{2}\Gamma(2n).$ (2)

$I_{e}$ is the intensity at the half-light radius $R_{e}$, and $n$, called the
Sérsic index, describes the compactness of a luminosity profile. $\Gamma$ and
$\gamma$ are respectively the Gamma function and lower incomplete Gamma
function. Generally speaking, disk galaxies have exponential profiles with
$n=1$, and ellipticals follow the de Vaucouleurs profiles with $n=4$. The 2D
image fitting is performed using galfit (Peng et al., 2002) with the point
spread function (PSF) taken into account. Other sources falling in the fitting
regions are detected and masked out using sextractor (Bertin & Arnouts, 1996).
Since an ongoing merger with companions may lead to a meaningless fitting
result with a low $n$, we visually inspect the images of galaxies with $n<2$
and abandon those with apparent major mergers.

In the left panel of Figure 1, we show the rest-frame $u-r$ color vs.
morphology diagram of the CMASS sample. Colors of galaxies is distributed
bimodally and weakly correlated with $n$. We adopt $u-r=2.25$ as the color cut
for blue and red galaxies with the average Sérsic index $3.06$ and $4.86$
respectively. The right panel of Figure 1 shows the galaxy size $R_{e}$ vs.
morphology diagram, in which the galaxy size slightly increases with
increasing $n$. We use the median galaxy size $R_{e}=9.7\ kpc$ as the dividing
line for large and small size galaxies.

In Figure 2, we show the stellar mass distributions for galaxies with
different morphologies (left), colors and sizes (right). The mass distribution
is nearly the same for galaxies with different morphologies. Therefore, the
halo mass difference for galaxies with different morphologies, as we will show
in the next section, is independent of the stellar mass. However, large and/or
red galaxies tend to be more massive than small and/or blue ones. To eliminate
the stellar mass dependence, we construct a controlled sample of large and red
galaxies that has the same stellar mass distribution as the small and blue one
(dashed lines), and will use it in the following analysis.

##  3 Results

Following Paper I, we calculate $\bar{n}_{2}w_{p}(r_{p})$ between the
photometric sample in 2 mass bins $[10^{9.5},10^{10.0},10^{11.0}]{\rm
M_{\odot}}$ and the spectroscopic sample within the stellar mass range
$10^{11.3}{\rm M_{\odot}}<M_{\star}<10^{11.7}{\rm M_{\odot}}$ that contains
most of the CMASS galaxies. In addition, to study the morphology, color and
size dependences, we divide the spectroscopic sample into 6 sub-samples
according to $n$, 2 sub-samples according to $u-r$ color or $R_{e}$. Errors
are estimated using jackknife resampling by further dividing each
spectroscopic sub-samples into 50 sub-samples. Completeness of the photometric
sample is considered using the z-band completeness limit $C_{95}(M_{\star})$
defined in Paper I (See their Figure 1).

Figure 4: Average halo mass estimated from abundance matching for central
galaxies with different morphologies, colors and sizes. Green dots show the
halo mass for different morphologies, blue and red squares show the halo mass
and average $n$ for blue and red galaxies, and blue and red triangles show the
halo mass and average $n$ for galaxies with small and large sizes.

The measurements are shown in Figure 3. Each panel shows the results for the
same mass bin but with different morphologies, colors or sizes. To better
illustrate the results, we only show the measurements for three morphology
bins while the conclusion hold for all six morphology bins. The measurements
are overall robust within $0.1\ {\rm Mpc}\ h^{-1}<r_{p}<10\ {\rm Mpc}\
h^{-1}$. For galaxies with different morphologies, $\bar{n}_{2}w_{p}(r_{p})$
increases with $n$, indicating that galaxies with more compact structure have
more satellites. Since the halo mass of central galaxies is highly correlated
with their satellite distribution, it implies that compact galaxies reside in
more massive dark matter halos. Similarly, red (large) galaxies have higher
$\bar{n}_{2}w_{p}(r_{p})$ than their blue (small) counterparts, and are
expected to have larger host halos.

To quantify the halo mass difference between galaxies with different
morphologies, colors and sizes, we use abundance matching (AM, Wang & Jing
(2010); Moster et al. (2013); Behroozi et al. (2019)) to estimate the halo
mass of galaxies. As in Paper I, we use the $\Lambda$CDM CosmicGrowth
Simulation (Jing & Suto, 2002; Jing, 2019) with cosmological parameters
$\Omega_{m}=0.268$, $\Omega_{\Lambda}=0.732$ and $\sigma_{8}=0.831$. The box
size is $600\ {\rm Mpc}\ h^{-1}$ with $3072^{3}$ dark matter particles and
softening length $\eta=0.01\ {\rm Mpc}\ h^{-1}$. The SHMR can be described by
a formula of double power-law form:

$M_{\star}=\left[\frac{2}{(\frac{M_{acc}}{M_{0}})^{-\alpha}+(\frac{M_{acc}}{M_{0}})^{-\beta}}\right]k\,,$
(3)

where $M_{acc}$ is defined as the viral mass $M_{vir}$ of the halo at the time
when the galaxy was last the central dominant object. We use the fitting
formula in Bryan & Norman (1998) to find $M_{vir}$. The scatter in
$\log(M_{*})$ at a given $M_{acc}$ is described with a Gaussian function of
the width $\sigma$. We populate galaxies to halos using results from Paper I
with $M_{0}=10^{11.65}{\rm M_{\odot}}h^{-1}$, $\alpha=0.33$, $\beta=2.39$,
$k=10^{10.20}{\rm M_{\odot}}$ and $\sigma=0.22$.

With the assumption that the halo mass is completely determined by the
satellite distribution (Han et al., 2016; Wang et al., 2021) and with the mock
galaxy catalog constructed above, the mean halo mass of central galaxies with
specific properties can be estimated through matching the observed excess
surface density ($\bar{n}_{2}w_{p}(r_{p})$) to halos with certain halo mass in
simulation. To compare observation with simulation, we define:

$\chi^{2}=\frac{1}{N_{p}}\sum_{N_{p}}\left[\frac{\log(\bar{n}_{2}w_{p}(r_{p}))_{sim}-\log(\bar{n}_{2}w_{p}(r_{p}))_{ob}}{\sigma(\log(\bar{n}_{2}w_{p}(r_{p}))_{ob})}\right]^{2}\,,$
(4)

where $N_{p}$ is the total number of points over which
$\bar{n}_{2}w_{p}(r_{p})$ is compared. We use the Markov chain Monte Carlo
(MCMC) sampler emcee (Foreman-Mackey et al., 2013) to perform a maximum
likelihood analysis.

Halo mass estimated from AM for the spectroscopic sample for different
morphologies and colors is shown in Figure 4. For morphology (green dots)
dependence, halo mass increases monotonically with Sérsic index $n$, and the
most compact galaxies ($n>6$) have the halo mass around $5.5$ times larger
than the disk galaxies ($n<2$). For color (squares), red galaxies reside in
halos $2.6$ times more massive than those hosting blue galaxies. And for
galaxy size. large galaxies reside in halos $2.3$ times more massive than
those hosting small galaxies.

##  4 conclusion and Discussion

In this paper, we report the morphology, color and size dependences of the
SHMR for massive ($10^{11.3}{\rm M_{\odot}}<M_{\star}<10^{11.7}{\rm
M_{\odot}}$) central galaxies. Using CMASS and HSC-SSP samples and with PAC
developed in Paper I, we calculate the excess surface density
$\bar{n}_{2}w_{p}(r_{p})$ of satellites and neighbours for the massive central
galaxies with different morphologies ($n$), colors ($u-r$) and sizes
($R_{e}$). We find that, at the same stellar mass, galaxies with more compact
morphology, red color and/or large size are surrounded by more satellites.
Using AM, we estimate the halo mass for central galaxies with different
morphologies, colors and sizes. The results show that more compact, red and/or
large central galaxies reside in more massive halos. Specifically, the most
compact galaxies ($n>6$) have the halo mass $5.5$ times larger than the disk
ones ($n<2$), red galaxies reside in halos $2.6$ times more massive than the
blue counterparts, and galaxies with large size have the halo mass $2.3$ times
larger than galaxies with small size.

The physical origin of the morphology or color dependence of SHMR is still not
well understood. One possible picture is that, for the same halo mass, compact
galaxies assemble and transform their morphology earlier and then their star
formation is quenched by mechanisms such as active galactic nucleus (AGN)
feedback. After the quenching, their halos stills keep growing under the
hierarchical formation framework. And for more disky galaxies, the star
formation is quenched later or even continues, resulting in a larger stellar
mass at a fixed halo mass. Our results also show that the morphology and color
dependences are in some way degenerate, and it seems the morphology dependence
is more fundamental at least for the massive galaxies.

The size dependence of SHMR may be explained by the minor merger scenario
(Naab et al., 2009; Hopkins et al., 2010) that the rate of minor mergers is
higher in more massive halos and minor mergers result in size increase. If
minor mergers are more frequent in more massive halos, this may introduce a
halo mass-size correlation at fixed stellar mass.

In future, with larger photometric and spectroscopic samples, we plan to check
whether the morphology, color and size dependences of SHMR hold for lower mass
central galaxies. We also plan to verify the estimated halo mass from AM using
weak lensing measurements.

The work is supported by NSFC (12133006, 11890691, 11621303) and by 111
project No. B20019. We gratefully acknowledge the support of the Key
Laboratory for Particle Physics, Astrophysics and Cosmology, Ministry of
Education. This work made use of the Gravity Supercomputer at the Department
of Astronomy, Shanghai Jiao Tong University.

## References

  * Ahn et al. (2012) Ahn, C. P., Alexandroff, R., Allende Prieto, C., et al. 2012, ApJS, 203, 21, doi: 10.1088/0067-0049/203/2/21
  * Aihara et al. (2019) Aihara, H., AlSayyad, Y., Ando, M., et al. 2019, PASJ, 71, 114, doi: 10.1093/pasj/psz103
  * Behroozi et al. (2019) Behroozi, P., Wechsler, R. H., Hearin, A. P., & Conroy, C. 2019, MNRAS, 488, 3143, doi: 10.1093/mnras/stz1182
  * Bertin & Arnouts (1996) Bertin, E., & Arnouts, S. 1996, A&AS, 117, 393, doi: 10.1051/aas:1996164
  * Bolton et al. (2012) Bolton, A. S., Schlegel, D. J., Aubourg, É., et al. 2012, AJ, 144, 144, doi: 10.1088/0004-6256/144/5/144
  * Boquien et al. (2019) Boquien, M., Burgarella, D., Roehlly, Y., et al. 2019, A&A, 622, A103, doi: 10.1051/0004-6361/201834156
  * Bryan & Norman (1998) Bryan, G. L., & Norman, M. L. 1998, ApJ, 495, 80, doi: 10.1086/305262
  * Charlton et al. (2017) Charlton, P. J. L., Hudson, M. J., Balogh, M. L., & Khatri, S. 2017, MNRAS, 472, 2367, doi: 10.1093/mnras/stx2068
  * Cooper et al. (2010) Cooper, M. C., Gallazzi, A., Newman, J. A., & Yan, R. 2010, MNRAS, 402, 1942, doi: 10.1111/j.1365-2966.2009.16020.x
  * Coupon et al. (2018) Coupon, J., Czakon, N., Bosch, J., et al. 2018, PASJ, 70, S7, doi: 10.1093/pasj/psx047
  * Cui et al. (2021) Cui, W., Davé, R., Peacock, J. A., Anglés-Alcázar, D., & Yang, X. 2021, Nature Astronomy, doi: 10.1038/s41550-021-01404-1
  * Desmond et al. (2017) Desmond, H., Mao, Y.-Y., Wechsler, R. H., Crain, R. A., & Schaye, J. 2017, MNRAS, 471, L11, doi: 10.1093/mnrasl/slx093
  * Dey et al. (2019) Dey, A., Schlegel, D. J., Lang, D., et al. 2019, AJ, 157, 168, doi: 10.3847/1538-3881/ab089d
  * Foreman-Mackey et al. (2013) Foreman-Mackey, D., Hogg, D. W., Lang, D., & Goodman, J. 2013, PASP, 125, 306, doi: 10.1086/670067
  * Guo et al. (2018) Guo, H., Yang, X., & Lu, Y. 2018, ApJ, 858, 30, doi: 10.3847/1538-4357/aabc56
  * Guo et al. (2019) Guo, H., Yang, X., Raichoor, A., et al. 2019, ApJ, 871, 147, doi: 10.3847/1538-4357/aaf9ad
  * Guo et al. (2010) Guo, Q., White, S., Li, C., & Boylan-Kolchin, M. 2010, MNRAS, 404, 1111, doi: 10.1111/j.1365-2966.2010.16341.x
  * Han et al. (2016) Han, J., Cole, S., Frenk, C. S., & Jing, Y. 2016, MNRAS, 457, 1208, doi: 10.1093/mnras/stv2900
  * Hearin et al. (2015) Hearin, A. P., Watson, D. F., & van den Bosch, F. C. 2015, MNRAS, 452, 1958, doi: 10.1093/mnras/stv1358
  * Hopkins et al. (2010) Hopkins, P. F., Bundy, K., Hernquist, L., Wuyts, S., & Cox, T. J. 2010, MNRAS, 401, 1099, doi: 10.1111/j.1365-2966.2009.15699.x
  * Jing (2019) Jing, Y. 2019, Science China Physics, Mechanics, and Astronomy, 62, 19511, doi: 10.1007/s11433-018-9286-x
  * Jing & Suto (2002) Jing, Y. P., & Suto, Y. 2002, ApJ, 574, 538, doi: 10.1086/341065
  * Leauthaud et al. (2016) Leauthaud, A., Bundy, K., Saito, S., et al. 2016, MNRAS, 457, 4021, doi: 10.1093/mnras/stw117
  * Mandelbaum et al. (2016) Mandelbaum, R., Wang, W., Zu, Y., et al. 2016, MNRAS, 457, 3200, doi: 10.1093/mnras/stw188
  * Marasco et al. (2020) Marasco, A., Posti, L., Oman, K., et al. 2020, A&A, 640, A70, doi: 10.1051/0004-6361/202038326
  * Maraston et al. (2013) Maraston, C., Pforr, J., Henriques, B. M., et al. 2013, MNRAS, 435, 2764, doi: 10.1093/mnras/stt1424
  * Moster et al. (2013) Moster, B. P., Naab, T., & White, S. D. M. 2013, MNRAS, 428, 3121, doi: 10.1093/mnras/sts261
  * Moster et al. (2018) —. 2018, MNRAS, 477, 1822, doi: 10.1093/mnras/sty655
  * Naab et al. (2009) Naab, T., Johansson, P. H., & Ostriker, J. P. 2009, ApJ, 699, L178, doi: 10.1088/0004-637X/699/2/L178
  * Peng et al. (2002) Peng, C. Y., Ho, L. C., Impey, C. D., & Rix, H.-W. 2002, AJ, 124, 266, doi: 10.1086/340952
  * Posti et al. (2019) Posti, L., Fraternali, F., & Marasco, A. 2019, A&A, 626, A56, doi: 10.1051/0004-6361/201935553
  * Rodríguez-Puebla et al. (2015) Rodríguez-Puebla, A., Avila-Reese, V., Yang, X., et al. 2015, ApJ, 799, 130, doi: 10.1088/0004-637X/799/2/130
  * Sérsic (1963) Sérsic, J. L. 1963, Boletin de la Asociacion Argentina de Astronomia La Plata Argentina, 6, 41
  * Somerville et al. (2018) Somerville, R. S., Behroozi, P., Pandya, V., et al. 2018, MNRAS, 473, 2714, doi: 10.1093/mnras/stx2040
  * Tinker et al. (2013) Tinker, J. L., Leauthaud, A., Bundy, K., et al. 2013, ApJ, 778, 93, doi: 10.1088/0004-637X/778/2/93
  * Wang & Jing (2010) Wang, L., & Jing, Y. P. 2010, MNRAS, 402, 1796, doi: 10.1111/j.1365-2966.2009.16007.x
  * Wang et al. (2013) Wang, L., Weinmann, S. M., De Lucia, G., & Yang, X. 2013, MNRAS, 433, 515, doi: 10.1093/mnras/stt743
  * Wang et al. (2021) Wang, W., Li, X., Shi, J., et al. 2021, arXiv e-prints, arXiv:2104.05355.  https://arxiv.org/abs/2104.05355
  * Wechsler & Tinker (2018) Wechsler, R. H., & Tinker, J. L. 2018, ARA&A, 56, 435, doi: 10.1146/annurev-astro-081817-051756
  * Xu et al. (2021) Xu, K., Zheng, Y., & Jing, Y. 2021, arXiv e-prints, arXiv:2109.11738.  https://arxiv.org/abs/2109.11738
  * Yang et al. (2012) Yang, X., Mo, H. J., van den Bosch, F. C., Zhang, Y., & Han, J. 2012, ApJ, 752, 41, doi: 10.1088/0004-637X/752/1/41
  * Zentner et al. (2014) Zentner, A. R., Hearin, A. P., & van den Bosch, F. C. 2014, MNRAS, 443, 3044, doi: 10.1093/mnras/stu1383
  * Zheng et al. (2007) Zheng, Z., Coil, A. L., & Zehavi, I. 2007, ApJ, 667, 760, doi: 10.1086/521074
  * Zu et al. (2021a) Zu, Y., Shan, H., Zhang, J., et al. 2021a, MNRAS, 505, 5117, doi: 10.1093/mnras/stab1712
  * Zu et al. (2021b) Zu, Y., Song, Y., Shao, Z., et al. 2021b, arXiv e-prints, arXiv:2108.06790.  https://arxiv.org/abs/2108.06790


# Naming, Describing, and Quantifying Visual Objects in Humans and LLMs

Alberto Testoni
ILLC, University of Amsterdam
<EMAIL_ADDRESS>Juell Sprott
University of Amsterdam
<EMAIL_ADDRESS>Sandro Pezzelle
ILLC, University of Amsterdam
<EMAIL_ADDRESS>
###### Abstract

While human speakers use a variety of different expressions when describing
the same object in an image, giving rise to a distribution of plausible labels
driven by pragmatic constraints, the extent to which current Vision & Language
Large Language Models (VLLMs) can mimic this crucial feature of language use
is an open question. This applies to common, everyday objects, but it is
particularly interesting for uncommon or novel objects for which a category
label may be lacking or fuzzy. Furthermore, humans show clear production
preferences for highly context-sensitive expressions, such as the quantifiers
‘few’ or ‘most’. In our work, we evaluate VLLMs (FROMAGe, BLIP-2, LLaVA) on
three categories (nouns, attributes, and quantifiers) where humans show great
subjective variability concerning the distribution over plausible labels,
using datasets and resources mostly under-explored in previous work. Our
results reveal mixed evidence on the ability of VLLMs to capture human naming
preferences, with all models failing in tasks that require high-level
reasoning such as assigning quantifiers.

Naming, Describing, and Quantifying Visual Objects in Humans and LLMs

Alberto Testoni ILLC, University of Amsterdam<EMAIL_ADDRESS>Juell Sprott
University of Amsterdam<EMAIL_ADDRESS>

Sandro Pezzelle ILLC, University of Amsterdam<EMAIL_ADDRESS>

##  1 Introduction

Figure 1: Datasets used and distribution of human answers/labels. In NOUN
(left), we focus on the frequency of color and texture attributes in the
descriptions. In ManyNames (middle), each object is associated with the
frequency of the nouns used to describe it. In QUANT (right), each image is
associated with a probability distribution over a list of quantifiers that
humans selected when answering ‘How many of the objects are animals?’.

Recent years have witnessed increasing popularity in the development of Large
Language Models (LLMs) given their notable performance in following
instructions, answering questions, and in many reasoning tasks, serving as
general-purpose assistants Huang and Chang (2023); Zhao et al. (2023). In
parallel, a new generation of powerful Vision and Language LLMs (VLLMs) with
excellent visual understanding and generation capabilities have emerged Gan et
al. (2022); Li et al. (2023a). Rapidly, these models have outperformed
previous approaches in many downstream tasks. In our work, we focus on the
Natural Language Generation skills of powerful VLLMs by analyzing an important
but under-explored problem, namely, their ability to capture human production
variability (in terms of distribution over plausible labels/descriptions) in
naming tasks.

Previous work highlighted that speakers display a wide range of variability
when asked to utter sentences, resulting in inter-speaker variability but also
variability over time for the same speaker (Levelt, 1993; Fan et al., 2018;
Alva-Manchego et al., 2021; Takmaz et al., 2024). In particular, in object
naming, speakers may refer to objects appearing in a visual scene in many
different ways (Graf et al., 2016). Objects generally belong to multiple
categories/super-categories, and all the lexicalized labels of such categories
are valid (Brown, 1958). However, although multiple labels are valid, humans
pragmatically adapt their naming preferences depending on the context (Olson,
1970; Rohde et al., 2012), resulting in some labels being more frequently
uttered than others. For instance, ‘mammal’ is a correct label to describe a
Gold Retriever, but pragmatically less likely than ‘dog’. Similarly, speakers
tend to prefer sub-ordinate words like ‘car’ instead of the potentially
ambiguous super-ordinate word ‘vehicle’ in case multiple vehicles appear in
the image. In our work, we are interested in capturing both these two
features: while many labels are equally valid and acceptable when naming or
describing entities, these labels distribute according to a certain likelihood
distribution.

In our work, we investigate this issue in three different production
conditions. First of all, we consider the ManyNames dataset (Silberer et al.,
2020a, b), where annotators assign labels to describe common objects in images
in a referential expression generation setting (Yu et al., 2016; Kazemzadeh et
al., 2014). We also explore two additional resources that have not received
much attention within the NLP community and that allow us to broaden the
horizons of this phenomenon. First, we analyze the NOUN dataset (Horst and
Hout, 2016), where speakers describe uncommon and novel objects: we focus on
both the choice of the adjectives and how they distribute in the across-
subject distribution. Finally, we investigate human production variability
arising from the context-sensitive nature of non-numerical quantifiers using
the data collected by Pezzelle et al. (2018).

We evaluate three VLLMs (FROMAGe, BLIP-2, LLaVA) on the above-mentioned tasks
in a zero-shot setting. We sample multiple times from the model using nucleus
sampling, mimicking various human speakers, and compare the generated samples
against human production patterns using different metrics (Jensen–Shannon
divergence and Pearson’s correlation, depending on the task at hand). Our
results show that models weakly to moderately mimic human distributions in
naming common and uncommon objects. Instead, they fail to mimic human
distributions when selecting quantifiers, as highlighted by our in-depth
analyses.

##  2 Tasks and Datasets

We use the images and corresponding human labels or descriptions from three
datasets in English, that we briefly describe below.

#### NOUN

The Novel Object and Unusual Name (NOUN) dataset (Horst and Hout, 2016)
contains 64 images of multipart, multicolored, and three-dimensional uncommon
and novel objects. The dataset was originally created for behavioral studies
on word learning and, to the best of our knowledge, it has not been used for
NLP research. We focus on the naming task, where participants were asked to
answer the question “What would you call this object?”. The answers are
sentences like: ‘a plastic object with red stuff on top’. For each object, the
proportion of colors (e.g., ‘red’, ‘bronze’) and textures (e.g., ‘soft’,
‘rough’) was calculated as the number of attributes given the number of
responses. An example from the dataset is reported in Figure 1 (left),
together with the ratio of colors and textures in human responses. In NOUN, we
examine human production preferences on a high level, by just looking at the
frequency according to which certain adjectives are used.

#### ManyNames

In ManyNames (Silberer et al., 2020a, b), the authors collected names for 25K
objects from VisualGenome (Krishna et al., 2017) by asking human annotators.
Each object is associated with an average number of 35.3 annotations. More
than 90% of the objects are associated with more than one unique label (5.7
average name types per object). An example is shown in Figure 1 (middle). When
describing the object in the red box, most annotators referred to it as
‘chair’, while around 30% said ‘sofa’, and the remaining ones used ‘couch’ and
‘armchair’. The images in ManyNames are classified into 7 domains (e.g.,
vehicles, people, etc.): for computational constraints, we evaluated 300
randomly sampled objects from each domain. Differently from NOUN, here we
examine production preferences on a more fine-grained level using the actual
distribution over multiple labels.

#### QUANT

To study how quantifiers are used when referring to quantities grounded in
images, Pezzelle et al. (2018) introduced a dataset of visual abstract scenes
containing a variable number of animals and artifacts and asked human
participants to answer the question “How many of the objects are animals?".
Participants could select the answer from a list of nine quantifiers: ‘none’,
‘almost none’, ‘the smaller part’, ‘few’, ‘some’, ‘many’, ‘most’, ‘almost
all’, ‘all’. The authors used images with 17 different proportions of animals
and artifacts (ranging from 0% to 100%). In our work, we tested 50 images for
each of the 17 proportions in the dataset, resulting in a total number of 850
images.111The actual images used in our experiment come from Testoni et al.
(2019), which built a large-scale dataset using the stimuli and pipeline by
Pezzelle et al. (2018).

##  3 Experiments

###  3.1 Generation

In our work, we test the performance of three models in a zero-shot setting:
BLIP-2 (Li et al., 2023b), FROMAGe (Koh et al., 2023), and LLaVA (Liu et al.,
2023b, a). All three models can be prompted for zero-shot generation.
Additional details are discussed in Appendix A.4. For each of the three tasks
described in Section 2, we used prompts that resembled the instructions
provided to human annotators during the dataset collection. ManyNames: Q:
Please name the object in the red box with the first name that comes to mind.
A:. NOUN: Q: What do you see in the image? A:. QUANT: Question: How many
animals are there in the image? Pick the best among the following options: ,
followed by the list of the nine quantifiers, each associated with a letter
(from A to I). The ordering of the quantifiers is randomized at each inference
step. We sample multiple times from each model using nucleus sampling decoding
(Holtzman et al., 2019), with $p=0.9$, $t=0.5$ (different hyperparameter
configurations did not significantly affect the overall results). For each
task, we sample the model 20 times and filter out ill-formed answers, such as
empty strings or question repetitions. After filtering, we randomly take 10
generations per image for ManyNames and NOUN, and 15 for QUANT. In this way,
we have the same number of generations for each image/object. We will release
our code upon acceptance at:
https://github.com/albertotestoni/ndq_visual_objects.

###  3.2 Evaluation

Each object in NOUN is associated with color and texture saliency, i.e., how
often speakers described the object using these attributes. We use a string-
match approach (see Appendix A.2) to analyze the model output and compute
color and texture saliency. We compute the Pearson’s r correlation between
human and model saliency, considering all objects. Each object in ManyNames is
associated with $H$ unique nouns assigned by human annotators and $M$ unique
nouns sampled from the model output, together with their frequency. Given
$A=H\cup M$, we construct two term-frequency vectors for human and model
output, $h$ and $m$, respectively, with $|h|=|m|=|A|$. Each noun in $A$ is
mapped to a unique position in $h$ and $m$ and each vector is filled with its
normalized frequency. We evaluate the models by computing the inverse
Jensen–Shannon (JS, bounded between 0 and 1) divergence (Lin, 1991) between
$h$ and $m$. See Figure 5 in the Appendix for an example. Each image in QUANT
is associated with a probability distribution over 9 quantifiers, depending on
the proportion of animals and artifacts. From the model outputs, we extract
the relative frequency of each quantifier and compute Pearson’s r correlation
with the human distribution. We then average the correlation results over all
images. Correlation is bounded between -1 and 1. Higher is better for all the
metrics.

Figure 2: NOUN, QUANT: correlation between human responses and model samples.
ManyNames: inverse JS divergence between the frequency of the nouns chosen by
annotators and the ones generated by the model.

###  3.3 Results

As we can observe from Figure 2, the results for ManyNames and NOUN (color
saliency) show a clear trend: all the models correlate, to some extent, with
human production, with LLAVA obtaining the highest correlations for both tasks
(around 0.5) and significantly outperforming (t-test, $p<0.01$) both BLIP2 and
FROMAGe.222Appendix A.1 shows per-domain results for ManyNames. These findings
combine with previous work showing the primacy of LLaVA over other models (Liu
et al., 2023b, a) However, the remaining tasks show critical weaknesses for
all models. First, none of the models achieve a statistically significant
correlation for texture saliency ($p>0.05$). We conjecture that texture
attributes are less common for the models compared to colors, and thus they
may be less accurate when generating them: we leave an in-depth analysis of
this issue for future work. Despite the correlation results being similar
across models, our manual inspection reveals interesting differences: while
the low performance of FROMAGe is due to an under-generation of texture
attributes, the opposite is true for LLaVA, with BLIP-2 being more flexible in
terms of texture attribute generation but not aligned with human variability
(see Figure 6 for an example). Finally, all models show almost no correlation
in assigning quantifiers to visual scenes, highlighting a severe limitation of
all models on this task. We scrutinize this issue in the following Section.

Figure 3: Correlation results (y-axis) broken down by proportion of targets
(animals) in the image (x-axis).

##  4 The Curious Case of Quantifiers

We run some analyses to investigate the poor performance of all models in the
QUANT task. First of all, we acknowledge that the multiple-choice prompting
used in QUANT is different and more complex than the prompts used for the
other datasets. Still, it is unlikely that this is the main reason behind the
poor performance of all models. Figure 3 shows the correlation results broken
down by the proportion of animals in the image. We observe that even though
the overall correlation results are similar across models (Figure 2), models
perform quite differently depending on the proportion of animals in the scene.
While BLIP-2 performs relatively well on the ‘extreme’ proportions (no animals
or all animals in the image, when speakers generally choose the quantifiers
‘none’ and ‘all’, respectively), LLaVA excels at intermediate proportions, and
FROMAGe performs better on proportions above 50%. Can we conclude that models
properly perform the task of assigning the most likely quantifiers for some
proportions? These results evoke two hypotheses: (a) The models are capable of
selecting plausible quantifiers only for some proportions or, vice versa, they
understand only some of the quantifiers analyzed; (b) The models have a bias
towards some specific quantifiers, regardless of the proportion of targets in
the scene, leading to a decent perform on some proportions as a side effect.
Our additional analyses, reported in Figures 7 and 8 in the Appendix, support
hypothesis (b): FROMAGe has a strong bias towards selecting the quantifier
‘many’; BLIP-2 frequently selects the extreme quantifiers ‘none’ and ‘all’,
and its selection is not influenced by the proportion of targets; LLaVA has a
bias towards selecting the quantifier ‘some’, regardless of the proportion of
targets. To further shed light on this result, we qualitatively assess the
‘counting’ skills of the models, a crucial skill to succeed in assigning
quantifiers. As the examples in Figure 9 in the Appendix illustrate, all
models struggle to successfully count how many animals appear in the image. We
hypothesize that the reason for the poor performance in assigning quantifiers
lies in the quantity estimation and comparison skills of the models.

##  5 Conclusion

While human speakers exhibit a wide range of human production variability in
naming tasks, mirroring pragmatic constraints and subjective preferences, it
is not clear to what extent VLLMs can mimic this peculiar trait of language
use. In our work, we investigate this issue in three tasks: naming common
objects, naming novel objects, and assigning quantifiers. Our results reveal
that best-performing models achieve a moderate correlation with human patterns
in some tasks (object names and color terms). However, all models dramatically
fail when assigning quantifiers, the only production setup that requires some
form of reasoning, i.e., the ability to reason over sets of objects and
process quantities. Based on our analyses, we hypothesize that the reason
behind this failure stems from the poor “counting” skills of the models.

## Limitations

In the following, we discuss some limitations of our study that may inspire
follow-up work in this direction. The poor performance on the quantification
tasks may stem from the higher complexity of the prompt used (multiple choice
prompting). Even though in our paper we discuss how analyzing the output
variability allows us to gain valuable insights even when the model is not
accurate, we can not rule out the possibility that a simpler prompt may lead
to more accurate results. As an initial step, we used a prompt that
corresponds to the instruction provided to the participants of the original
experiment in Pezzelle et al. (2018).

Moreover, it is worth noting that the human production variability analyzed in
our experiments is obtained by aggregating data coming from multiple speakers.
Even though we do aim at this, we acknowledge that it is unlikely that one
single model can mimic such a rich variability. Our study is more focused on
understanding to what extent current Vision & Language LLMs can mimic this
feature, showing the suitability of some tasks and datasets not explored in
previous work.

Finally, we computed the color and saliency feature for NOUN using a string-
matching approach based on a manually defined list of keywords (as described
in Appendix A.2. We acknowledge that this approach may underestimate the color
and texture saliency in the model output. Although in this case, the small
size of the dataset allowed us to verify that this is not the case, we believe
that it is important to take this point into account when running experiments
on a larger scale. Moreover, as a limitation of the NOUN dataset (and not of
our experimental setup), we do not have access to the actual color and texture
labels used by human participants during the dataset collection. For this
reason, in NOUN we do not consider the actual distribution of the attributes
used by human speakers but just their overall frequency.

## References

  * Alva-Manchego et al. (2021) Fernando Alva-Manchego, Carolina Scarton, and Lucia Specia. 2021.  The (un) suitability of automatic evaluation metrics for text simplification.  _Computational Linguistics_ , 47(4):861–889.
  * Brown (1958) Roger Brown. 1958.  How shall a thing be called?  _Psychological review_ , 65(1):14.
  * Fan et al. (2018) Angela Fan, Mike Lewis, and Yann Dauphin. 2018.  Hierarchical neural story generation.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 889–898, Melbourne, Australia. Association for Computational Linguistics.
  * Gan et al. (2022) Zhe Gan, Linjie Li, Chunyuan Li, Lijuan Wang, Zicheng Liu, Jianfeng Gao, et al. 2022.  Vision-language pre-training: Basics, recent advances, and future trends.  _Foundations and Trends® in Computer Graphics and Vision_ , 14(3–4):163–352.
  * Graf et al. (2016) Caroline Graf, Judith Degen, Robert XD Hawkins, and Noah D Goodman. 2016.  Animal, dog, or dalmatian? level of abstraction in nominal referring expressions.  In _CogSci_.
  * Holtzman et al. (2019) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019.  The curious case of neural text degeneration.  _arXiv preprint arXiv:1904.09751_.
  * Horst and Hout (2016) Jessica S Horst and Michael C Hout. 2016.  The novel object and unusual name (noun) database: A collection of novel images for use in experimental research.  _Behavior research methods_ , 48:1393–1409.
  * Huang and Chang (2023) Jie Huang and Kevin Chen-Chuan Chang. 2023.  Towards reasoning in large language models: A survey.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 1049–1065, Toronto, Canada. Association for Computational Linguistics.
  * Kazemzadeh et al. (2014) Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. 2014.  ReferItGame: Referring to objects in photographs of natural scenes.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 787–798, Doha, Qatar. Association for Computational Linguistics.
  * Koh et al. (2023) Jing Yu Koh, Ruslan Salakhutdinov, and Daniel Fried. 2023.  Grounding language models to images for multimodal inputs and outputs.  In _International Conference on Machine Learning_ , pages 17283–17300. PMLR.
  * Krishna et al. (2017) Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. 2017.  Visual genome: Connecting language and vision using crowdsourced dense image annotations.  _International journal of computer vision_ , 123:32–73.
  * Levelt (1993) Willem JM Levelt. 1993.  _Speaking: From intention to articulation_.  MIT press.
  * Li et al. (2023a) Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, and Jianfeng Gao. 2023a.  Multimodal foundation models: From specialists to general-purpose assistants.  _arXiv preprint arXiv:2309.10020_ , 1(2):2.
  * Li et al. (2023b) Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023b.  Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.  _arXiv preprint arXiv:2301.12597_.
  * Lin (1991) Jianhua Lin. 1991.  Divergence measures based on the shannon entropy.  _IEEE Transactions on Information theory_ , 37(1):145–151.
  * Liu et al. (2023a) Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023a.  Improved baselines with visual instruction tuning.  _arXiv preprint arXiv:2310.03744_.
  * Liu et al. (2023b) Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023b.  Visual instruction tuning.  _arXiv preprint arXiv:2304.08485_.
  * Olson (1970) David R Olson. 1970.  Language and thought: aspects of a cognitive theory of semantics.  _Psychological review_ , 77(4):257.
  * Pezzelle et al. (2018) Sandro Pezzelle, Raffaella Bernardi, and Manuela Piazza. 2018.  Probing the mental representation of quantifiers.  _Cognition_ , 181:117–126.
  * Rohde et al. (2012) Hannah Rohde, Scott Seyfarth, Brady Clark, Gerhard Jäger, and Stefan Kaufmann. 2012.  Communicating with cost-based implicature: A game-theoretic approach to ambiguity.  In _The 16th workshop on the semantics and pragmatics of dialogue, paris, september_.
  * Silberer et al. (2020a) Carina Silberer, Sina Zarrieß, and Gemma Boleda. 2020a.  Object naming in language and vision: A survey and a new dataset.  In _Proceedings of the Twelfth Language Resources and Evaluation Conference_ , pages 5792–5801, Marseille, France. European Language Resources Association.
  * Silberer et al. (2020b) Carina Silberer, Sina Zarrieß, Matthijs Westera, and Gemma Boleda. 2020b.  Humans meet models on object naming: A new dataset and analysis.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 1893–1905, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  * Takmaz et al. (2024) Ece Takmaz, Sandro Pezzelle, and Raquel Fernández. 2024.  Describing images fast and slow: Quantifying and predicting the variation in human signals during visuo-linguistic processes.  _arXiv preprint arXiv:2402.01352_.
  * Testoni et al. (2019) Alberto Testoni, Sandro Pezzelle, and Raffaella Bernardi. 2019.  Quantifiers in a multimodal world: Hallucinating vision with language and sound.  In _Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics_ , pages 105–116, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Yu et al. (2016) Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. 2016.  Modeling context in referring expressions.  In _Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14_ , pages 69–85. Springer.
  * Zhao et al. (2023) Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023.  A survey of large language models.  _arXiv preprint arXiv:2303.18223_.

##  Appendix A Appendix

###  A.1 ManyNames Appendix

Figure 8 shows the results on the ManyNames dataset broken down by the image
domain. LLaVA outperforms other models in most of the domains, but for
clothing and people it is comparable to BLIP-2. Note that all model reach have
the poorest performance on these two domains. As highlighted by Silberer et
al. (2020a, b) and confirmed by our manual inspection, models confuse people
and clothing objects much more frequently than humans do. ManyNames is
licensed under Creative Commons Attribution 4.0 International.

Figure 4: Inverse Jensen–Shannon divergence broken down by the image domain in
ManyNames.

###  A.2 NOUN Appendix

We define the following list of color and texture attributes to analyze the
samples generated by the model with a string-matching approach.

Colors = [“Red”, “Orange”, “Yellow”, “Green”, “Blue”, “Purple”, “Pink”,
“Brown”, “Gray”, “Black”, “White”, “Beige”, “Turquoise”, “Teal”, “Magenta”,
“Lavender”, “Indigo”, “Maroon”, “Gold”, “Silver”, “Bronze”, “Copper”, “Olive”,
“Navy”, “Sky blue”, “Cream”, “Peach”, “Rose”, “Fuchsia”, “Coral”, “Mint”,
“Chartreuse”, “Salmon”, “Sienna”, “Slate”, “Tan”, “Crimson”, “Ivory”, “Khaki”,
“Lilac”, “Mauve”, “Mustard”, “Rust”, “Scarlet”, “Tangerine”, “Vermilion”,
“Violet”, “Wheat”, “Brick red”, “Caramel”]

Textures = [“Smooth”, “Rough”, “Fuzzy”, “Soft”, “Hard”, “Bumpy”, “Slick”,
“Sticky”, “Grainy”, “Sandy”, “Slippery”, “Jagged”, “Sharp”, “Coarse”, “Silky”,
“Velvety”, “Wet”, “Dry”, “Glossy”, “Matte”, “Sparkly”, “Metallic”, “Wooden”,
“Leathery”, “Plastic”, “Rubber”, “Furry”, “Woolly”, “Feathery”, “Smooth”,
“Satin”, “Lace”, “Crochet”, “Knitted”, “Embroidered”, “Linen”, “Silk”,
“Velvet”, “Suede”, “Corduroy”, “Denim”, “Felt”, “Tweed”, “Mesh”, “Hairy”,
“Crisp”, “Crumbly”, “Flaky”, “Puffy”, “Spongy”, “Crunchy”, “Chewy”, “Gummy”,
“Slimy”, “Starchy”, “Syrupy”, “Icy”, “Rocky”, “Stony”, “Sandy”, “Peppery”,
“Salty”, “Sour”, “Sweet”, “Tangy”, “Tart”, “Spicy”, “Herbaceous”, “Earthy”,
“Mossy”, “Woody”, “Smoky”, “Smokey”, “Rusty”, “Corroded”, “Weathered”,
“Rugged”, “Smooth”, “Polished”, “Shiny”, “Gleaming”, “Dull”, “Muddy”,
“Cloudy”, “Milky”, “Transparent”, “Translucent”, “Opaque”]

Figure 6 shows an example of the output of different models, together with
their color and texture saliency as well as human saliency values. NOUN is
released without a specific license.

Figure 5: Toy example to show how the effect of over-generating a noun that
was not often assigned by humans (model 1) and generating a noun that was not
selected by humans (model 2) on the inverse Jensen–Shannon divergence metric.
Figure 6: Example of the model output in the NOUN dataset.

###  A.3 QUANT Appendix

Figures 7 and 8 show additional analyses on the QUANT dataset. They are
discussed in Section 4. Figure 9 shows some qualitative examples of the
models’ output when asked to answer the question ‘How many animals are there
in the image?’. Note that the images are randomly selected from QUANT. QUANT
is released without a specific license.

Figure 7: Density plot reporting the frequency distribution of responses for
the 9 quantifiers (y-axis) against the proportion of targets in the scene
(x-axis). Figure 8: How often each quantifier (x-axis) is selected by the
model (y-axis, expressed as %), regardless of the proportion of targets (i.e.,
animals) in the image. Figure 9: How many animals are there in the image? All
models fail to successfully count the number of animals in the image. Note
that models generally output a number but sometimes FROMAGe outputs the
quantifier ‘many’ which, interestingly, is the quantifier the model is
strongly biased towards as illustrated in Figure 7 and discussed in Section 4.

###  A.4 Models Appendix

While FROMAGe is trained with a contrastive learning objective for image
captioning and it is shown to perform particularly well with longer textual
contexts, BLIP-2 jointly optimizes three pre-training objectives that share
the same input format and model parameters: image-text contrastive learning,
image-grounded text generation, and image-text matching. The main innovation
of LLaVA is the use GPT-4 generated visual instruction tuning data. Moreover,
LLaVA has a simpler scheme to connect image and language representations
compared to BLIP-2 and FROMAGe. We used blip2-opt-2.7b and llava-v1.5-7b,
while for FROMAGe we used the model made available by Koh et al. (2023).
FROMAGe and LLaVA are released with Apache-2.0 license. BLIP-2 is distributed
with BSD 3-Clause License. We run our experiments in accordance with the model
license.

###  A.5 Additional Details

The data used in our work do not contain any information that names or
uniquely identifies individual people or offensive content. FROMAGe has 5M
trainable parameters and a total number of around 7.2B parameters. BLIP-2 has
188M trainable parameters and 2.7B total parameters. LLaVA has 7B parameters.
All the models are evaluated on a single GPU (NVIDIA RTX A5000). We
experimented with a few configurations of hyperparameters for nucleus sampling
generation ( described in Section 3). We did not find significant differences
across different hyperparameters. We used the SciPy library
(https://scipy.org/) to compute the correlation/divergence results. We used
the NLTK library (https://www.nltk.org/) to extract nouns from the model
output for ManyNames.