# Ultralow-threshold thin-film lithium niobate optical parametric oscillator

Juanjuan Lu  Ayed Al Sayem  Zheng Gong  Joshua B. Surya  Department of
Electrical Engineering, Yale University, New Haven, Connecticut 06511, USA
Chang-Ling Zou  Department of Optics, University of Science and Technology of
China, Hefei 230026, P. R. China.  Hong X. Tang<EMAIL_ADDRESS>Department
of Electrical Engineering, Yale University, New Haven, Connecticut 06511, USA

###### Abstract

Materials with strong $\chi^{(2)}$ optical nonlinearity, especially lithium
niobate, play a critical role in building optical parametric oscillators
(OPOs). However, chip-scale integration of low-loss $\chi^{(2)}$ materials
remains challenging and limits the threshold power of on-chip $\chi^{(2)}$
OPO. Here we report the first on-chip lithium niobate optical parametric
oscillator at the telecom wavelengths using a quasi-phase matched, high-
quality microring resonator, whose threshold power ($\sim$30 $\mu$W) is 400
times lower than that in previous $\chi^{(2)}$ integrated photonics platforms.
An on-chip power conversion efficiency of 11% is obtained at a pump power of
93 $\mu$W. The OPO wavelength tuning is achieved by varying the pump frequency
and chip temperature. With the lowest power threshold among all on-chip OPOs
demonstrated so far, as well as advantages including high conversion
efficiency, flexibility in quasi-phase matching and device scalability, the
thin-film lithium niobate OPO opens new opportunities for chip-based tunable
classical and quantum light sources and provides an potential platform for
realizing photonic neural networks.

††preprint: APS/123-QED

Optical parametric oscillators (OPOs) are critical sources of coherent
radiation over a wide spectral range and have been workhorses for spectroscopy
in the near- and mid-infrared regime Dunn and Ebrahimzadeh (1999); Arslanov
_et al._ (2013); Maidment, Schunemann, and Reid (2016). They are also widely
used as non-classic light sources for applications in quantum information
processing, including quantum random number generation, quantum key
distribution, and recently demonstrated Ising machines Andersen  _et al._
(2016); Madsen  _et al._ (2012); Inagaki  _et al._ (2016); Yamamoto  _et al._
(2017). Microcavity-based OPO has been particularly attractive due to its
miniaturization and power-efficiency. With ultra-high optical quality (Q)
factor of $10^{7}$–$10^{8}$ afforded by the whispering gallery resonator (WGR)
system, low-threshold OPOs have been achieved in various bulk WGRs Kippenberg,
Spillane, and Vahala (2004); Savchenkov  _et al._ (2004); Fürst  _et al._
(2010); Beckmann  _et al._ (2011); Herr  _et al._ (2018); Sayson  _et al._
(2019); Meisenheimer  _et al._ (2017), especially the highly polished lithium
niobate (LN) WGRs with $\chi^{(2)}$ nonlinearity.

Over the past decade, the demand for scalability in photonic components has
stimulated efforts to produce on-chip OPOs via either $\chi^{(2)}$ or
$\chi^{(3)}$ nonlinearity Lu  _et al._ (2019a, 2020a); Ji  _et al._ (2017);
Guidry  _et al._ (2020); Chang  _et al._ (2020); Bruch  _et al._ (2019). Among
these, $\chi^{(2)}$ OPO has only been demonstrated with an aluminum nitride
(AlN) microring resonator while its threshold power is tens of milliwatts
Bruch  _et al._ (2019), which is orders of magnitude larger than state-of-the-
art value of lithium niobate WGR. Hence, low threshold and scalable
$\chi^{(2)}$ OPO remains challenging. Recently, thin film LN has emerged as a
compelling integrated nonlinear and quantum photonics platform, whose
favorable performances including but not limited to electro-optic modulation
Wang  _et al._ (2018a); Zhang  _et al._ (2019); Li  _et al._ (2020), frequency
comb generation Wang  _et al._ (2019); Okawachi  _et al._ (2020); Gong  _et
al._ (2020), frequency conversion Wang  _et al._ (2018b); Chen  _et al._
(2019); Lu  _et al._ (2019b, 2020b), and entangled photon pair generation Zhao
_et al._ (2020a); Ma  _et al._ (2020), have by now been well demonstrated.

In this Letter, we present an ultralow-threshold, chip-integrated OPO using a
periodically poled lithium niobate microring resonator (PPLNMR). With high
optical confinement and strong spatial mode overlap via the quasi-phase
matching, a threshold power as low as 30 $\mu$W is demonstrated for the
parametric oscillation at the infrared band, which is a state-of-the-art value
reported among all current integrated photonics platforms. Meanwhile, an
absolute power conversion efficiency of 11% is obtained at a pump power of 93
$\mu$W. A tuning bandwidth of 180 nm (23 THz) could be addressed through a
small change of 5 nm (2.5 THz) in pump wavelength (frequency). Alternatively,
the temperature tuning is investigated to fine control the signal and idler
wavelengths and a degenerate OPO is achieved at an optimal temperature.
Overall, low-threshold, wide-tunability, as well as flexibility in phase-
matching by domain poling make thin-film LN OPO the ideal candidate for
compact, versatile light sources for various on-chip optical applications,
such as sensing, frequency comb spectroscopy, and quantum squeezers.

Figure 1: Design and fabrication of a quasi-phase-matched optical parametric
oscillator. (a) Illustration of the parametric oscillation using a
$\chi^{(2)}$ PPLNMR. Signal and idler lights are generated by a near-visible
pump. $\omega_{p}$, $\omega_{s}$ and $\omega_{i}$ represent frequencies for
pump, signal and idler light, respectively. (b) Cross-sectional view of the
microring indicating the key geometric parameters. (c) Optical image of the
fabricated PPLNMR devices. (d) Schematics of the mode frequencies and their
respective transmission spectrum with the extracted Q values shown in the
inset. The subscripts $0,1$ represent the intrinsic and coupling Q factors,
respectively.

Figure 1(a) illustrates the principle of our lithium niobate microring OPO
device, in which $\chi^{(2)}$-based parametric down-conversion process allows
the generation of the infrared signal and idler lights with a near-visible
pump. $\omega_{p}$/$\omega_{s}$/$\omega_{i}$ represent the pump/signal/idler
mode frequencies with their energy conservation
($\omega_{p}=\omega_{s}+\omega_{i}$) shown in the inset. When the phase-
matching and triply-resonant conditions are satisfied for these three optical
modes, the parametric process would be enhanced by the high-Q resonator. The
theoretical pump power threshold $P_{th}$ for parametric oscillation could be
obtained when the optical loss and parametric gain are balanced Bruch  _et
al._ (2019):

$P_{th}=\frac{\hbar\omega_{p}}{8g^{2}}\frac{\kappa_{s}\kappa_{i}\kappa_{p}^{2}}{\kappa_{p,1}}=\frac{\hbar\omega_{s}\omega_{i}\omega_{p}^{2}}{64g^{2}}\frac{Q_{p,1}}{Q_{s}Q_{i}Q_{p}^{2}},$
(1)

where $g$ is the vacuum coupling strength between the three interacting modes,
$Q(\kappa)_{s/i/p}$ is their respective loaded Q factor (total dissipation
rate), and $Q(\kappa)_{p,1}$ is the coupling Q factor (external dissipation
rate) of the pump mode. Therefore, a large $g$ and high Q factors are always
demanding to reduce the OPO threshold power.

In the microring resonator, the coupling rate $g$ is dependent on the material
$\chi^{(2)}$ coefficient and modal overlap factor $\gamma$ through the
relation: $g\propto\chi^{(2)}\gamma$ Lu  _et al._ (2019b). Here, to optimize
$g$, we utilize the largest $\chi^{(2)}$ tensor element $d_{33}$ of z-cut LN
thin film, and also improve the mode overlap factor $\gamma$ close to unity by
employing fundamental transverse-magnetic modes. In contrast to previous study
of on-chip OPO in AlN platform Bruch  _et al._ (2019), where high-order mode
is leveraged to compensate the dispersion at the visible wavelength, we
realize the quasi-phase matching (momentum conservation) between fundamental
modes via the radial periodic-poling of the LN microring. Accordingly, for a
microring whose key geometric parameters are indicated in Fig. 1(b), an
azimuthal grating number $M=146$ (poling period $\Lambda$=2$\pi R/M$=3 $\mu$m)
is designed to compensate for the momentum mismatch $\Delta
m=|m_{s}+m_{i}-m_{p}|$ for the parametric oscillation at 1550 nm with a pump
at 775 nm. The phase-matching of the device was verified through the reversal
process of degenerate OPO, i.e. second-harmonic generation (SHG), as
elaborated in our previous work Lu  _et al._ (2020b). Figure 1(c) is an
optical image of the fabricated PPLNMR device (See Methods for fabrication
details). The cavity transmission spectra of the signal, idler, and pump modes
for the subsequent OPO are plotted in Fig. 1(d), where Lorentzian fittings are
applied to extract loaded Q factors of $2.9\times 10^{5}$, $2.2\times 10^{5}$,
and $3.7\times 10^{5}$, respectively. Their intrinsic Q factors are calculated
to be $3.6\times 10^{5}$, $3.9\times 10^{5}$, and $5.9\times 10^{5}$ at under-
coupled conditions, which are confirmed by measuring the dependence of loaded
quality factor on the coupling gap in an array of fabricated devices.

When pumping the device by a near-visible tunable laser (New Focus TLB-6712,
766–781 nm), the parametric oscillation at the infrared wavelengths could be
observed if the pump power exceeds the threshold. In practice, due to the
resonance frequency mismatch ($\delta=\omega_{p}-\omega_{s}-\omega_{i}$)
induced during the fabrication process, the OPO threshold rises to
$P_{th}\cdot(1+\delta^{2}/\kappa_{i}\kappa_{s})$. By controlling the chip
temperature to adjust $\delta$ and varying the pump power, the parametric
oscillation with the lowest threshold is observed when the device is pumped at
the resonance near 774.1 nm at an optimal temperature of 125∘C, as indicated
in Fig. 2(a). The measured on-chip OPO power (blue circles) is plotted against
the on-chip pump power, where a threshold power $P_{th}$ of $\sim$30 $\mu$W is
identified and corresponds to an intra-cavity pump photon number of
$\sim$5.4$\times 10^{4}$. The parametric oscillation spectrum at a pump power
of 120 $\mu$W is captured using an optical spectrum analyzer (Yokogawa AQ6374,
350–1750 nm), where distinct signal and idler waves are resolved at 1529.7 and
1568.4 nm respectively, as shown in the inset.

Figure 2:  Power dependence of the PPLNMR-based OPO. (a) The experimental
(blue circles) and theoretical (solid line) on-chip infrared power versus on-
chip pump power for the parametric oscillation process. An example snapshot of
off-chip pump (orange), and oscillation (blue) spectra after the device are
shown in the inset. (b) On-chip conversion efficiency of pump to both signal
and idler fields versus pump power, where a theoretical fit (solid line) is
applied. Inset: scanning electron micrograph of a PPLNMR mock-up etched in
hydrofluoric acid.

Substituting the measured threshold, and Q factors [Fig. 1(d)] in Eq. (1), we
extract a $g/2\pi$ of 1.2 MHz, consistent with the value obtained via the
independent SHG measurement Lu  _et al._ (2020b). Theoretically, the power of
the signal and idler OPO emission under the triply-resonant condition could be
derived as Bruch  _et al._ (2019)

$P_{s+i}=(\frac{2Q_{s}}{Q_{s,1}}+\frac{2Q_{i}}{Q_{i,1}})\frac{Q_{p}}{Q_{p,1}}P_{th}(\sqrt{P_{p}/P_{th}}-1).$
(2)

The theoretical predictions (solid black lines) based on Eq. (2) are plotted
in Figs. 2(a) and 2(b) and match well with the experimental data (blue
circles). With an elevated on-chip near-visible pump power of 93.0 $\mu$W, we
record an off-chip infrared power of 1.5 $\mu$W, corresponding to an on-chip
OPO power of 10.5 $\mu$W, which further translates to an absolute power
conversion efficiency $\eta$, defined as $\eta=P_{s+i}/P_{p}$, of 11% [Fig.
2(b)]. According to Eq. (2), the conversion efficiency reaches its maximum on
condition that $P_{p}=4P_{th}$, where
$\eta_{\mathrm{max}}=({2Q_{s}}/{Q_{s,1}}+{2Q_{i}}/{Q_{i,1}})({Q_{p}}/{Q_{p,1}})/4$.
The efficiency depletion at the high-power regime is mainly attributed to the
back-conversion of the signal and idler fields. Different coupling conditions
could be adapted by varying the gap between the bus waveguide and microring
for various applications. For example, a $\eta_{\mathrm{max}}$ of 0.25 is
derived with the assumption that all three interacting modes are critical-
coupled whilst strong over-coupling condition provides a $\eta_{\mathrm{max}}$
approaching unity. In our case, the under-coupling condition provides an
ultralow threshold of 30 $\mu$W though limits our current
$\eta_{\mathrm{max}}$ to 11%.

We further demonstrate the feasibility of our ultralow-threshold OPO in
wavelength tuning over a large spectrum range, which promises flexible
coherent optical sources for future on-chip applications. It is well-known
that, in the microring resonators, the phase-matching condition could be
satisfied simultaneously by many signal-idler mode pairs, with their
wavelength difference increasing at intervals of 2 times of free spectral
range (FSR) for a given pump mode and temperature. However, due to the
inherent dispersion, the frequency mismatch $\delta$ varies for different mode
pairs, and only one pair of modes oscillates with the lowest threshold.
Therefore, we investigate the tuning of signal and idler wavelengths by
varying the resonator temperature as well as the pump mode number.

Figure 3: OPO wavelength tuning by varying both the temperature and pump
wavelength. Simulated (solid colored lines) and measured (colored circles) OPO
wavelengths as a function of pump wavelength at various temperatures. As an
example, the right panel plots the recorded spectra with different pump
wavelengths at 125∘C. The left panel indicates the occurrence of degenerate
parametric oscillation by varying the temperature.

As indicated by the solid color lines in the middle panel of Fig. 3, the
theory predicts that the quasi-phase-matched signal and idler wavelengths
could be continuously tuned as a function of the pump wavelength and the chip
temperature (see Methods for simulation details). However, due to the
constrain of resonance condition, we can only experimentally access discrete
points on the lines, as marked by the blue dots. On one hand, for a given
temperature, the spectral separation between the signal and idler increases
with the decreasing pump wavelengths. For example, by fixing the operation
temperature at 125∘C, the OPO wavelength could be tuned from $1450$ to
$1650\,\mathrm{nm}$, with the pump wavelength change of only $5\,\mathrm{nm}$.
The detailed spectra are plotted on the right panel, whose signal and idler
wavelengths (blue circles) are mapped to the left panel and show good
agreement with the theoretical predication. On the other hand, for a given
pump mode, the temperature tuning allows for the fine-control of the OPO
wavelengths. As indicated on the left panel, for a fixed pump mode around
$\sim$770.5 nm (red shaded region), the signal-idler separation is observed to
narrow as the temperature increases and an degenerate OPO is experimentally
demonstrated at an optimal temperature of 140∘C, which match well with the
prediction. Hence, our OPO device has shown good potential to realize the
relatively precise control of oscillation wavelengths by combining pump
wavelength and temperature tunings. Noting that the demonstrated wavelength
tuning is broad but inherently discrete, continuous tuning (mode-hop free) is
always demanding and remains to be investigated via several promising
strategies: One approach is the material refractive index change via either
thermo-optic or electro-optic effect, which allows tens of GHz tuning range Hu
_et al._ (2020); Surya  _et al._ (2020). Another is cascading multiple
microrings with varying geometry parameters on each individual bus waveguide
Guo  _et al._ (2018); Surya  _et al._ (2018), so that the tuning range of
multiple resonators could possibly overlap and cover one single FSR.

Finally, we compare the performances of various integrated platforms,
including both $\chi^{(3)}$ and $\chi^{(2)}$ mechanisms, for their respective
state-of-the-art cavity-enhanced OPOs in terms of the material, refractive
index $n_{0}$, nonlinear coefficients, device structure, operating
wavelengths, Q factor, and threshold power, as presented in Tab. 1.

Table 1: Performances of various nonlinear integrated platforms for microcavity-enhanced parametric oscillation. Materials | $n_{0}$ |  $\chi^{(3)}$ ($10^{-18}\mathrm{m}^{2}/\mathrm{W}$) |  $\chi^{(2)}$(pm/V) | Structure | Process |  $\lambda_{p}-\lambda_{s}$(nm) | Q($\times 10^{6}$) |  $P_{\mathrm{th}}$(mW) | Ref.
---|---|---|---|---|---|---|---|---|---
LN | 2.2 | 0.18 | 40 | microring | $\chi^{(2)}$ | 775–1550 | 0.6 | 0.030 | this work
$\chi^{(3)}$ | 1560–1540 | 1.1 | 80 | Wang  _et al._ (2019)
AlN | 2.1 | 0.23 | 6 | microring | $\chi^{(2)}$ | 780–1560 | 1.0 | 12 | Bruch  _et al._ (2019)
$\chi^{(3)}$ | 1560–1530 | 1.6 | 25 | Liu  _et al._ (2018)
Si3N4 | 2 | 0.25 | — | microring | $\chi^{(3)}$ | 1560-1540 | 37 | 0.33 | Ji  _et al._ (2017)
900–700 | 2.5 | 0.90 | Lu  _et al._ (2019a)
AlGaAs | 3.4 | 26 | 120 | microring | $\chi^{(3)}$ | 1540–1520 | 1.5 | 0.036 | Chang  _et al._ (2020)
GaP | 3.1 | 11 | 82 | microring | $\chi^{(3)}$ | 1560–1540 | 0.2 | 3.0 | Wilson  _et al._ (2020)
SiC | 2.7 | 1 | 12 | microring | $\chi^{(3)}$ | 1560–1530 | 1.1 | 8.5 | Guidry  _et al._ (2020)
Si | 3.5 | 10 | — | microring | $\chi^{(3)}$ | 2600–2500 | 0.6 | 3.1 | Griffith  _et al._ (2015)
SiO2 | 1.4 | 0.022 | — | microtoroid | $\chi^{(3)}$ | 1550–1500 | 100 | 0.050 | Del’Haye  _et al._ (2007)
microdisk | 1550–1549 | 1100 | 0.95 | Wu  _et al._ (2020)

Theoretically, $P_{th}\propto 1/Q^{2}$ for the $\chi^{(3)}$ OPO Ji  _et al._
(2017). However, in the case of $\chi^{(2)}$ process, $P_{th}\propto 1/Q^{3}$
[Eq. (1)]. Therefore, to reach the same threshold, $\chi^{(2)}$ OPO has a less
stringent requirement on its Q factor. Despite the fact that silicon
photonics, including Si, Si3N4 and SiO2, have much higher Q factors due to
their mature fabrication techniques, the demonstration in this work, to the
best of our knowledge, delivers the lowest threshold power among all on-chip
OPOs so far. Moreover, due to the three-wave mixing nature, $\chi^{(2)}$ OPO
offers broader spectral-separation between the pump and signal(idler) fields
intrinsically, which promises for sufficient on-chip filtering. We note that
piezoelectric III–V materials, such as Al(Ga)N, (Al)GaAs and GaP, feature both
significant $\chi^{(2)}$ and $\chi^{(3)}$ nonlinear coefficients, while
relying on challenging waveguide geometrical engineering to achieve modal
phase matching between fundamental and high-order modes which limits their OPO
performance as well as operation wavelengths. Overall, the periodically poled
thin-film lithium niobate microresonator prove to be an efficient OPO with the
lowest power threshold. In addition, due to its flexibility in ferroelectric
domain control for quasi-phase matching and greatly enhanced light-matter
interaction afforded by its high effective $\chi^{(2)}$ nonlinearity and Q
factor, our device on a integrated photonic chip opens new possibilities for
various applications and alternative device designs could be envisioned based
on target applications. Firstly, PPLNMR device promises chip-based tunable
classical and quantum light sources. For example, the poling grating number
$M$ could be tailored to realize parametric oscillation at the mid-infrared
wavelengths for the molecule spectroscopy Tittel, Richter, and Fried (2003).
Over-coupling the degenerate signal modes could lead to efficient on-chip
squeezers, which play a crucial role in continuous variable quantum
information processing Zhao  _et al._ (2020b). Secondly, due to the multiple
modes and nonlinear optical processes present in the same device, the OPO
dynamics could be an indicator for novel physics of light-matter interactions
in the microresonator Li  _et al._ (2018). For example, inspired by the recent
demonstration on AlN Bruch  _et al._ (2020), ultra-efficient pockels soliton
microcomb could be foreseen in the same device with the efforts of
coordinating its Kerr, quadratic, and photorefractive effects He  _et al._
(2019). Lastly, by coupling several PPLNMRs, significant nonlinear interplay
between the OPOs offers an platform for realizing the photonic neural networks
and Ising models, which find applications in artificial machine learning
Feldmann  _et al._ (2019); Yamamoto  _et al._ (2020).

In conclusion, we demonstrate $\chi^{(2)}$ optical parametric oscillation in
lithium niobate nanophotonics. Leveraging the high optical confinement, as
well as optimized quasi-phase matching condition, an ultra-low power threshold
(30 $\mu$W) of optical parametric oscillation is obtained and an on-chip
conversion efficiency of 11 % is measured with an on-chip pump power of 93
$\mu$W. The tunability of the OPO wavelengths is also investigated via the
pump-wavelength and temperature tuning, and the experimental results indicate
a wide tuning range. By further pushing the Q factor to that of bulk LN WGRs,
approaching $10^{8}$, a pump threshold down to several picowatts could be
attainable in PPLNMR devices, allowing the study of OPO at a few-photon level
and is critical for the scalability of photonic quantum systems. In light of
the most recent works on near-octave comb generation, efficient SHG, and
bright photon pair generation, PPLNMR devices hold promise for the chip-based
tunable coherent light sources as well as the monolithic realization of
nonlinear and quantum photonics.

## Methods

Nanofabrication. The devices are fabricated on a commercial LNOI wafer
(supplied by NANOLN) with 610 nm thick Z-cut LN thin film on 1.8 $\mu$m
silicon dioxide (SiO2) on a silicon substrate. The bus waveguide is ultimately
tapered to a width of 4 $\mu$m at both facets to improve the fiber-to-chip
coupling efficiency. The pattern is defined by a 100 kV electron beam
lithography system (Raith EBPG 5000+) with a negative FOx-16 resist, which is
then developed in 25$\%$ TMAH solution to ensure a high contrast. The exposed
pattern is transferred onto the LN thin film using an optimized inductively
couple plasma (ICP) reactive ion etching (RIE) process with Ar+ plasma. For
the subsequent poling process, the radial nickel electrodes are initially
patterned on top of the LN microring via the lift-off process. The periodic
domain inversion is then enabled by keeping the silicon substrate as the
electrical ground while applying several 600 V, 250 ms pulses on the
electrodes at an elevated temperature of 250 ∘C. After removing the nickel
electrodes, the chip is cleaved to expose the waveguide facets for fiber-to-
chip coupling. The insertion losses are calibrated to be 8.4 and 11.1 dB/facet
for the infrared and near-visible lights, respectively.

Numerical simulation. The parametric signal and idler wavelengths as a
function of the pump wavelength and temperature are numerically investigated
using a commercial finite-difference-eigenmode solver (Lumerical MODE) based
on the equations:

$\displaystyle\frac{1}{\lambda_{p}}=\frac{1}{\lambda_{s}}+\frac{1}{\lambda_{i}},$
(3)
$\displaystyle\frac{n_{p}(T)}{\lambda_{p}}=\frac{n_{s}(T)}{\lambda_{s}}+\frac{n_{i}(T)}{\lambda_{i}}+\frac{M}{2\pi
R},$ (4)

where $\lambda_{p/s/i}$ and $n_{p/s/i}$ denote the wavelengths and refractive
indices for the pump, signal, and idler lights, respectively. In the
simulation, we used the Sellmeier equation for congruent LN in Ref. Zelmon,
Small, and Jundt (1997) and its wavelength-dependent thermo-optic coefficient
is also considered Moretti  _et al._ (2005).

## references

  * Dunn and Ebrahimzadeh (1999) M. H. Dunn and M. Ebrahimzadeh, “Parametric generation of tunable light from continuous-wave to femtosecond pulses,” Science 286, 1513 (1999).
  * Arslanov  _et al._ (2013) D. D. Arslanov, M. Spunei, J. Mandon, S. M. Cristescu, S. T. Persijn, and F. J. M. Harren, “Continuous-wave optical parametric oscillator based infrared spectroscopy for sensitive molecular gas sensing,” Laser & Photonics Reviews 7, 188 (2013).
  * Maidment, Schunemann, and Reid (2016) L. Maidment, P. G. Schunemann, and D. T. Reid, “Molecular fingerprint-region spectroscopy from 5 to 12 $\mu$m using an orientation-patterned gallium phosphide optical parametric oscillator,” Opt. Lett. 41, 4261 (2016).
  * Andersen  _et al._ (2016) U. L. Andersen, T. Gehring, C. Marquardt, and G. Leuchs, “30 years of squeezed light generation,” Physica Scripta 91, 053001 (2016).
  * Madsen  _et al._ (2012) L. S. Madsen, V. C. Usenko, M. Lassen, R. Filip, and U. L. Andersen, “Continuous variable quantum key distribution with modulated entangled states,” Nat. Commun. 3, 1 (2012).
  * Inagaki  _et al._ (2016) T. Inagaki, K. Inaba, R. Hamerly, K. Inoue, Y. Yamamoto, and H. Takesue, “Large-scale ising spin network based on degenerate optical parametric oscillators,” Nat. Photonics 10, 415 (2016).
  * Yamamoto  _et al._ (2017) Y. Yamamoto, K. Aihara, T. Leleu, K.-i. Kawarabayashi, S. Kako, M. Fejer, K. Inoue, and H. Takesue, “Coherent Ising machines—optical neural networks operating at the quantum limit,” npj Quantum Information 3, 49 (2017).
  * Kippenberg, Spillane, and Vahala (2004) T. J. Kippenberg, S. M. Spillane, and K. J. Vahala, “Kerr-nonlinearity optical parametric oscillation in an ultrahigh-Q toroid microcavity,” Phys. Rev. Lett. 93, 083904 (2004).
  * Savchenkov  _et al._ (2004) A. A. Savchenkov, A. B. Matsko, D. Strekalov, M. Mohageg, V. S. Ilchenko, and L. Maleki, “Low threshold optical oscillations in a whispering gallery mode ${\mathrm{C}\mathrm{a}\mathrm{F}}_{2}$ resonator,” Phys. Rev. Lett. 93, 243905 (2004).
  * Fürst  _et al._ (2010) J. U. Fürst, D. V. Strekalov, D. Elser, A. Aiello, U. L. Andersen, C. Marquardt, and G. Leuchs, “Low-threshold optical parametric oscillations in a whispering gallery mode resonator,” Phys. Rev. Lett. 105, 263904 (2010).
  * Beckmann  _et al._ (2011) T. Beckmann, H. Linnenbank, H. Steigerwald, B. Sturman, D. Haertle, K. Buse, and I. Breunig, “Highly tunable low-threshold optical parametric oscillation in radially poled whispering gallery resonators,” Phys. Rev. Lett. 106, 143903 (2011).
  * Herr  _et al._ (2018) S. J. Herr, C. S. Werner, K. Buse, and I. Breunig, “Quasi-phase-matched self-pumped optical parametric oscillation in a micro-resonator,” Opt. Express 26, 10813 (2018).
  * Sayson  _et al._ (2019) N. L. B. Sayson, T. Bi, V. Ng, H. Pham, L. S. Trainor, H. G. Schwefel, S. Coen, M. Erkintalo, and S. G. Murdoch, “Octave-spanning tunable parametric oscillation in crystalline kerr microresonators,” Nat. Photonics 13, 701 (2019).
  * Meisenheimer  _et al._ (2017) S.-K. Meisenheimer, J. U. Fürst, K. Buse, and I. Breunig, “Continuous-wave optical parametric oscillation tunable up to an 8 $\mu$m wavelength,” Optica 4, 189 (2017).
  * Lu  _et al._ (2019a) X. Lu, G. Moille, A. Singh, Q. Li, D. A. Westly, A. Rao, S.-P. Yu, T. C. Briles, S. B. Papp, and K. Srinivasan, “Milliwatt-threshold visible–telecom optical parametric oscillation using silicon nanophotonics,” Optica 6, 1535 (2019a).
  * Lu  _et al._ (2020a) X. Lu, G. Moille, A. Rao, D. A. Westly, and K. Srinivasan, “On-chip optical parametric oscillation into the visible: generating red, orange, yellow, and green from a near-infrared pump,” Optica 7, 1417 (2020a).
  * Ji  _et al._ (2017) X. Ji, F. A. Barbosa, S. P. Roberts, A. Dutt, J. Cardenas, Y. Okawachi, A. Bryant, A. L. Gaeta, and M. Lipson, “Ultra-low-loss on-chip resonators with sub-milliwatt parametric oscillation threshold,” Optica 4, 619 (2017).
  * Guidry  _et al._ (2020) M. A. Guidry, K. Y. Yang, D. M. Lukin, A. Markosyan, J. Yang, M. M. Fejer, and J. Vučković, “Optical parametric oscillation in silicon carbide nanophotonics,” Optica 7, 1139 (2020).
  * Chang  _et al._ (2020) L. Chang, W. Xie, H. Shu, Q.-F. Yang, B. Shen, A. Boes, J. D. Peters, W. Jin, C. Xiang, S. Liu, G. Moille, S.-P. Yu, X. Wang, K. Srinivasan, S. B. Papp, K. Vahala, and J. E. Bowers, “Ultra-efficient frequency comb generation in AlGaAs-on-insulator microresonators,” Nat. Commun. 11, 1 (2020).
  * Bruch  _et al._ (2019) A. W. Bruch, X. Liu, J. B. Surya, C.-L. Zou, and H. X. Tang, “On-chip $\chi^{(2)}$ microring optical parametric oscillator,” Optica 6, 1361 (2019).
  * Wang  _et al._ (2018a) C. Wang, M. Zhang, X. Chen, M. Bertrand, A. Shams-Ansari, S. Chandrasekhar, P. Winzer, and M. Lončar, “Integrated lithium niobate electro-optic modulators operating at cmos-compatible voltages,” Nature 562, 101 (2018a).
  * Zhang  _et al._ (2019) M. Zhang, B. Buscaino, C. Wang, A. Shams-Ansari, C. Reimer, R. Zhu, J. M. Kahn, and M. Lončar, “Broadband electro-optic frequency comb generation in a lithium niobate microring resonator,” Nature 568, 373 (2019).
  * Li  _et al._ (2020) M. Li, J. Ling, Y. He, U. A. Javid, S. Xue, and Q. Lin, “Lithium niobate photonic-crystal electro-optic modulator,” Nature Communications 11, 4123 (2020).
  * Wang  _et al._ (2019) C. Wang, M. Zhang, M. Yu, R. Zhu, H. Hu, and M. Loncar, “Monolithic lithium niobate photonic circuits for kerr frequency comb generation and modulation,” Nat. Commun. 10, 1 (2019).
  * Okawachi  _et al._ (2020) Y. Okawachi, M. Yu, B. Desiatov, B. Y. Kim, T. Hansson, M. Lončar, and A. L. Gaeta, “Chip-based self-referencing using integrated lithium niobate waveguides,” Optica 7, 702 (2020).
  * Gong  _et al._ (2020) Z. Gong, X. Liu, Y. Xu, and H. X. Tang, “Near-octave lithium niobate soliton microcomb,” Optica 7, 1275 (2020).
  * Wang  _et al._ (2018b) C. Wang, C. Langrock, A. Marandi, M. Jankowski, M. Zhang, B. Desiatov, M. M. Fejer, and M. Lončar, “Ultrahigh-efficiency wavelength conversion in nanophotonic periodically poled lithium niobate waveguides,” Optica 5, 1438 (2018b).
  * Chen  _et al._ (2019) J.-Y. Chen, Z.-H. Ma, Y. M. Sua, Z. Li, C. Tang, and Y.-P. Huang, “Ultra-efficient frequency conversion in quasi-phase-matched lithium niobate microrings,” Optica 6, 1244 (2019).
  * Lu  _et al._ (2019b) J. Lu, J. B. Surya, X. Liu, A. W. Bruch, Z. Gong, Y. Xu, and H. X. Tang, “Periodically poled thin film lithium niobate microring resonators with a second-harmonic generation efficiency of 250,000%/W,” Optica 6, 1455 (2019b).
  * Lu  _et al._ (2020b) J. Lu, M. Li, C.-L. Zou, A. A. Sayem, and H. X. Tang, “Toward 1% single-photon anharmonicity with periodically poled lithium niobate microring resonators,” Optica 7, 1654 (2020b).
  * Zhao  _et al._ (2020a) J. Zhao, C. Ma, M. Rüsing, and S. Mookherjea, “High quality entangled photon pair generation in periodically poled thin-film lithium niobate waveguides,” Phys. Rev. Lett. 124, 163603 (2020a).
  * Ma  _et al._ (2020) Z. Ma, J.-Y. Chen, Z. Li, C. Tang, Y. M. Sua, H. Fan, and Y.-P. Huang, “Ultra-bright quantum photon sources on chip,” (2020), arXiv:2010.04242 [physics.optics] .
  * Hu  _et al._ (2020) X.-X. Hu, J.-Q. Wang, Y.-H. Yang, J. B. Surya, Y.-L. Zhang, X.-B. Xu, M. Li, C.-H. Dong, G.-C. Guo, H. X. Tang, and C.-L. Zou, “All-optical thermal control for second-harmonic generation in an integrated microcavity,” Opt. Express 28, 11144 (2020).
  * Surya  _et al._ (2020) J. B. Surya, J. Lu, Y. Xu, and H. X. Tang, “Stable tuning of photorefractive micro-cavities using an auxiliary laser,” (2020), arXiv:2012.05293 [physics.optics] .
  * Guo  _et al._ (2018) X. Guo, C.-L. Zou, H. Jung, Z. Gong, A. Bruch, L. Jiang, and H. X. Tang, “Efficient generation of a near-visible frequency comb via cherenkov-like radiation from a kerr microcomb,” Phys. Rev. Applied 10, 014012 (2018).
  * Surya  _et al._ (2018) J. B. Surya, X. Guo, C.-L. Zou, and H. X. Tang, “Control of second-harmonic generation in doubly resonant aluminum nitride microrings to address a rubidium two-photon clock transition,” Opt. Lett. 43, 2696 (2018).
  * Liu  _et al._ (2018) X. Liu, C. Sun, B. Xiong, L. Wang, J. Wang, Y. Han, Z. Hao, H. Li, Y. Luo, J. Yan, T. Wei, Y. Zhang, and J. Wang, “Integrated high-Q crystalline AlN microresonators for broadband kerr and raman frequency combs,” ACS Photonics 5, 1943 (2018).
  * Wilson  _et al._ (2020) D. J. Wilson, K. Schneider, S. Hönl, M. Anderson, Y. Baumgartner, L. Czornomaz, T. J. Kippenberg, and P. Seidler, “Integrated gallium phosphide nonlinear photonics,” Nat. Photonics 14, 57 (2020).
  * Griffith  _et al._ (2015) A. G. Griffith, R. K. Lau, J. Cardenas, Y. Okawachi, A. Mohanty, R. Fain, Y. H. D. Lee, M. Yu, C. T. Phare, C. B. Poitras, A. L. Gaeta, and M. Lipson, “Silicon-chip mid-infrared frequency comb generation,” Nat. Commun. 6, 1 (2015).
  * Del’Haye  _et al._ (2007) P. Del’Haye, A. Schliesser, O. Arcizet, T. Wilken, R. Holzwarth, and T. J. Kippenberg, “Optical frequency comb generation from a monolithic microresonator,” Nature 450, 1214 (2007).
  * Wu  _et al._ (2020) L. Wu, H. Wang, Q. Yang, Q.-X. Ji, B. Shen, C. Bao, M. Gao, and K. Vahala, “Greater than one billion Q factor for on-chip microresonators,” Optics Lett. 45, 5129 (2020).
  * Tittel, Richter, and Fried (2003) F. K. Tittel, D. Richter, and A. Fried, “Mid-infrared laser applications in spectroscopy,” in  _Solid-state mid-infrared laser sources_ (Springer, 2003) pp. 458–529.
  * Zhao  _et al._ (2020b) Y. Zhao, Y. Okawachi, J. K. Jang, X. Ji, M. Lipson, and A. L. Gaeta, “Near-degenerate quadrature-squeezed vacuum generation on a silicon-nitride chip,” Phys. Rev. Lett. 124, 193601 (2020b).
  * Li  _et al._ (2018) M. Li, C.-L. Zou, C.-H. Dong, X.-F. Ren, and D.-X. Dai, “Enhancement of second-harmonic generation based on the cascaded second- and third-order nonlinear processes in a multimode optical microcavity,” Phys. Rev. A 98, 013854 (2018).
  * Bruch  _et al._ (2020) A. W. Bruch, X. Liu, Z. Gong, J. B. Surya, M. Li, C.-L. Zou, and H. X. Tang, “Pockels soliton microcomb,” Nat. Photonics (2020), 10.1038/s41566-020-00704-8.
  * He  _et al._ (2019) Y. He, Q.-F. Yang, J. Ling, R. Luo, H. Liang, M. Li, B. Shen, H. Wang, K. Vahala, and Q. Lin, “Self-starting bi-chromatic LiNbO3 soliton microcomb,” Optica 6, 1138 (2019).
  * Feldmann  _et al._ (2019) J. Feldmann, N. Youngblood, C. D. Wright, H. Bhaskaran, and W. H. P. Pernice, “All-optical spiking neurosynaptic networks with self-learning capabilities,” Nature 569, 208 (2019).
  * Yamamoto  _et al._ (2020) Y. Yamamoto, T. Leleu, S. Ganguli, and H. Mabuchi, “Coherent ising machines–quantum optics and neural network perspectives,” Applied Physics Letters 117, 160501 (2020), https://doi.org/10.1063/5.0016140 .
  * Zelmon, Small, and Jundt (1997) D. E. Zelmon, D. L. Small, and D. Jundt, “Infrared corrected sellmeier coefficients for congruently grown lithium niobate and 5 mol.% magnesium oxide–doped lithium niobate,” J. Opt. Soc. Am. B 14, 3319 (1997).
  * Moretti  _et al._ (2005) L. Moretti, M. Iodice, F. G. Della Corte, and I. Rendina, “Temperature dependence of the thermo-optic coefficient of lithium niobate, from 300 to 515 K in the visible and infrared regions,” J. Appl. Phys. 98, 036101 (2005).

Acknowledgements. This work is supported by Department of Energy, Office of
Basic Energy Sciences, Division of Materials Sciences and Engineering under
Grant DE-SC0019406. The authors thank Michael Rooks, Yong Sun, Sean Rinehart,
and Kelly Woods for assistance in device fabrication.

Competing interests. The authors declare no competing interests.


# Towards Adversarially Robust Vision-Language Models:
Insights from Design Choices and Prompt Formatting Techniques

Rishika Bhagwatkar  Shravan Nayak  Reza Bayat  Alexis Roger  Daniel Z Kaplan
Pouya Bashivan  Irina Rish

###### Abstract

Vision-Language Models (VLMs) have witnessed a surge in both research and
real-world applications. However, as they are becoming increasingly prevalent,
ensuring their robustness against adversarial attacks is paramount. This work
systematically investigates the impact of model design choices on the
adversarial robustness of VLMs against image-based attacks. Additionally, we
introduce novel, cost-effective approaches to enhance robustness through
prompt formatting. By rephrasing questions and suggesting potential
adversarial perturbations, we demonstrate substantial improvements in model
robustness against strong image-based attacks such as Auto-PGD. Our findings
provide important guidelines for developing more robust VLMs, particularly for
deployment in safety-critical environments.

##  1 Introduction

Figure 1:  Performance of LLaVA-7B on the COCO dataset when the adversarial
images are given along with different types of prompts (Original, AC, AP and
Random). Clean accuracy represents the model’s performance on unperturbed
images.

VLMs process images in conjunction with text prompts, enabling them to perform
a wide array of tasks, such as image captioning, visual question answering
(VQA), and cross-modal retrieval (Liu et al., 2023; Laurenccon et al., 2023;
Awadalla et al., 2023; Radford et al., 2021). While there is extensive
research on advancing architecture and scaling, recent works demonstrate that
VLMs are not immune to adversarial vulnerabilities — subtle, intentionally
crafted perturbations to input data that can lead to significant errors in the
output (Schlarmann et al., 2024). These vulnerabilities can mislead users with
harmful or toxic responses, undermining the models’ robustness and integrity.

White-box attacks are a common type of adversarial threat. These attacks
assume complete access to a model’s parameters, enabling attackers to exploit
specific vulnerabilities. Since many VLMs are open-source, attackers can
easily analyze and exploit them. Current VLMs have several design choices,
including the vision encoder, large language model (LLM), mapping network,
image resolution, and the training data (Liu et al., 2023; Laurenccon et al.,
2023; Awadalla et al., 2023). Despite the importance of these factors, their
impact on adversarial robustness is under-explored. In this study, we evaluate
how these design choices during VLM training influence their susceptibility to
white-box adversarial attacks on the input images.

In addition to design choices, the selection and quality of prompts can
significantly impact the performance and robustness of VLMs (Awal et al.,
2024). Effective prompts can enhance the models’ understanding and response to
inputs, affecting their robustness to adversarial attacks. Recent works have
focused on adversarial training to increase robustness (Schlarmann et al.,
2024; Mao et al., 2023), but it is resource-intensive and costly, often
requiring millions of samples (Wang et al., 2023; Bartoldson et al., 2024). As
a practical and cost-effective alternative, we investigate whether prompt
formatting can enhance the adversarial robustness of VLMs. This approach
explores if simple linguistic modifications can increase robustness, offering
a low-cost alternative to adversarial training.

Through evaluating both design choices and prompt formatting, we aim to
provide comprehensive insights into enhancing the adversarial robustness of
VLMs. Our main contributions are summarized as follows:

  1. 1.

We provide an in-depth analysis of how various design choices of VLMs impact
their robustness to white-box visual adversarial attacks.

  2. 2.

We investigate a novel approach to prompt formatting for enhancing the
adversarial robustness of VLMs.

  3. 3.

To the best of our knowledge, we are the first to offer actionable insights
and practical recommendations for using text prompting techniques to enhance
the robustness of VLMs in deployment.

##  2 Related Works

Vision Language Models. VLMs traditionally align visual tokens from the vision
encoder with the linguistic space of the language model using various mapping
networks, such as the Q-former in BLIP2 (Li et al., 2023) and the multilayer
perceptron in LLaVA (Liu et al., 2023). Recent studies investigate how choices
like vision encoder type, language model, resolution of images, and training
duration affect the accuracy on clean inputs (Karamcheti et al., 2024). In
contrast, our study specifically aims to explore how these choices affect
robust accuracy.

Adversarial Robustness of VLMs. Research into the adversarial robustness of
multi-modal foundation models like BLIP2 (Li et al., 2023), OpenFlamingo
(Awadalla et al., 2023), CLIP (Radford et al., 2021), and LLaVA (Liu et al.,
2023) has highlighted their susceptibility to both targeted and untargeted
visual attacks (Cui et al., 2023a; Zhao et al., 2023). Studies also explore
the potential of using pretrained VLMs to craft adversarial image and text
perturbations that can compromise black-box models fine-tuned for various
tasks (Zhao et al., 2023; Dong et al., 2023). Additionally, the
transferability of these attacks is well-studied, with techniques developed to
enhance efficacy using surrogate models (Yin et al., 2023).

Advancements in Defense Mechanisms. Many studies focusing on the adversarial
robustness of VLMs using CLIP as a vision encoder have revealed its
susceptibility to adversarial attacks (Fang et al., 2022; Tu et al., 2023;
Nguyen et al., 2022). To counter this, TeCoA (Mao et al., 2023) proposes
adversarial fine-tuning to maintain zero-shot capabilities. Further,
RobustCLIP (Schlarmann et al., 2024) proposes an unsupervised method
leveraging adversarial training on the ImageNet (Deng et al., 2009) dataset to
improve robustness across vision-language tasks. Additionally, efforts to
enhance robustness include prompt tuning, where one study suggests enriching
prompts with contextual image-derived information for improving adversarial
robustness (Cui et al., 2023b). Another approach optimizes prompts through
adversarial fine-tuning on ImageNet with specific parameters (Zhang et al.,
2023). Our research, however, focuses on analyzing the impact of prompt
formatting on model performance without additional training or image-based
information extraction.

##  3 Experiments

In this section, we outline the attack setups used during evaluations, the
tasks assessed, and the specific models examined in our model design choice
experiments.

###  3.1 Attack Setup

This work focuses on white-box gradient-based untargeted attacks on image
inputs, where it is assumed that the attacker has complete knowledge of the
model, including architecture and parameters. The objective in crafting
adversarial samples under this scenario is to subtly perturb the input so that
the model produces an incorrect output. Mathematically, it can be formulated
as $\max_{\delta}\mathcal{L}(f(x+\delta),y)$ where $f$ is the model, $x$ is
original input, $\delta$ is the adversarial perturbation learnt within the
$\|\delta\|_{\infty}\leq\epsilon$ constraint and $y$ is the original label.
Hence the goal is to find a perturbation $\delta$ that maximizes the loss
while respecting the perturbation bound.

Our evaluation encompasses three gradient-based adversarial attacks, ordered
in increasing complexity: Fast Gradient Sign Method (FGSM) (Goodfellow et al.,
2014), Projected Gradient Descent (PGD) (Madry et al., 2017), and Auto-PGD
(APGD) (Croce & Hein, 2020). We employ PGD and APGD attacks with 100
iterations while FGSM uses single iteration by design. Our evaluation focuses
on $\ell_{\infty}$ bounded perturbations, with the perturbation magnitudes
$\epsilon\in\\{4/255,8/255,16/255\\}$. This range allows us to systematically
assess the robustness of models against varying strengths of adversarial
attacks.

###  3.2 Tasks

Our evaluation covers two primary tasks: Image Captioning and VQA. For image
captioning, we use the validation splits of the COCO (Lin et al., 2014) and
Flickr30k (Plummer et al., 2015) datasets to assess caption accuracy and
relevance. In the VQA domain, we evaluate using the validation splits of VQAv2
(Antol et al., 2015), TextVQA (Singh et al., 2019), OK-VQA (Marino et al.,
2019), and VizWiz (Gurari et al., 2018) datasets. We report the robust VQA
accuracy for datasets associated with VQA tasks and robust CIDEr scores for
the captioning datasets. Higher is better for both metrics. We randomly sample
1000 examples from the validation set of each task and use this for the
adversarial evaluations of all models to ensure a fair comparison. The models
selected for evaluating the impact of design choices on adversarial robustness
are detailed in Table 1.

Table 1: Models used for evaluation of various components of VLMs. Each row
corresponds to a VLM built with the given vision encoder and LLM.

| Vision Encoder | Language Model
---|---|---
| Image
---
Representations
| CLIP ViT-L/14 @ 224px
---
SigLIP ViT-SO/14 @ 224px
DINOv2 ViT-L/14 @ 224px
ImageNet-21K+1K ViT-L/16 @ 224px
Vicuna v1.5 7B
| Image
---
Resolution
| CLIP ViT-L/14 @ 224px
---
SigLIP ViT-SO/14 @ 224px
CLIP ViT-L/14 @ 336px
SigLIP ViT-SO/14 @ 384px
Vicuna v1.5 7B
Size of LLM | CLIP ViT-L/14 @ 336px |  | Vicuna v1.5 7B
---
Vicuna v1.5 13B
| Ensemble of
---
visual encoders
| DINOv2 ViT-L/14 +
---
CLIP ViT-L/14 @ 336px
DINOv2 ViT-L/14 +
SigLIP ViT-L/14 @ 384px
Vicuna v1.5 7B

##  4 Results

Figure 2: Comparison between VLMs having different vision encoders (left),
different input resolutions (center) and different LLM size (right). The
comparison is based on the APGD accuracy averaged over all tasks as shown in
Tables 2, 3, 4 and 5.

###  4.1 Model Design Choices

In our analysis, we examine the impact of various model design choices on
adversarial robustness. Specifically, we focus on: (a) the choice of vision
encoder; (b) the input resolution used by the vision encoder; (c) the sizes of
the language models; and (d) the ensemble use of multiple vision encoders.
Each of these aspects is detailed in the sections below. We report results
using $\epsilon=8/255$. Please check Appendix A for results with other values
$\epsilon=4/255,16/255$. We highlight the best robust FGSM, PGD, APGD accuracy
across benchmarks for all attacks and models.

####  4.1.1 Impact of Vision Encoder

We systematically evaluate the effects of employing different vision encoders,
each trained under distinct conditions. We compare VLMs that use four
different image encoders: CLIP (Radford et al., 2021), SigLIP (Zhai et al.,
2023), DINOv2 (Oquab et al., 2023), and ImageNet (Dosovitskiy et al., 2020).
As shown in Table 2, SigLIP slightly outperforms CLIP, with both noticeably
surpassing DINOv2 and ImageNet VLMs on weaker attacks. However, the difference
diminishes for stronger attacks. We hypothesize that the Vision Transformer
(ViT) used in CLIP and SigLIP has been trained across a wide spectrum of
internet-collected images and hence has seen many more distributions during
training than DINOv2 and ImageNet. The results also resonate with the choice
of vision encoders in recent state-of-the-art VLMs (Liu et al., 2023;
Karamcheti et al., 2024).

Table 2: Comparison between VLMs having different image encoders but the same
language model - Vicuna v1.5 7B. Note: Higher values ($\uparrow$) indicate
better performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQav2
| None | 116.35 | 76.15 | 59.48 | 37.10 | 41.18 | 73.89
| FGSM | 94.18 | 56.83 | 47.58 | 22.40 | 33.70 | 57.85
| PGD | 13.36 | 9.11 | 13.90 | 7.42 | 8.67 | 31.65
CLIP | APGD | 6.32 | 4.41 | 10.11 | 4.80 | 8.16 | 27.56
| None | 118.69 | 78.29 | 60.92 | 38.82 | 42.43 | 74.49
| FGSM | 94.09 | 54.43 | 48.24 | 24.38 | 39.85 | 60.63
| PGD | 20.46 | 11.03 | 14.96 | 7.52 | 10.91 | 34.70
SigLIP | APGD | 7.32 | 4.87 | 10.44 | 5.30 | 9.54 | 30.38
| None | 104.84 | 54.78 | 57.00 | 10.37 | 38.07 | 64.80
| FGSM | 81.81 | 38.24 | 40.08 | 8.03 | 33.78 | 46.24
| PGD | 3.07 | 1.80 | 9.06 | 1.83 | 10.60 | 25.21
DINOv2 | APGD | 2.09 | 1.22 | 7.16 | 2.00 | 10.20 | 22.47
| None | 101.59 | 54.92 | 56.34 | 10.70 | 39.29 | 68.36
| FGSM | 69.13 | 32.42 | 38.40 | 6.60 | 32.45 | 46.13
| PGD | 5.38 | 3.43 | 9.26 | 1.79 | 10.78 | 22.73
In1k | APGD | 2.74 | 2.04 | 7.58 | 1.52 | 10.22 | 20.79

####  4.1.2 Resolution of Vision Encoder

Generally, a higher input resolution improves the quality of visual
representations, potentially boosting model performance (Karamcheti et al.,
2024). Owing to the availability of high-resolution variants, we specifically
evaluate models equipped with CLIP and SigLIP vision encoders at two distinct
resolutions to thoroughly understand these effects. Based on Table 3, while
increasing resolution enhances robustness against stronger attacks for high-
resolution CLIP models (on most tasks), the effectiveness of the increased
resolution in SigLIP models appears to be task-dependent. However, we observe
that robust accuracy significantly deteriorates under APGD attacks in all
cases except for VQAv2. For results on other $\epsilon$ values, please refer
to Appendix A.

Table 3: Comparison between VLMs having different input resolutions of CLIP
and SigLIP. All of them have the same language model: Vicuna v1.5 7B. Note:
Higher values ($\uparrow$) indicate better performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQav2
| None | 116.35 | 76.15 | 59.48 | 37.10 | 41.18 | 73.89
| FGSM | 94.18 | 56.83 | 47.58 | 22.40 | 33.70 | 57.85
| PGD | 13.36 | 9.11 | 13.90 | 7.42 | 8.67 | 31.65
CLIP-224px | APGD | 6.32 | 4.41 | 10.11 | 4.80 | 8.16 | 27.56
| None | 119.02 | 77.21 | 59.18 | 36.73 | 39.94 | 70.00
| FGSM | 95.55 | 63.10 | 48.20 | 24.03 | 35.24 | 57.19
| PGD | 22.84 | 13.87 | 20.46 | 10.13 | 22.73 | 27.07
CLIP-336px | APGD | 12.54 | 7.15 | 15.68 | 8.08 | 9.31 | 26.73
| None | 118.69 | 78.29 | 60.92 | 38.82 | 42.43 | 74.49
| FGSM | 94.09 | 54.43 | 48.24 | 24.38 | 39.85 | 60.63
| PGD | 20.46 | 11.03 | 14.96 | 7.52 | 10.91 | 34.70
SigLIP-224px | APGD | 7.32 | 4.87 | 10.44 | 5.30 | 9.54 | 30.38
| None | 124.11 | 87.08 | 62.18 | 55.05 | 41.14 | 77.22
| FGSM | 92.39 | 57.55 | 51.38 | 32.91 | 37.28 | 62.47
| PGD | 15.69 | 8.29 | 18.04 | 9.61 | 9.98 | 35.97
SigLIP-384px | APGD | 6.90 | 3.22 | 12.72 | 6.73 | 8.77 | 31.21

####  4.1.3 Size of language model

We evaluate a series of VLMs utilizing the same vision encoder, and same LLM
architecture, with the only difference being the language model’s size.
Specifically, we examine models equipped with the Vicuna language model (Zheng
et al., 2024) in two sizes: 7B and 13B. According to the results in Table 4,
the model’s vulnerability to adversarial attacks and the significant drop in
robust accuracy remain consistent, regardless of the model’s scale. Hence,
increasing the size of the language model does not seem to enhance robustness.
One potential reason for this could be that adversarial attacks compromise the
representations from the vision encoder. As a result, LLMs even at 13B scale
may struggle to effectively interpret these flawed representations, making
robustness to image-based attacks less sensitive to language model size.
Therefore, enhancing the vision encoder’s adversarial robustness is sufficient
as shown in prior work (Schlarmann et al., 2024). Please check Appendix A for
results on other $\epsilon$ values.

Table 4: Comparison between models having different scales of language models.
Both of the models have the same vision encoder, CLIP-336, but different
scales of the LLM. Note: Higher values ($\uparrow$) indicate better
performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQav2
| None | 119.02 | 77.21 | 59.18 | 36.73 | 39.94 | 70.00
| FGSM | 95.55 | 63.10 | 48.20 | 24.03 | 35.24 | 57.19
| PGD | 22.84 | 13.87 | 20.46 | 10.13 | 22.73 | 27.07
LLaVA-7B | APGD | 12.54 | 7.15 | 15.68 | 8.08 | 9.31 | 26.73
| None | 123.71 | 77.63 | 62.86 | 40.04 | 41.19 | 75.39
| FGSM | 106.40 | 64.93 | 50.90 | 26.28 | 36.48 | 62.49
| PGD | 14.60 | 8.96 | 15.65 | 9.08 | 23.55 | 34.49
LLaVA-13B | APGD | 6.98 | 4.35 | 23.13 | 10.45 | 7.47 | 33.56

####  4.1.4 Ensemble of vision encoders

We also explore the vulnerability of VLMs that employ an ensemble of vision
encoders. Although recent studies suggest that multiple encoders can
significantly improve performance (Karamcheti et al., 2024; Kar et al., 2024),
our research aims to assess whether compromising just one encoder can affect
the entire model. This approach allows us to analyze if knowledge of the
weakest link is sufficient to compromise the entire model when an ensemble of
encoders is used. We specifically examined models combining DINOv2 with either
CLIP or SigLIP. In our experiments, we perturbed the images processed by
DINOv2 while keeping inputs to the other encoder intact. Results in Table 5
show that attacking only DINOv2 is sufficient to compromise the model under
stronger attacks, despite the other vision encoder providing clean inputs.
This highlights a significant vulnerability in ensemble approaches: even with
enhanced performance capabilities, the robustness of the entire system can be
jeopardized by targeting a single encoder. Please check Appendix A for results
on other $\epsilon$ values.

Table 5: Comparison between VLMs that have an ensemble of vision encoders. The
comparison is made when only the input to the Dino image encoder is perturbed.
Note: Higher values ($\uparrow$) indicate better performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQav2
| None | 113.75 | 74.16 | 58.88 | 15.08 | 39.30 | 74.35
| FGSM | 100.75 | 55.29 | 47.54 | 8.98 | 39.11 | 58.20
| PGD | 13.14 | 7.40 | 12.44 | 3.11 | 12.94 | 33.32
Dino+CLIP | APGD | 5.99 | 4.36 | 11.08 | 2.88 | 12.51 | 30.29
| None | 125.94 | 85.44 | 61.12 | 50.52 | 44.27 | 79.39
| FGSM | 109.79 | 73.84 | 53.94 | 40.84 | 41.84 | 67.75
| PGD | 39.76 | 22.64 | 19.30 | 12.83 | 13.37 | 39.78
Dino+SigLIP | APGD | 25.23 | 15.72 | 17.26 | 12.03 | 12.25 | 37.75

###  4.2 Prompt Formatting

Considering that our adversarial examples are generated solely by perturbing
visual inputs, we hypothesize that modifying the original prompts could be
particularly effective in countering the effects of such perturbations. We
test this hypothesis with the LLaVA 7B and LLaVA 13B models, employing
different types of prompts for COCO and VQAv2. Our evaluation includes
adversarial examples created using FGSM, PGD, and APGD attacks, with PGD, APGD
based on 100 iterations. Specific details on the prompts used and the
corresponding results are detailed in the subsequent subsections.

####  4.2.1 Captioning

Our experiments evaluated various prompt formatting strategies, including: (1)
Original \- using the original prompt; (2) Adversarial Certainty (AC) Prompt
\- explicitly informing the model that the image is adversarially perturbed;
(3) Adversarial Possibility (AP) Prompt \- suggesting the possibility that the
image might be adversarially perturbed; and (4) Random \- appending a random
sentence or string at the beginning of the prompt. These are listed in Table
15 in Appendix B. From the results presented in Fig. 1 and Table 16 in
Appendix C, it is evident that indicating the possibility of adversarial
perturbations (AP prompt) assists the model significantly more than explicitly
stating that the image is perturbed (AC prompt). Further, the improvements
from simply adding a random string or sentence are substantial, even
comparable to the effects observed with the AP prompt. This indicates that the
models pay more attention to the inputs when they struggle to establish a
clear relationship between them.

####  4.2.2 Visual Question Answering

Here, we explored four strategies: (1) Rephrase \- rephrasing the original
question to create a semantically similar question; (2) Expand \- increasing
the length of the questions; (3) Adversarial Certainty (AC) Prompt \-
explicitly informing the model that the image is adversarially perturbed; and
(4) Adversarial Possibility (AP) Prompt \- suggesting the possibility that the
image might be adversarially perturbed. We utilize a finetuned model of the
Mistral 7B LLM (Teknium, 2023) to generate questions according to the above-
mentioned strategies. All the instructions used to obtain the modified
questions are listed in Table 14 in Appendix B. According to the results
presented in Fig. 3 and Table 17 in Appendix C, simply rephrasing the
questions significantly improved performance compared to the other methods,
such as extending the question length or explicitly warning about potential
adversarial perturbations. Moreover, indicating the possibility of an
adversarial perturbation yielded the best robustness performance, reinforcing
our observations with the COCO dataset discussed earlier.

Figure 3: Performance of LLaVA-7B on VQA using questions generated by
different types of prompts.

##  5 Discussion and Conclusion

Our evaluation highlighted critical insights into how various design elements
affect the adversarial robustness of VLMs. First, we observed that vision
encoders trained across diverse data distributions only improve resistance
against simpler, less sophisticated attacks, demonstrating limited
effectiveness against more complex threats. Additionally, increasing the
resolution of image encoders did not correlate with enhanced adversarial
robustness, suggesting that benefits seen in the clean accuracy do not extend
to improved robustness. Similarly, scaling up the size of the language model
did not increase the model’s robustness to attacks, indicating that larger
models are not inherently more robust. Most notably, our results revealed that
using multiple vision encoders does not guarantee robustness; rather,
knowledge about the most vulnerable encoder is enough to compromise the entire
system.

Building on our findings, we further explored the influence of prompt
formatting on enhancing adversarial robustness. Our experiments revealed that
even naively rephrasing the questions significantly improves robustness in
VQA. Similarly, merely suggesting the possibility of an adversarial image
during captioning led to a notable performance boost. More importantly, we
found that we do not need to add additional context from the image or fine-
tune additional tokens to make models adversarially robust, as opposed to
prior work (Cui et al., 2023b; Zhang et al., 2023).

These findings underscore the critical impact of model design and prompt
formulation on a model’s robustness to adversarial attacks, demonstrating that
even minimal modifications to the textual prompt can significantly enhance the
model’s robustness against visual attacks.

##  6 Impact Statement

As VLMs see increased real-world deployment, ensuring their robustness against
adversarial attacks is critical. Our research makes two key contributions:
providing optimal model design choices for safe deployment and demonstrating
how prompt formatting can enhance adversarial robustness. Our lightweight
technique offers a practical alternative to computationally intensive
adversarial training, reducing the computational footprint. While enhancing
robustness against multimodal attacks using prompt formatting remains
unexplored, our work addresses the crucial task of defending against strong
image-based attacks that can lead to misinformation or harmful content
generation. This research aims to support future advancements in the safe
deployment of AI systems.

##  7 Acknowledgements

We acknowledge support from the Canada CIFAR AI Chair Program and from the
Canada Excellence Research Chairs Program. P.B. was supported by FRQ-S
Research Scholars Junior 1 grant 310924, and the William Dawson Scholar award.
This research was enabled in part by computational resources provided by the
Digital Research Alliance of Canada and Mila Quebec AI Institute.

## References

  * Antol et al. (2015) Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., and Parikh, D.  VQA: Visual Question Answering.  In _International Conference on Computer Vision (ICCV)_ , 2015.
  * Awadalla et al. (2023) Awadalla, A., Gao, I., Gardner, J., Hessel, J., Hanafy, Y., Zhu, W., Marathe, K., Bitton, Y., Gadre, S. Y., Sagawa, S., Jitsev, J., Kornblith, S., Koh, P. W., Ilharco, G., Wortsman, M., and Schmidt, L.  Openflamingo: An open-source framework for training large autoregressive vision-language models.  _ArXiv_ , abs/2308.01390, 2023.  URL https://api.semanticscholar.org/CorpusID:261043320.
  * Awal et al. (2024) Awal, R., Zhang, L., and Agrawal, A.  Investigating prompting techniques for zero- and few-shot visual question answering, 2024.
  * Bartoldson et al. (2024) Bartoldson, B., Diffenderfer, J., Parasyris, K., and Kailkhura, B.  Adversarial robustness limits via scaling-law and human-alignment studies.  _ArXiv_ , abs/2404.09349, 2024.
  * Croce & Hein (2020) Croce, F. and Hein, M.  Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.  In _International Conference on Machine Learning_ , 2020.
  * Cui et al. (2023a) Cui, X., Aparcedo, A., Jang, Y. K., and Lim, S.-N.  On the robustness of large multimodal models against image adversarial attacks, 2023a.
  * Cui et al. (2023b) Cui, X., Aparcedo, A., Jang, Y. K., and Lim, S.-N.  On the robustness of large multimodal models against image adversarial attacks.  _arXiv preprint arXiv:2312.03777_ , 2023b.
  * Deng et al. (2009) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pp. 248–255. Ieee, 2009.
  * Dong et al. (2023) Dong, Y., Chen, H., Chen, J., Fang, Z., Yang, X., Zhang, Y., Tian, Y., Su, H., and Zhu, J.  How robust is google’s bard to adversarial image attacks?, 2023.
  * Dosovitskiy et al. (2020) Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ , 2020.
  * Fang et al. (2022) Fang, A., Ilharco, G., Wortsman, M., Wan, Y., Shankar, V., Dave, A., and Schmidt, L.  Data determines distributional robustness in contrastive language image pre-training (clip).  In _International Conference on Machine Learning_ , 2022.
  * Goodfellow et al. (2014) Goodfellow, I. J., Shlens, J., and Szegedy, C.  Explaining and harnessing adversarial examples.  _CoRR_ , abs/1412.6572, 2014.  URL https://api.semanticscholar.org/CorpusID:6706414.
  * Gurari et al. (2018) Gurari, D., Li, Q., Stangl, A. J., Guo, A., Lin, C., Grauman, K., Luo, J., and Bigham, J. P.  Vizwiz grand challenge: Answering visual questions from blind people, 2018.
  * Kar et al. (2024) Kar, O. F., Tonioni, A., Poklukar, P., Kulshrestha, A., Zamir, A., and Tombari, F.  BRAVE: Broadening the visual encoding of vision-language models.  _arXiv preprint arXiv:2404.07204_ , 2024.
  * Karamcheti et al. (2024) Karamcheti, S., Nair, S., Balakrishna, A., Liang, P., Kollar, T., and Sadigh, D.  Prismatic vlms: Investigating the design space of visually-conditioned language models.  In _International Conference on Machine Learning (ICML)_ , 2024.
  * Laurenccon et al. (2023) Laurenccon, H., Saulnier, L., Tronchon, L., Bekman, S., Singh, A., Lozhkov, A., Wang, T., Karamcheti, S., Rush, A. M., Kiela, D., Cord, M., and Sanh, V.  Obelisc: An open web-scale filtered dataset of interleaved image-text documents.  _ArXiv_ , abs/2306.16527, 2023.  URL https://api.semanticscholar.org/CorpusID:259287020.
  * Li et al. (2023) Li, J., Li, D., Savarese, S., and Hoi, S. C. H.  Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.  In _International Conference on Machine Learning_ , 2023.  URL https://api.semanticscholar.org/CorpusID:256390509.
  * Lin et al. (2014) Lin, T.-Y., Maire, M., Belongie, S. J., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L.  Microsoft coco: Common objects in context.  In _European Conference on Computer Vision_ , 2014.
  * Liu et al. (2023) Liu, H., Li, C., Wu, Q., and Lee, Y. J.  Visual instruction tuning.  In _NeurIPS_ , 2023.
  * Madry et al. (2017) Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A.  Towards deep learning models resistant to adversarial attacks.  _ArXiv_ , abs/1706.06083, 2017.  URL https://api.semanticscholar.org/CorpusID:3488815.
  * Mao et al. (2023) Mao, C., Geng, S., Yang, J., Wang, X., and Vondrick, C.  Understanding zero-shot adversarial robustness for large-scale models.  In _The Eleventh International Conference on Learning Representations_ , 2023.  URL https://openreview.net/forum?id=P4bXCawRi5J.
  * Marino et al. (2019) Marino, K., Rastegari, M., Farhadi, A., and Mottaghi, R.  Ok-vqa: A visual question answering benchmark requiring external knowledge.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019.
  * Nguyen et al. (2022) Nguyen, T., Ilharco, G., Wortsman, M., Oh, S., and Schmidt, L.  Quality not quantity: On the interaction between dataset design and robustness of clip.  _ArXiv_ , abs/2208.05516, 2022.
  * Oquab et al. (2023) Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.  Dinov2: Learning robust visual features without supervision.  _arXiv preprint arXiv:2304.07193_ , 2023.
  * Plummer et al. (2015) Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., and Lazebnik, S.  Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models.  In _2015 IEEE International Conference on Computer Vision (ICCV)_ , pp. 2641–2649, December 2015.  doi: 10.1109/ICCV.2015.303.
  * Radford et al. (2021) Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I.  Learning transferable visual models from natural language supervision.  In _International Conference on Machine Learning_ , 2021.  URL https://api.semanticscholar.org/CorpusID:231591445.
  * Schlarmann et al. (2024) Schlarmann, C., Singh, N. D., Croce, F., and Hein, M.  Robust clip: Unsupervised adversarial fine-tuning of vision embeddings for robust large vision-language models, 2024.
  * Singh et al. (2019) Singh, A., Natarjan, V., Shah, M., Jiang, Y., Chen, X., Parikh, D., and Rohrbach, M.  Towards vqa models that can read.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 8317–8326, 2019.
  * Teknium (2023) Teknium.  Openhermes-2-mistral-7b.  https://huggingface.co/teknium/OpenHermes-2-Mistral-7B, 2023.
  * Tu et al. (2023) Tu, W., Deng, W., and Gedeon, T.  A closer look at the robustness of contrastive language-image pre-training (CLIP).  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.
  * Wang et al. (2023) Wang, Z., Pang, T., Du, C., Lin, M., Liu, W., and Yan, S.  Better diffusion models further improve adversarial training.  In _International Conference on Machine Learning_ , pp. 36246–36263. PMLR, 2023.
  * Yin et al. (2023) Yin, Z., Ye, M., Zhang, T., Du, T., Zhu, J., Liu, H., Chen, J., Wang, T., and Ma, F.  VLATTACK: Multimodal adversarial attacks on vision-language tasks via pre-trained models.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.  URL https://openreview.net/forum?id=qBAED3u1XZ.
  * Zhai et al. (2023) Zhai, X., Mustafa, B., Kolesnikov, A., and Beyer, L.  Sigmoid loss for language image pre-training.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 11975–11986, 2023.
  * Zhang et al. (2023) Zhang, J., Ma, X., Wang, X., Qiu, L., Wang, J., Jiang, Y.-G., and Sang, J.  Adversarial prompt tuning for vision-language models, 2023.
  * Zhao et al. (2023) Zhao, Y., Pang, T., Du, C., Yang, X., Li, C., man Cheung, N., and Lin, M.  On evaluating adversarial robustness of large vision-language models.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.
  * Zheng et al. (2024) Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al.  Judging llm-as-a-judge with mt-bench and chatbot arena.  _Advances in Neural Information Processing Systems_ , 36, 2024.

##  Appendix A Model Design Choice Results

We provide results for studying the impact of various model design choices for
$\epsilon=4/255$ and $16/255$ here.

###  A.1 Impact of Vision Encoder

We can observe that for a lower $\epsilon$ value, i.e., $4/255$ CLIP performs
better. However, for higher $\epsilon$ values, i.e. $8/255$ and $16/255$,
SigLIP performs better.

Table 6: Comparison between VLMs having different image encoders but the same
language model for $\epsilon=4/255$. All of them have the same language model:
Vicuna v1.5 7B. Note: Higher values ($\uparrow$) indicate better performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQAv2
| None | 116.35 | 76.15 | 59.48 | 37.10 | 41.18 | 73.89
| FGSM | 106.01 | 64.45 | 52.18 | 26.87 | 36.93 | 63.55
| PGD | 89.95 | 54.54 | 44.40 | 6.73 | 32.06 | 53.81
CLIP | APGD | 87.07 | 50.51 | 42.52 | 19.03 | 8.80 | 50.16
| None | 118.69 | 78.29 | 60.92 | 38.82 | 42.43 | 74.49
| FGSM | 99.75 | 60.60 | 49.84 | 27.24 | 39.74 | 61.75
| PGD | 68.94 | 38.42 | 33.30 | 9.54 | 26.65 | 44.51
SigLIP | APGD | 59.67 | 33.12 | 14.45 | 11.89 | 24.12 | 41.87
| None | 104.84 | 54.78 | 57.00 | 10.37 | 38.07 | 64.80
| FGSM | 77.68 | 37.81 | 39.78 | 7.28 | 32.50 | 45.40
| PGD | 4.86 | 3.13 | 9.80 | 1.99 | 10.91 | 25.67
Dinov2 | APGD | 2.45 | 2.17 | 8.00 | 1.96 | 10.69 | 23.29
| None | 101.59 | 54.92 | 56.34 | 10.70 | 39.29 | 68.36
| FGSM | 71.67 | 34.74 | 38.44 | 6.70 | 31.62 | 45.37
| PGD | 11.17 | 5.62 | 11.28 | 2.43 | 11.90 | 15.00
ImageNet | APGD | 5.24 | 3.69 | 9.86 | 2.04 | 10.80 | 17.14

Table 7: Comparison between VLMs having different image encoders but the same
language model for $\epsilon=16/255$. All of them have the same language
model: Vicuna v1.5 7B. Note: Higher values ($\uparrow$) indicate better
performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQAv2
| None | 116.35 | 76.15 | 59.48 | 37.10 | 41.18 | 73.89
| FGSM | 93.27 | 56.35 | 48.52 | 20.22 | 36.73 | 59.99
| PGD | 10.32 | 6.22 | 11.88 | 5.87 | 8.23 | 29.57
CLIP | APGD | 3.33 | 2.57 | 8.40 | 3.84 | 7.89 | 23.84
| None | 118.69 | 78.29 | 60.92 | 38.82 | 42.43 | 74.49
| FGSM | 88.06 | 50.05 | 48.62 | 22.22 | 40.80 | 60.55
| PGD | 11.77 | 6.59 | 12.45 | 6.57 | 10.32 | 31.04
SigLIP | APGD | 4.31 | 2.79 | 7.88 | 3.78 | 8.97 | 23.50
| None | 104.84 | 54.78 | 57.00 | 10.37 | 38.07 | 64.80
| FGSM | 81.38 | 39.35 | 41.18 | 7.96 | 36.22 | 48.61
| PGD | 3.00 | 1.48 | 7.70 | 1.54 | 10.68 | 24.78
Dinov2 | APGD | 1.57 | 1.12 | 6.34 | 1.34 | 9.70 | 20.73
| None | 101.59 | 54.92 | 56.34 | 10.70 | 39.29 | 68.36
| FGSM | 62.62 | 29.90 | 39.42 | 7.20 | 33.98 | 46.78
| PGD | 3.12 | 2.13 | 8.10 | 1.64 | 9.34 | 22.69
ImageNet | APGD | 2.13 | 0.95 | 5.84 | 1.80 | 9.98 | 18.99

###  A.2 Resolution of Vision Encoder

We can observe that at a lower $\epsilon$ value of $4/255$, lower resolution
models are better. However, at a higher $\epsilon$ value of $16/255$, the
effectiveness of increased resolution for both CLIP and SigLIP models becomes
task-dependent.

Table 8: Comparison between VLMs having different input resolutions of CLIP
and SigLIP for $\epsilon=4/255$. All of them have the same language model:
Vicuna v1.5 7B. Note: Higher values ($\uparrow$) indicate better performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQAv2
| None | 116.35 | 76.15 | 59.48 | 37.10 | 41.18 | 73.89
| FGSM | 106.01 | 64.45 | 52.18 | 26.87 | 36.93 | 63.55
| PGD | 89.95 | 54.54 | 44.40 | 6.73 | 32.06 | 53.81
CLIP-224px | APGD | 87.07 | 50.51 | 42.52 | 19.03 | 8.80 | 50.16
| None | 119.02 | 77.21 | 59.18 | 36.73 | 39.94 | 70.00
| FGSM | 96.79 | 64.25 | 49.70 | 25.32 | 33.92 | 56.52
| PGD | 18.54 | 13.57 | 16.26 | 9.44 | 10.67 | 28.60
CLIP-336px | APGD | 30.20 | 22.11 | 22.86 | 11.73 | 22.86 | 28.76
| None | 118.69 | 78.29 | 60.92 | 38.82 | 42.43 | 74.49
| FGSM | 99.75 | 60.60 | 49.84 | 27.24 | 39.74 | 61.75
| PGD | 68.94 | 38.42 | 33.30 | 9.54 | 26.65 | 44.51
SigLIP-224px | APGD | 59.67 | 33.12 | 14.45 | 11.89 | 24.12 | 41.87
| None | 124.11 | 87.08 | 62.18 | 55.05 | 41.14 | 77.22
| FGSM | 93.46 | 57.28 | 50.88 | 36.18 | 35.50 | 63.22
| PGD | 25.51 | 13.84 | 20.32 | 13.38 | 11.51 | 38.15
SigLIP-384px | APGD | 12.53 | 8.40 | 16.34 | 9.10 | 9.52 | 34.83

Table 9: Comparison between VLMs having different input resolutions of CLIP
and SigLIP for $\epsilon=16/255$. All of them have the same language model:
Vicuna v1.5 7B. Note: Higher values ($\uparrow$) indicate better performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQAv2
| None | 116.35 | 76.15 | 59.48 | 37.10 | 41.18 | 73.89
| FGSM | 93.27 | 56.35 | 48.52 | 20.22 | 36.73 | 59.99
| PGD | 10.32 | 6.22 | 11.88 | 5.87 | 8.23 | 29.57
CLIP-224px | APGD | 3.33 | 2.57 | 8.40 | 3.84 | 7.89 | 23.84
| None | 119.02 | 77.21 | 59.18 | 36.73 | 39.94 | 70.00
| FGSM | 101.98 | 56.08 | 49.24 | 22.20 | 37.39 | 58.30
| PGD | 10.08 | 5.25 | 17.48 | 6.70 | 8.52 | 24.39
CLIP-336px | APGD | 13.88 | 9.05 | 13.26 | 7.34 | 27.54 | 19.98
| None | 118.69 | 78.29 | 60.92 | 38.82 | 42.43 | 74.49
| FGSM | 88.06 | 50.05 | 48.62 | 22.22 | 40.80 | 60.55
| PGD | 11.77 | 6.59 | 12.45 | 6.57 | 10.32 | 31.04
SigLIP-224px | APGD | 4.31 | 2.79 | 7.88 | 3.78 | 8.97 | 23.50
| None | 124.11 | 87.08 | 62.18 | 55.05 | 41.14 | 77.22
| FGSM | 94.90 | 57.40 | 52.24 | 30.93 | 39.77 | 63.53
| PGD | 9.53 | 5.17 | 14.48 | 8.26 | 9.19 | 33.16
SigLIP-384px | APGD | 3.15 | 1.75 | 9.14 | 4.00 | 7.85 | 27.82

###  A.3 Size of Language Model

Here we can observe that increasing the model size only helps in gaining
robustness against weaker attacks (FGSM). However, the vulnerability and drop
in performance against iterative attacks (PGD and APGD) remain almost the same
regardless of the model’s size.

Table 10: Comparison between models having different scales of LLM but the
same vision encoder for $\epsilon=4/255$. Note: Higher values ($\uparrow$)
indicate better performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQAv2
| None | 119.02 | 77.21 | 59.18 | 36.73 | 39.94 | 70.00
| FGSM | 96.79 | 64.25 | 49.70 | 25.32 | 33.92 | 56.52
| PGD | 18.54 | 13.57 | 16.26 | 9.44 | 10.67 | 28.60
LLaVA-7B | APGD | 30.20 | 22.11 | 22.86 | 11.73 | 22.86 | 28.76
| None | 123.71 | 77.63 | 62.86 | 40.04 | 41.19 | 75.39
| FGSM | 123.71 | 77.63 | 62.86 | 40.04 | 41.19 | 75.39
| PGD | 18.54 | 13.57 | 16.26 | 9.44 | 10.67 | 28.60
LLaVA-13B | APGD | 30.20 | 22.11 | 21.30 | 11.73 | 22.86 | 28.76

Table 11: Comparison between models having different scales of LLM but the
same vision encoder for $\epsilon=16/255$. Note: Higher values ($\uparrow$)
indicate better performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQAv2
| None | 119.02 | 77.21 | 59.18 | 36.73 | 39.94 | 70.00
| FGSM | 101.98 | 56.08 | 49.24 | 22.20 | 37.39 | 58.30
| PGD | 10.08 | 5.25 | 17.48 | 6.70 | 8.52 | 24.39
LLaVA-7B | APGD | 13.88 | 9.05 | 13.26 | 7.34 | 27.54 | 19.98
| None | 123.71 | 77.63 | 62.86 | 40.04 | 41.19 | 75.39
| FGSM | 99.83 | 58.40 | 52.90 | 24.85 | 37.89 | 60.98
| PGD | 10.08 | 9.05 | 13.26 | 6.70 | 8.52 | 24.39
LLaVA-13B | APGD | 13.88 | 5.25 | 17.48 | 7.34 | 27.54 | 19.98

###  A.4 Ensemble of Vision Encoders

The observations for both $\epsilon=4/255$ and $16/255$ are same as for
$\epsilon=8/255$. Targeting the weakest image encoder is enough to jeopardize
the entire system. Conversely, having the strongest vision encoder in the
ensemble ensures the best robust performance.

Table 12: Comparison between VLMs that have an ensemble of vision encoders.
The comparison is made when only the input to the Dino image encoder is
perturbed for $\epsilon=4/255$. Note: Higher values ($\uparrow$) indicate
better performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQAv2
| None | 113.75 | 74.16 | 58.88 | 15.08 | 39.30 | 74.35
| FGSM | 99.01 | 57.19 | 47.14 | 8.92 | 38.39 | 58.50
| PGD | 21.00 | 12.22 | 14.96 | 3.34 | 13.95 | 35.03
DinoCLIP | APGD | 10.71 | 7.12 | 12.96 | 3.30 | 12.49 | 33.10
| None | 125.94 | 85.44 | 61.12 | 50.52 | 44.27 | 79.39
| FGSM | 107.87 | 74.10 | 52.92 | 40.36 | 40.77 | 67.24
| PGD | 52.89 | 32.14 | 23.10 | 15.73 | 14.78 | 42.87
DinoSigLIP | APGD | 35.34 | 22.86 | 19.18 | 13.83 | 12.96 | 39.58

Table 13: Comparison between VLMs that have an ensemble of vision encoders.
The comparison is made when only the input to the Dino image encoder is
perturbed for $\epsilon=16/255$. Note: Higher values ($\uparrow$) indicate
better performance.

|  | Task
---|---|---
| Attack | COCO | Flickr30k | OK-VQA | TextVQA | VizWiz | VQAv2
| None | 113.75 | 74.16 | 58.88 | 15.08 | 39.30 | 74.35
| FGSM | 103.37 | 57.80 | 48.38 | 9.29 | 40.30 | 58.87
| PGD | 8.14 | 5.73 | 11.22 | 2.86 | 12.12 | 32.38
DinoCLIP | APGD | 3.11 | 2.37 | 8.94 | 2.57 | 12.63 | 27.47
| None | 125.94 | 85.44 | 61.12 | 50.52 | 44.27 | 79.39
| FGSM | 111.52 | 74.29 | 54.76 | 42.78 | 42.74 | 68.40
| PGD | 31.29 | 17.58 | 18.52 | 12.72 | 12.61 | 39.06
DinoSigLIP | APGD | 17.77 | 10.57 | 14.86 | 10.94 | 12.38 | 35.66

##  Appendix B Prompt Formatting

Task | Instruction
---|---
Rephrase |  | You will be given a question. Your task is to rephrase the question so that it
---
is semantically similar to the original question and will have the same answer
as the original question.
Expand |  | You will be given a short question. Your task is to generate a longer question so that it
---
is semantically similar to the original question and will have the same answer
as the original question.
AC |  | You will be given a question. However, the image associated with the question will be
---
adversarially perturbed. Your task is to generate a longer question so that it
is semantically
similar to the original question and will have the same answer as the original
question.
AP |  | You will be given a question. However, the image associated with the question could be
---
adversarially perturbed. Your task is to generate a longer question so that it
is semantically
similar to the original question and will have the same answer as the original
question.

Table 14: Instructions used to obtain the modified questions for VQA.

Prompt Type | Prompt |
---|---|---
Original | Provide a short caption for this image. |
AC |  | Consider the given image being adversarially perturbed. Provide a short caption for this image.
---
AP |  | Given image could be adversarially perturbed. Provide a short caption for this image.
---
Random sent. |  | Clouds drift quietly over the ancient, forgotten city. Provide a short caption for this image.
---
Random str. |  | ryFo8ZVcyNMtLgryNOg64UTjySyEb79e5aq6IJxGuz0GzWNtoz. Provide a short caption for this image.
---

Table 15: Various types of prompts tested for image captioning.

##  Appendix C Prompt Formatting Results

Table 16: Performance of LLaVA models on image captioning (COCO) when adversarially perturbed images (using $\epsilon=8/255$) are provided along with different types of prompts. Note: Higher values ($\uparrow$) indicate better performance. | Prompt | FGSM | PGD | APGD | Clean
---|---|---|---|---|---
LLaVA 7B | Original | 95.55 | 22.84 | 12.54 | 119.02
AC | 63.86 | 60.01 | 54.05 | 64.11
AP | 105.41 | 101.61 | 91.46 | 112.78
Random str | 108.23 | 105.35 | 94.61 | 120.90
Random sent | 101.12 | 97.11 | 88.45 | 108.00
Clean Acc | 119.02
LLaVA 13B | Original | 106.40 | 14.60 | 6.98 | 123.71
AC | 113.77 | 106.40 | 114.65 | 122.10
AP | 114.48 | 108.83 | 113.54 | 125.28
Random str | 110.74 | 105.15 | 111.69 | 120.49
Random sent | 113.29 | 106.71 | 111.13 | 120.72
Clean acc | 123.71
Table 17: Performance of LLaVA models on VQAv2 when adversarially perturbed images (using $\epsilon=8/255$) are provided along with questions generated using different types of prompts. Note: Higher values ($\uparrow$) indicate better performance. | Prompt | FGSM | PGD | APGD | Clean
---|---|---|---|---|---
LLaVA 7B | Original | 57.19 | 27.07 | 26.73 | 70.00
Rephrase | 59.01 | 58.03 | 48.84 | 68.30
AC | 60.21 | 58.82 | 50.68 | 69.99
AP | 60.13 | 58.81 | 49.95 | 69.78
Expand | 48.59 | 48.54 | 42.24 | 57.14
Clean Acc | 70.00
LLaVA 13B | Original | 62.49 | 34.49 | 33.56 | 75.39
Rephrase | 59.01 | 60.05 | 54.77 | 71.02
AC | 51.38 | 61.49 | 55.56 | 72.00
AP | 63.59 | 61.29 | 63.2 | 71.79
Expand | 53.03 | 50.03 | 45.93 | 58.59
Clean Acc | 75.39