# Convective evaporation of vertical films

François Boulogne  Laboratoire de Physique des Solides, CNRS, Univ. Paris-Sud,
Université Paris-Saclay, Orsay 91405, France  Benjamin Dollet  Univ. Grenoble
Alpes, CNRS, LIPhy, 38000 Grenoble, France

###### Abstract

Motivated by the evaporation of soap films, which has a significant effect on
their lifetime, we performed an experimental study on the evaporation of
vertical surfaces with model systems based on hydrogels. From the analogy
between heat and mass transfer, we adopt a model describing the natural
convection in the gas phase due to a density contrast between dry and
saturated air. Our measurements show a good agreement with this model, both in
terms of scaling law with the Grashof number and in terms of order of
magnitude. We discuss the corrections to take into account, notably the
contribution of edge effects, which have a small but visible contribution when
lateral and bottom surface areas are not negligible compared to the main
evaporating surface area.

##  1 Introduction

Giant soap films are spectacular structures since they are uncommon in nature
[1, 2, 3]. Indeed, soap films are well-known to be fragile and short standing.
To maintain such large films artificially, the soap solution is constantly
injected at the top to balance the effects of drainage and evaporation [1, 2].
These are common in scientific outreach events, as shown in Fig. 1 where they
attract interest from the public due to its gigantic size. Film evaporation
depends on the environmental conditions such as the weather or air currents in
the room. Thus, daily fluctuations of these parameters influence the success
of the demonstration.

Several scientific studies report effects of evaporation on the soap film
stability [4]. Yaminsky et al. evidenced that impurities influence the
lifetime of soap film only if placed in an unsaturated atmosphere [5]. In
addition, Li et al. pointed out that the nonuniformity of the evaporative flux
induces Marangoni effects, which trigger the film burst [6]. Moreover,
chemistry can have a significant role on the evaporation kinetics as dense
monolayers at interfaces can reduce significantly the evaporation rate [7, 8].
Therefore, the complexity in studying evaporation on soap films arises from
the interplay of chemistry, hydrodynamics of liquid films with interfacial
effects, and the dynamics of evaporation.

In 1887, Maxwell described the evaporation of a suspended spherical drop as a
diffusive process of water in the gas phase. While this description is
successful in many situations such as the evaporation of small sessile drops
[9, 10], natural convection has been recently revealed in experimental studies
with various liquids [11, 12, 13, 14, 15, 16, 17, 18, 19].

Indeed, water vapor is less dense than dry air. Above a critical size of the
evaporative surface, a convective gas flow takes place, which affects the
vapor concentration field. Typically, the critical size above which convection
can be expected is about a few millimeters in room conditions for water.
Models have been developed to describe particular configurations such as the
evaporation of pendant drops [20] and horizontal circular disks [21]. Similar
convection effects occur in the dissolution of sessile drops [22, 23].

Figure 1: Giant soap film of $10$ m height that brings the question of
evaporation on its stability. The picture is extracted from Ref. [2].
©European Physical Society. Reproduced by permission of IOP Publishing. All
rights reserved. Credit F. Mondot. The photograph was taken at Collège
Villeneuve, Grenoble.

In this paper, we aim to describe the convective evaporation of vertical
films. First, based on an analogy with heat transfer on flat vertical
surfaces, we present the theory describing the natural convective flow in such
geometry with a prefactor determined numerically. Because the evaporation of
soap films is particularly challenging to study due to their limited lifetime
and to the complex interplay between chemical and hydrodynamical effects, we
make the choice to perform experiments on a model system, which consists of
hydrogels. We compare our measurements to the theoretical prediction and we
concluded on the significance for future soap films studies.

##  2 Experimental methods

###  2.1 Hydrogels

Figure 2: Schematics of the experimental setup. The left part shows the
principle of the humidity controller and the right part is the box in which a
hydrogel is hung on a precision scale mounted on pillars.

Hydrogels are synthesized in polydimethyl acrylamid (PDMA) [24]. The quantity
of the monomer (N,N-dimethylacrylamide denoted DMA) is set by the mass ratio
$m_{\rm DMA}/(m_{\rm DMA}+m_{w})=0.1$, where $m_{w}$ is the mass of pure
water. The crosslinker (N,N’-methylene-bis-acrylamide denoted MBA) is added at
a molar concentration ratio of $[\rm{MBA}]/[\rm{DMA}]=2\times 10^{-2}$.
Finally, a quantity of initiators (potassium persulfate and
N,N,N’,N’-tetramethylethylenediamine) is set to a molar ratio of 1% of the
monomer quantity. All the chemical are purchased from Sigma-Aldrich (France).

The solution is poured in cells made of two glass slides separated by a rubber
of about $3$ mm in thickness, and hold with drawing clips. The cell is sealed
with a piece of parafilm and left in room conditions for 24 h to complete the
reaction. Then, the gel is carefully unmolded and placed in a tank filled with
pure water for three days to swell the hydrogel.

The hydrogel is cut with a razor blade in a rectangular shape of the desired
size. The swollen hydrogel has a thickness $b=4.5\pm 0.5$ mm.

###  2.2 Evaporative flux measurements

The full setup is represented in figure 2. Our experiments of controlled
evaporation are performed in a box made in polycarbonate with dimensions of
$50\times 50\times 50$ cm3. The total evaporative flux of the slab of hydrogel
is measured from the time evolution of its weight. A precision scale (Ohaus
Pioneer 210 g) with a precision of $0.1$ mg is used with the hook available
below the apparatus. To hang the hydrogel on this hook, we made a clip with
two polycarbonate rectangular plates covered with sand papers. These two parts
are placed on the top of the hydrogel and tightened with nylon screws. This
clip is then attached to the hook on the scale with a thin wire.

The scale is interfaced with a Python code using the pyserial library to
record the weight every second. The humidity is regulated with a PID
controller based on an Arduino Uno and a humidity sensor (Honeywell
HIH-4021-003) positioned far from the evaporating surface. Dry air is produced
by circulating ambient air with an air pump (Tetra APS 300) in a container
filled with desiccant made of anhydrous calcium sulfate (Drierite). Moist air
is obtained by bubbling air in water. The relative humidity is set to
$R_{H}=50$% in all of our experiments.

###  2.3 Parameters and order of magnitudes

Buoyancy is the driving force of the free convection flow. It is therefore
necessary to estimate the density contrast between the ambient air and the air
saturated in water vapor. The air density $\rho$ at a pressure $P_{0}$, a
temperature $T$ and a relative humidity $R_{H}$ is given by [25]

$\rho=\frac{1}{z_{m}}\frac{P_{0}}{{\cal
R}T}M_{d}\left[1-fR_{H}\left(1-\frac{M_{w}}{M_{d}}\right)\frac{P_{s}}{P_{0}}\right],$
(1)

where $z_{m}$ and $f$ are the compressibility and enhancement factors, ${\cal
R}$ the ideal gas coefficient, $M_{d}$ and $M_{w}$ the molar density of dry
and saturated air, and $P_{s}$ the saturated pressure. At room temperature, we
have $f=1$, $z_{m}=1$ [25] and a saturated pressure $P_{s}=2.3$ kPa [26].
Thus, the density contrast at a relative humidity $R_{H}=0.5$ is
$|\rho_{s}-\rho_{\infty}|/\rho_{\infty}=5\times 10^{-3}$. In addition, the
kinematic viscosity is $\nu=1.5\times 10^{-5}$ m2/s and the diffusion
coefficient of water vapor in air is ${\cal D}=1\times 10^{-5}$ m2/s. To
justify the Boussinesq approximation used in the following Section 3, it is
useful to estimate the typical relative variations of the viscosity and
diffusivity. According to Tsilingiris [25], the relative dynamical viscosity
variation is $1.2\times 10^{-2}$ and the relative diffusivity variation is
$4\times 10^{-3}$, which are both weak.

The Grashof number ${\rm Gr}$ is defined as the ratio between the buoyant
forces driving convection and the viscous forces damping this flow, i.e.

${\rm Gr}=\frac{g}{\nu^{2}}\frac{\rho_{\infty}-\rho_{s}}{\rho_{\infty}}{\cal
L}^{3},$ (2)

where ${\cal L}$ is the characteristic vertical length scale of the surface of
evaporation. This Grashof number will be derived from the equations of
hydrodynamics in Section 3.2.

This number predicts the dynamics of evaporation [11, 12, 13, 14, 15, 16, 17,
19]. For small Grashof numbers, the evaporation is diffusive such that the
ambient air can be considered as quiescent. However, for large Grashof
numbers, convection takes place. The transition can be written as a critical
length scale ${\cal L}^{\star}$ defined for ${\rm Gr}=1$, i.e.

${\cal
L}^{\star}=\left(\frac{g}{\nu^{2}}\frac{\rho_{\infty}-\rho_{s}}{\rho_{\infty}}\right)^{-1/3}.$
(3)

For water in room conditions with a relative humidity of ${\cal R}_{H}=50$%,
${\cal L}^{\star}$ is typically $2$ mm. Thus, for vertical film of few
centimeters or more in height, convection is expected to produce an effect on
the evaporative flux.

In the next Section, we recall the equations describing the natural convective
flow near a vertical surface. From these equations, we will derive the
evaporative flux that can be compared with experiments.

##  3 Theory

Figure 3: Sketch presenting the notations used in the theory to describe the
gas flow near the vertical hydrogel represented in blue.

We consider the convective evaporation of a vertical rectangular surface of
width $w$, height $h$ and thickness $b$. The model focuses on the flow due to
the largest surfaces of area $wh$, referred thereafter as plates. As a first
approximation, the flow is assumed bidimensional, independent on $z$. The
effect of the two vertical edges of area $hb$ will be discussed afterwards.
The velocity field in the gas phase is denoted
${\bm{u}}=u{\bm{e}}_{x}+v{\bm{e}}_{y}$. All the notations are presented in
Fig. 3.

Equations of mass transfer due to evaporation are analogous to heat transfer.
Therefore, we closely follow the derivations of Schmidt and Beckmann on heat
transfer published in 1930 [27], also detailed later by Ostrach [28]. The
reader may also refer to textbooks such as [29]. We choose to rewrite in
details these derivations with notations adapted to mass transfer for the
convenience of the reader.

###  3.1 Formulation of the problem

We denote $\rho_{s}$ the density of saturated air corresponding to a vapor
concentration $c_{s}$. We assume that the gas phase obeys the ideal gas law so
that the density $\rho$ varies linearly with the vapor concentration $c$,

$\rho(c)=\rho_{0}-\Delta\rho\frac{c}{c_{s}},$ (4)

with $\Delta\rho=\rho_{0}-\rho_{s}$ and $\rho_{0}$ the density of dry air.

In Cartesian coordinates, the continuity equation is

$\frac{\partial u}{\partial x}+\frac{\partial v}{\partial y}=0.$ (5)

The momentum equation is written within the Boussinesq approximation, which
considers that the gas density is constant and equal to the density
$\rho_{\infty}$ far from the surface, except in the term driving the
convection by a buoyancy effect, namely $\rho(c)g$. Assuming a stationary
flow, we have

$\rho_{\infty}\left(u\frac{\partial u}{\partial x}+v\frac{\partial u}{\partial
y}\right)=-\rho(c)g+\mu\left(\frac{\partial^{2}u}{\partial
x^{2}}+\frac{\partial^{2}u}{\partial y^{2}}\right),$ (6)

where $\mu$ is the gas viscosity. The term $\partial^{2}u/\partial x^{2}$ is
negligible compared to $\partial^{2}u/\partial y^{2}$ due to the boundary
layer geometry present in the limit of large Grashof numbers.

The volatile solvent concentration field follows a diffusion-convection
equation

$u\frac{\partial c}{\partial x}+v\frac{\partial c}{\partial y}={\cal
D}\frac{\partial^{2}c}{\partial y^{2}},$ (7)

where ${\cal D}$ is the diffusion coefficient of the vapor in the gas phase.

Equations (5-7) must satisfy five boundary conditions. Two are related to the
concentration fields with a saturated vapor concentration near the surface and
a concentration $c_{\infty}$ far from it. In addition, three hydrodynamical
conditions are set, with a no-slip condition on the surface for both
components $u$ and $v$, and a vanishing vertical component $u$ far from the
surface. All these conditions write

$\begin{split}c(x,y=0)&=c_{s},\\\
c(x,y\rightarrow\infty)&=c_{\infty},\end{split}\qquad\begin{split}u(y=0)&=0,\\\
v(y=0)&=0,\\\ u(y\rightarrow\infty)&=0.\end{split}$ (8)

###  3.2 Non-dimensionalized equations

To solve the set of equations (5, 6, 7) with the boundary conditions (8), we
non-dimensionalize the system. First, we define the dimensionless
concentration $\tilde{c}=(c-c_{\infty})/(c_{s}-c_{\infty})$ and the
streamfunction $\Psi$ as

$u=\frac{\partial\Psi}{\partial y},\qquad v=-\frac{\partial\Psi}{\partial x}.$
(9)

Therefore, equation (7) becomes

$\frac{\partial\Psi}{\partial y}\frac{\partial\tilde{c}}{\partial
x}+\frac{\partial\Psi}{\partial x}\frac{\partial\tilde{c}}{\partial y}={\cal
D}\frac{\partial^{2}\tilde{c}}{\partial y^{2}},$ (10)

and the momentum equation (6) is rewritten as

$\frac{\partial\Psi}{\partial y}\frac{\partial^{2}\Psi}{\partial x\partial
y}-\frac{\partial\Psi}{\partial x}\frac{\partial^{2}\Psi}{\partial
y^{2}}=-\frac{\rho_{\infty}-\rho_{s}}{\rho_{\infty}}g\tilde{c}+\nu\frac{\partial^{3}\Psi}{\partial
y^{3}},$ (11)

where $\nu=\mu/\rho_{\infty}$ and with the boundary conditions

$\begin{split}\tilde{c}(x,y=0)&=1,\\\
\tilde{c}(x,y\rightarrow\infty)&=0,\end{split}\qquad\begin{split}\frac{\partial\Psi}{\partial
y}(x,y=0)&=0,\\\ \frac{\partial\Psi}{\partial x}(x,y=0)&=0,\\\
\frac{\partial\Psi}{\partial y}(x,y\rightarrow\infty)&=0.\end{split}$ (12)

We introduce the self-similar variable

$\xi=\left(\frac{g}{4\nu^{2}}\frac{\rho_{\infty}-\rho_{s}}{\rho_{\infty}}\right)^{1/4}\frac{y}{x^{1/4}},$
(13)

that we can also write $\xi=4^{-1/4}{\rm Gr}_{x}^{1/4}y/x$ with the local
Grashof number defined as

${\rm
Gr}_{x}=\frac{g}{\nu^{2}}\frac{\rho_{\infty}-\rho_{s}}{\rho_{\infty}}x^{3}.$
(14)

For $x=h={\cal L}^{\star}$, we recover the definition of the global Grashof
number, as defined by Eq. (2).

We rewrite the concentration field $\tilde{c}(x,y)=\hat{c}(\xi)$ and the
streamfunction $\Psi=4^{3/4}\nu\,{\rm Gr}_{x}^{1/4}\,f(\xi)$ where $f$ is the
dimensionless streamfunction. Equations (10) and (11) become respectively

$\displaystyle\hat{c}^{\prime\prime}+3\,{\rm Sc}\,f\hat{c}^{\prime}=0,$ (15a)
$\displaystyle f^{\prime\prime\prime}+3ff^{\prime\prime}-2f^{\prime
2}+\hat{c}=0,$ (15b)

where we introduced the Schmidt number ${\rm Sc}=\nu/{\cal D}$. The related
boundary conditions (12) become

$\begin{split}\hat{c}(\xi=0)=1,\\\
\hat{c}(\xi\rightarrow\infty)=0,\end{split}\qquad\begin{split}f^{\prime}(\xi=0)&=0,\\\
f(\xi=0)&=0,\\\ f^{\prime}(\xi\rightarrow\infty)&=0.\end{split}$ (16)

###  3.3 Resolution

Figure 4: Concentration and velocity fields near a vertical plate with
parameters corresponding to our experimental conditions obtained from a
numerical integration. The dimensionless concentration $\tilde{c}$ is
indicated with the colored background and the velocity field ${\bm{u}}$ is
represented by the arrows. (a) The axes respect the orthonormality and (b) the
$y$-axis is stretched for the sake of legibility. The white domain represents
the hydrogel.

We solve the system of ordinary differential equations (15) with the
conditions (16). As two of these conditions are specified at
$\xi\rightarrow\infty$, we use a shooting method by guessing the values of
$\hat{c}_{0}^{\prime}=\hat{c}^{\prime}(\xi=0)=({\rm d}\hat{c}/{\rm
d}\xi)|_{\xi=0}$ and $\hat{f}^{\prime\prime}(\xi=0)$. A fast convergence
toward a solution satisfying $\hat{c}(\xi\rightarrow\infty)=0$ and
$f^{\prime}(\xi\rightarrow\infty)=0$ is obtained with the method described by
Nachtsheim and Swigert [30]. Briefly, this method consists in writing and
solving differential equations on the shooting parameters to determine the
corrections on the starting parameters. We implemented this algorithm with the
Python language, the libraries numpy and scipy [31, 32]. Our implementation
has been successfully validated with the data available in [30] and our source
code is available in Supplementary Materials.

The resolution is performed for ${\rm Sc}=0.75$ and we represent the
dimensionless vapor concentration $\tilde{c}$ and the velocity field
${\bm{u}}$ in Fig. 4. The thickness of the boundary layer along the plate
grows as $x^{1/4}$ as indicated by the self-similar variable $\xi$ defined in
Eq. (13) and the flow is oriented mainly upward with a horizontal component
toward the plate. In addition, we obtain $\hat{c}_{0}^{\prime}\approx-0.51$
for our conditions. The value of this coefficient is used hereafter in Sec.
3.4 to estimate the evaporative flux.

###  3.4 Evaporative flux

As the concentration field surrounding the plate is now determined, we can
calculate the evaporative flux. The local flux is defined as

$j=-{\cal D}\left.\frac{\partial c}{\partial y}\right|_{y=0}.$ (17)

With the definition of the dimensionless concentration $\hat{c}$, we obtain,
on the surface of the plate of area $wh$, the local convective flux

$j_{\rm plate}=-{\cal D}(c_{s}-c_{\infty})\hat{c}_{0}^{\prime}\frac{{\rm
Gr}_{x}^{1/4}}{x}.$ (18)

Note that the numerics is only necessary to determine the prefactor
$\hat{c}_{0}^{\prime}$ of Eq. (18) and that the scaling is obtained
analytically.

The total evaporative flux due to convection is $Q_{\rm plate}=\int j_{\rm
plate}\,{\rm d}S$, which leads after integration to

$Q_{\rm plate}=2w{\cal
D}(c_{s}-c_{\infty})\left(-\frac{4}{3}\hat{c}_{0}^{\prime}\right){\rm
Gr}^{1/4},$ (19)

where ${\rm Gr}$ is defined by equation (2) for the characteristic length
scale ${\cal L}=h$.

In the next Section, we discuss the validity of this prediction that we
compare to our experimental results obtained by drying vertical hydrogels of
different sizes and aspect ratios.

##  4 Discussion

In Fig. 5, we present data on the time variation of the relative weight
$\Delta m$ for some of the hydrogels used in our experiments. To obtain the
total flux $Q$, we fit our measurements with a linear law $\Delta m(t)=-Qt$.
This procedure is repeated for hydrogels of different sizes. The linear fits
indicate that the evaporative flux remains constant over the duration of the
experiment. Thus, the variations of the polymer concentration in the hydrogel
is negligible enough to ensure a constant saturated vapor concentration. In a
previous work, we also checked that this saturated vapor concentration can be
assimilated to the one of pure water [19].

Figure 5: Raw data of the time evolution of the relative weight $\Delta m$ for
$w=80$ mm and for different hydrogel height $h$. Black solid lines are linear
fits of the different sets of data.

In Fig. 6(a), we show experimental measurements of the evaporative flux $Q$
for two hydrogel widths, $w=30$ mm and $w=80$ mm and heights ranging between
$15$ mm and $210$ mm. We plot these fluxes as a function of the Grashof number
${\rm Gr}$, which is varied over three orders of magnitude. As expected, the
evaporative flux is systematically larger for wider gels at a given Grashof
number, i.e. at a given height $h$. For both widths, the measurements show a
good agreement with the prediction in ${\rm Gr}^{1/4}$ obtained from Eq. (19),
which means that the power law on the Grashof number is well captured by the
model.

Figure 6: (a) Experimental evaporative flux $Q$ as a function of the Grashof
number ${\rm Gr}$ for two gel widths $w$. Solid lines are fits in ${\rm
Gr}^{1/4}$. (b) Comparison between the experimental fluxes $Q$ and the
theoretical prediction $Q_{\rm conv}$ from Eqs (19) and Eq (20). Open symbols
are for $w=30$ mm and closed symbols for $w=80$ mm. Blue points correspond to
$Q_{\rm conv}=Q_{\rm plate}$ and red points to $Q_{\rm conv}=Q_{\rm
plate}+Q_{\rm side}$. Each colored solid line is a linear fit on all data
points of the same color and the black solid line represents $Q=Q_{\rm conv}$.

In Fig. 6(b), we compare our measurements with the prediction given by Eq.
(19). By fitting all our measurements $Q$ to the prediction $Q_{\rm
conv}\simeq Q_{\rm plate}$, we obtain $Q\approx 1.31\,Q_{\rm conv}$, which is
particularly reasonable at this level of theoretical description and for the
precision of our experiments. Nevertheless, in this representation, we can
notice for the smallest hydrogel widths that $Q_{\rm conv}$ underestimate the
measured flux. We interpret this deviation by the absence of consideration of
edge effects in the theoretical description that become visible for our
smallest samples. Edge effects may arise from the two side edges and the
bottom edge. Also, tip effects at the corners introduce a divergence in the
local flux, which is not taken into account. The major difficulty in
describing all these effects is that they are not simply additional
corrections because edges and tips can have a mutual influence.

Despite these difficulties, we attempt to write some corrections to the
prediction given by Eq. (19). First, we consider that the local flux $j_{\rm
conv}$ obtained by considering the surfaces of area $wh$, also holds for the
two vertical surfaces of area $bh$. Thus, the integral is calculated over a
total area $2(w+b)h$ and the flux due to convection $Q_{\rm conv}\simeq Q_{\rm
plate}+Q_{\rm side}$ is

$Q_{\rm conv}\simeq 2(w+b){\cal
D}(c_{s}-c_{\infty})\left(-\frac{4}{3}\hat{c}_{0}^{\prime}\right){\rm
Gr}^{1/4}.$ (20)

In Fig. 6(b), we observe that this correction reduces the deviation for $w=30$
mm and provide an estimate closer to the experimental results, $Q\approx
1.24\,Q_{\rm conv}$.

A second correction that could be considered is related to the bottom edge. In
our previous study [21], we argued that the boundary layer approximation
breaks down at the leading edge of the plate [33] over a length scale ${\cal
L}^{\star}$. This correction must be also related to the evaporative flux over
the bottom side of area $bw$. Applying these small corrections lead to a
better agreement with the measurements but, presently, their validities are
particularly difficult to discuss. Thus, we prefer to do not overinterpret our
results as the main trend is well captured. Further developments, especially
from numerical simulations, could help to refine the model on these aspects.

##  5 Conclusion

Motivated by the question of the soap film bursting, we identified that one of
the key aspects is related to the evaporation of the soap film. As shown by
the recent literature, evaporation can induce a convective flow in the
atmosphere, which depends on the geometry and influences the evaporation
kinetics. Therefore, to understand the effect of evaporation of soap films, it
is crucial to develop such models.

In this paper, we followed derivations established for heat transfer to
calculate the evaporative flux on a vertical plate. To verify the validity of
this description, we performed model experiments with slabs of hydrogels of
various widths and heights. From the time evolution of the weight, we measured
the total evaporation rate that we compared to the theoretical prediction. We
show that our experiments are in good agreement with this model, following the
power law ${\rm Gr}^{1/4}$. Although that the model does not consider edge and
tip effects, it provides a reasonable estimation of the prefactor. We indicate
that corrections could lead to a better agreement, especially for small films
where edge effects are more significant. We suggest that a more complete
description including these effects would require numerical simulations.

Nevertheless, our results shows that the model presented in this paper
provides a good description of the evaporation of vertical rectangular films,
provided that the boundary layer remains laminar all along the surface. In the
near future, we would like to investigate the evaporation of soap films in
light with this theory to rationalize the mechanisms that relates the bursting
of soap films to their evaporation. We expect that different additional
mechanisms can have a contribution to the evaporation of soap films, which are
not present in the model system based on hydrogels. First, the composition of
the soap solution can lead to a vapor pressure different from water,
eventually with an effect of the increasing concentration of non-volatile
molecules. The evaporation of thin films may generate temperature gradients,
and therefore thermal Marangoni flows in the soap film, in addition to the
solutal Marangoni flows, the gravitational drainage, and the marginal
regeneration in free standing films. The film thinning due to evaporation and
liquid flow leads the appearance of the so-called black film where van der
Waals attraction forces balance the double-layer repulsion forces, and
therefore modifies the chemical potential of the liquid [34].

## Acknowledgments

We are particularly grateful to Emmanuelle Rio for the initial discussions
that motivated this study. We also thank Lorène Champougny, Jonas Miguet,
Christophe Poulard and Frédéric Restagno for discussions. F.B. thanks Mélanie
Decraene for her assistance. This study has been carried out with a funding
support by the ANR (ANR-11-BS04-0030-WAFPI project).

## References

  * [1] M. A. Rutgers, X. L. Wu, and W. B. Daniel.  Conducting fluid dynamics experiments with vertically falling soap films.  Review of Scientific Instruments, 72(7):3025–3037, 2001.
  * [2] P. Ballet and F. Graner.  Giant soap curtains for public presentations.  European Journal of Physics, 27(4):951, 2006.
  * [3] C. Cohen, B. Darbois-Texier, E. Reyssat, J.H. Snoeijer, D. Quéré, and C. Clanet.  On the shape of giant soap bubbles.  Proceedings of the National Academy of Sciences, page 201616904, 2017.
  * [4] X. Li, R. Shaw, and P. Stevenson.  Effect of humidity on dynamic foam stability.  International Journal of Mineral Processing, 94(1–2):14 – 19, 2010.
  * [5] V. V. Yaminsky, S. Ohnishi, E. A. Vogler, and R. G. Horn.  Stability of aqueous films between bubbles. part 2. effects of trace impurities and evaporation.  Langmuir, 26(11):8075–8080, 2010.  PMID: 20146432.
  * [6] X. Li, S. I. Karakashev, G. M. Evans, and P. Stevenson.  Effect of environmental humidity on static foam stability.  Langmuir, 28(9):4060–4068, 2012.
  * [7] V.K. La Mer, editor.  Retardation of Evaporation by Monolayers: Transport Processes.  Academic Press, 1962.
  * [8] D. Langevin.  Influence of interfacial rheology on foam and emulsion properties.  Advances in Colloid and Interface Science, 88(1–2):209 – 222, 2000.
  * [9] R. D. Deegan, O. Bakajin, T. F. Dupont, G. Huber, S. R. Nagel, and T. A. Witten.  Capillary flow as the cause of ring stains from dried liquid drops.  Nature, 389(6653):827–829, 1997.
  * [10] J.M. Stauber, S.K. Wilson, B.R. Duffy, and K. Sefiane.  On the lifetimes of evaporating droplets.  Journal of Fluid Mechanics, 744, 2014.
  * [11] N. Shahidzadeh-Bonn, S. Rafaï, A. Azouni, and D. Bonn.  Evaporating droplets.  Journal of Fluid Mechanics, 549:307–313, 2006.
  * [12] G. J. Dunn, S. K. Wilson, B. R. Duffy, S. David, and K. Sefiane.  The strong influence of substrate conductivity on droplet evaporation.  Journal of Fluid Mechanics, 623:329–351, 2009.
  * [13] B. M. Weon, J. H. Je, and C. Poulard.  Convection-enhanced water evaporation.  AIP Advances, 1(1):012102, 2011.
  * [14] P.L. Kelly-Zion, C.J. Pursell, S. Vaidya, and J. Batra.  Evaporation of sessile drops under combined diffusion and natural convection.  Colloids and Surfaces A, 381(1–3):31 – 36, 2011.
  * [15] P.L. Kelly-Zion, C.J. Pursell, N. Hasbamrer, B. Cardozo, K. Gaughan, and K. Nickels.  Vapor distribution above an evaporating sessile drop.  International Journal of Heat and Mass Transfer, 65:165 – 172, 2013\.
  * [16] F. Carle, B. Sobac, and D. Brutin.  Experimental evidence of the atmospheric convective transport contribution to sessile droplet evaporation.  Applied Physics Letters, 102(6):061603, 2013.
  * [17] S. Somasundaram, T. N. C. Anand, and S. Bakshi.  Evaporation-induced flow around a pendant droplet and its influence on evaporation.  Physics of Fluids, 27(11), 2015.
  * [18] F. Carle, S. Semenov, M. Medale, and D. Brutin.  Contribution of convective transport to evaporation of sessile droplets: Empirical model.  International Journal of Thermal Sciences, 101:35 – 47, 2016.
  * [19] F. Boulogne, F. Ingremeau, and H. A. Stone.  Coffee-stain growth dynamics on dry and wet surfaces.  Journal of Physics: Condensed Matter, 29(7):074001, 2017.
  * [20] S. Dehaeck, A. Rednikov, and P. Colinet.  Vapor-based interferometric measurement of local evaporation rate and interfacial temperature of evaporating droplets.  Langmuir, 30(8):2002–2008, 2014.
  * [21] B. Dollet and F. Boulogne.  Natural convection above circular disks of evaporating liquids.  Phys. Rev. Fluids, 2:053501, 2017.
  * [22] E. Dietrich, S. Wildeman, C.W. Visser, K. Hofhuis, S. Kooij, H.J.W. Zandvliet, and D. Lohse.  Role of natural convection in the dissolution of sessile droplets.  Journal of Fluid Mechanics, 794:45–67, 2016.
  * [23] G. Laghezza, E. Dietrich, J.M. Yeomans, R.A. Ledesma-Aguilar, S. Kooij, H. Zandvliet, and D. Lohse.  Collective and convective effects compete in patterns of dissolving surface droplets.  Soft Matter, 12:5787–5796, 2016.
  * [24] G. Sudre.  Tunable Adhesion of Hydrogels.  PhD thesis, Université Pierre et Marie Curie - Paris VI, 2011.
  * [25] P.T. Tsilingiris.  Thermophysical and transport properties of humid air at temperature range between 0 and 100 ∘C.  Energy Conversion and Management, 49(5):1098 – 1110, 2008.
  * [26] R.M. Tennent.  Science Data Book.  Oliver and Boyd, 1971.
  * [27] E. Schmidt and W. Beckmann.  Das Temperatur- und Geschwindigkeitsfeld vor einer wärme abgebenden senkrechten Platte bei natürlicher Konvektion.  Technische Mechanik und Thermodynamik, 1(11):391–406, 1930.
  * [28] S. Ostrach.  An analysis of laminar free-convection flow and heat transfer about a flat plate paralled to the direction of the generating body force.  Technical report, NASA, 1953.
  * [29] A. Bejan.  Heat Transfer.  John Wiley & Sons, 1993.
  * [30] P. R. Nachtsheim and P. Swigert.  Satisfaction of asymptotic boundary conditions in numerical solution of systems of nonlinear equations of boundary-layer type.  Technical report, NASA, 1965.
  * [31] S. van der Walt, S. C. Colbert, and G. Varoquaux.  The numpy array: A structure for efficient numerical computation.  Computing in Science & Engineering, 13(2):22–30, 2011.
  * [32] E. Jones, T. Oliphant, P. Peterson, et al.  SciPy: Open source scientific tools for Python, 2001–.
  * [33] L. G. Leal.  Advanced Transport Phenomena.  Cambridge University Press, 2007.
  * [34] O. Bélorgey and J. J. Benattar.  Structural properties of soap black films investigated by x-ray reflectivity.  Phys. Rev. Lett., 66:313–316, 1991.


# Unified Vision-Language Representation Modeling for E-Commerce Same-style
Products Retrieval

Ben Chen, Linbo Jin, Xinxin Wang, Dehong Gao*, Wen Jiang, Wei Ning  Alibaba
Group699 Wang Shang RoadHangzhouZhejiangChina43017-6221 chenben.cb, yuyi.jlb,
rooney.wxx, dehong.gdh, wen.jiangw<EMAIL_ADDRESS>
(2022)

###### Abstract.

Same-style products retrieval plays an important role in e-commerce platforms,
aiming to identify the same products which may have different text
descriptions or images. It can be used for similar products retrieval from
different suppliers or duplicate products detection of one supplier. Common
methods use the image as the detected object, but they only consider the
visual features and overlook the attribute information contained in the
textual descriptions, and perform weakly for products in image less important
industries like machinery, hardware tools and electronic component, even if an
additional text matching module is added. In this paper, we propose a unified
vision-language modeling method for e-commerce same-style products retrieval,
which is designed to represent one product with its textual descriptions and
visual contents. It contains one sampling skill to collect positive pairs from
user click logs with category and relevance constrained, and a novel
contrastive loss unit to model the image, text, and image+text representations
into one joint embedding space. It is capable of cross-modal product-to-
product retrieval, as well as style transfer and user-interactive search.
Offline evaluations on annotated data demonstrate its superior retrieval
performance, and online testings show it can attract more clicks and
conversions. Moreover, this model has already been deployed online for similar
products retrieval in alibaba.com, the largest B2B e-commerce platform in the
world.

Same-style products retrieval, Vision-language representation, Contrastive
loss, User interactive search

††copyright: acmcopyright††journalyear: 2022††doi:
XXXXXXX.XXXXXXX††conference: The Web Conference 2023; April 01– May 04, 2022;
Austin, USA††ccs: Information systems Multimedia and multimodal retrieval

##  1\. Introduction

Same-Style Products Retrieval (SSPR) is an important task in e-commerce search
system, with the aim to identify products which are the same but released by
suppliers with different images and titles. It is an essential requirement for
the similar products retrieval and duplicate products detection. Specifically,
buyers want to view more of the same products from different suppliers, so
that they can find the favorite with lower price and better quality, and we
should provide this access when they are searching. While for suppliers, they
tend to deliver same products for hacking more traffic, so an efficient
duplicate products detection skill is in need to suppress this behavior for
better users’ experience and platform health.

Figure 1. The overview of the similar products retrieval platform in
alibaba.com, where ”From more suppliers” means the same-style products from
different suppliers.

The overview of similar products retrieval platform, where products are
recalled based on the learned embedding provided by proposed method.

Existing methods for SSPR mainly consider the visual attributes such as
product shape, style and color. They usually adopt an image-to-image matching
model to score the images similarity of all candidate products, and then
select the ones whose score is greater than a certain threshold as the same
products. But these image-based methods overlook the textual attributes
(specific mode, material, brand e.t.c.) contained in the textual descriptions,
and perform weakly for products in image less important industries, e.g.
machinery, hardware tools and electronic component. Some supplementary
measures will adopt an additional text-to-text matching model to further
filter products with irrelevant attributes. However, suppliers may add some
unrelated keywords into the title for more exposures, and these keywords will
mislead the model to make wrong judgments. Therefore, to accurately
distinguish the same products, we should equally consider the image and
textual descriptions, and utilize both visual and textual features to model
the product.

In recent years, more and more vision-language pre-training (VLP) models (Su
et al., 2019; Lu et al., 2019; Li et al., 2020b, a; Gao et al., 2020; Li et
al., 2022) have emerged to explore how to align the representations of
different modals. They try to map the textual and visual features into one
joint embedding space based on the self-attention mechanism (Vaswani et al.,
2017; Devlin et al., 2019), and achieve impressive performance in many cross-
modal tasks like text-image retrieval, image captioning, visual question
answering. However, These VLP models mainly focus on text and image matching
but pay less attention to the image+text to image+text matching for SSPR, of
which the key challenge is the alignment of image, text and image+text
presentations to reduces interference of irrelevant attributes.

To accurately represent one product with its textual description and visual
content, this paper propose an unified vision-language representation method.
It consists of three types of contrastive loss, with the aim to reduce the
representation discrepancy of the product image, text, and image+text from
different perspectives. The integrated objective function unit combining these
losses can effectively model these three representations into a joint
embedding space, and make the key features involved in image and text dominate
the generation of embeddings. This method is applied in the fine-tune stage,
and can easily adapt to various VLP models with less burden. In addition, In
order to collect enough data to train an effective model, we also propose an
intuitive training data sampling technique, which can gather high reliable
positive pairs from user click logs by constraining categories and relevances.

Figure 2. User-interactive search cases with proposed model. A mug with owl
design + ”3D frog” words can search the frog shape mugs, and a red dress +
”white” get the white dresses.

A red dress + ”white” word can get corresponding white dresses, shows the
potential for use-interactive search.

We execute extensive offline evaluations on annotated same-style data for
product (image, text, image+text) to product (image, text, image + text)
retrieval, and the significant performance boosts demonstrate the proposed
method’s effectiveness for cross-modal representation. Online tests also prove
that it can improve the same-style products coverage rate, and attract more
clicks and conversions. This method has already been deployed online for
similar products retrieval in alibaba.com, the largest B2B e-commerce platform
in the world, as seen in Figure 1. Moreover, it also shows the potential for
applications like style transfer and user-interactive search (shown in Figure
2).

##  2\. Method

###  2.1. Query-Item Click Graph

For training an effective model to retrieval the same-style products, one
primary shortcoming is the lack of high-reliable training data. Because the
manual annotation samples are accurate but expensive, while online available
data is noisy and can not cover all categories. Here we collect reliable
samples from user query-item click log, and impose category and relevance
restrictions to ensure the accuracy of data. This sampling skill is based on a
basic cognition that the query reflect user’s intents and their click
behaviors indicates their attention to different products. Two samples with
deeper click level in the same query should have higher similarity.

In alibaba.com, the user’s click behavior can be divided into 5 levels: page-
click ($c$) $\leq$ add-to-cart ($a$) $\leq$ contact-supplier ($s$) $\leq$
order ($o$) $\leq$ pay ($p$). We construct the query-item click graph with
query nodes containing the search texts and the item nodes composed of ¡image,
title, keywords, brand¿. Then we use the sum of different click level scores
as weight to build a co-clicking edge between query $q_{i}$ and item node
$c_{i}$ pairs. Click level score is represented as a coefficient
$\boldsymbol{\lambda}$ multiply the click number and final weight is:

(1)
$Weight=\lambda_{1}cnt_{c}+\lambda_{2}cnt_{a}+\lambda_{3}cnt_{s}+\lambda_{4}cnt_{o}+\lambda_{5}cnt_{p}.$

For deeper clicks, this coefficient should be set larger to alleviate the
effect of ¡$q_{i}$, $c_{i}$¿ pairs which users click by mistake.

###  2.2. Constrained Graph Sampling

After constructing the click graph, we use the weighted sampling strategy to
generate training samples. Specifically, we collect no more than 4 items of
one query according to the weight from large to small, and build positive
sample in pairs. In addition, we set three constraints to ensure the sampling
accuracy:

#### Query contains at least two words and one core keyword

Some users have no clear search intents and will enter query with ambiguous
semantics, e.g. dress, sport shoes. These queries cover a relatively broad
category and the items clicked by users are also scattered. So we select the
query with at least two words, and one core keyword that clearly represents
the user’s needs.

#### Two items must be of same sub-category

E-commerce platform has a hierarchical and fine-grained category structure,
and one category contains dozens of sub-categories. For example, ”woman dress”
contains 20 sub-categories like ”evening dress”, ”career dress”, ”casual
dress”, and ”casual dress”. A query ”red long dress with v-neck” will
retrieval items cross these categories. To collect same-style ones, we should
keep two items of same sub-category.

#### Image-text pairs should have high similarity

Deep click levels limitation can not entirely guarantee that two items are of
same-style. So we compute image similarity with embedding outputted from
Resnet50 (He et al., 2016) and text similarity with embedding outputted from
Bert (Devlin et al., 2019) for each pairs, and eliminate those pairs with
image similarity or text similarity values lower than 0.7.

After the constrained sampling, we select those pairs that contain at least
one core keyword in common as the final training samples. This operation is
also designed to reduce the sampling error.

###  2.3. Contrastive Loss Unit

The basic architecture of vision-language pre-training (VLP) models is usually
composed of a visual embedding module, a textual embedding module and a fusion
encoder (Yu et al., 2021, 2022). Of these architectures the image is usually
encoded with an off-the-shelf ResNet (He et al., 2016), or Faster-RCNN (Ren et
al., 2015), or Visual Transformer (Dosovitskiy et al., 2020) model. While
textual embeddings are generated by dividing the text description into a
sequence of word tokens and then fed into one transformer model (Devlin et
al., 2019; Conneau et al., 2019; Liu et al., 2019). Then an encoder will fuse
both embeddings to create a contextual interacted multimodal representation.

As for the training objectives, common VLP models use image+text contrastive
loss (ITC) to align the embedding space of text and image, while adopt binary
classification image+text matching loss (ITM) for learning a multimodal
representation to predict whether image and text is in pair, as well as some
additional objectives like language modeling loss to enhance the embedding
learning (Gao et al., 2020; Radford et al., 2021; Li et al., 2022, 2021).
However, they pay less attention to the feature alignment at the item
(image+text) level.

Here we further consider how to generate one multimodal representation used
for image+text to image+text matching in same-style products retrieval, and
proposed a novel contrastive loss unit acting in the fine-tune stage. It
contains three types of objectives: product to product matching losses (PPM),
product self-distinctiveness losses (PDC) and product locality consistency
losses (PLC). They are designed to learn a cross-modal aligned representation
from different perspectives, and all contribute to model image, text, and
image+text into one joint embedding space.

Figure 3. The overview of proposed contrastive loss unit. We mainly show the
PPM construction method for brevity.

The proposed contrastive loss unit acts in the fine-tune stage, which models
the product text, image, and image+text representation in one joint embedding
space.

Given an item-item pair of same-style, we take the left item as trigger item
$t_{i}$, and the right item as recall item $r_{i}$, where the subscript $i$
indicates the $i$-th pair in dataset. The multimodal image+text embeddings
$t^{m}_{i}$, $r^{m}_{i}$ are the average of all embeddings of the last layer.
The single image visual embeddings $t^{v}_{i}$, $r^{v}_{i}$ and textual
embeddings $t^{t}_{i}$, $r^{t}_{i}$ are the average of corresponding
embeddings of the last layer, with the masked text description and image are
replaced with the all-zero sequences to simulate single modal inputs. Finally,
all these embeddings are normalized.

#### Product to product matching losses (PPM)

This objective aims to align the representations of trigger item and recall
item into one joint embedding space. We assume that if two items are of same-
style, the multimodal image+text embeddings of one should have the high
similarity to the other. Furthermore, the image, text description (text,
keywords, brand, e.t.c) of the trigger item should also be aligned to the
multimodal embeddings of the recall item, making the key features involved in
image and text dominate the generation of embeddings.

Specifically, during fine-tuning stage, each items ¡$t_{i}$, $r_{i}$¿ of same-
style are taken as positive sample pair, while the trigger item $t_{i}$ and
recall item $r_{j}$ from other trigger item in the same batch combine the
negative sample pair ¡$t_{i}$, $r_{j}$¿. We compute the similarity score
between three embeddings of trigger item and image+text embeddings of the
recall item as the follows:

(2) $s^{m\leftrightarrow
m}_{ij}=\textbf{$t^{m}_{i}$}\cdot\textbf{$r^{m}_{j}$},\quad
s^{v\leftrightarrow
m}_{ij}=\textbf{$t^{v}_{i}$}\cdot\textbf{$r^{m}_{j}$},\quad
s^{t\leftrightarrow m}_{ij}=\textbf{$t^{t}_{i}$}\cdot\textbf{$r^{m}_{j}$}.$

Then we use hinge loss to guide item pairs of same-style to have greater
similarity values compared with those of different styles:

(3) $\displaystyle
L_{PPM}=\frac{1}{N^{2}}\underset{i}{\sum}\underset{j}{\sum}$
$\displaystyle\frac{1}{3}[max(0,\alpha_{1}(1-y_{ij})+s^{m\leftrightarrow
m}_{ij}-s^{m\leftrightarrow m}_{ii})$
$\displaystyle+max(0,\alpha_{1}(1-y_{ij})+s^{v\leftrightarrow
m}_{ij}-s^{v\leftrightarrow m}_{ii})$
$\displaystyle+max(0,\alpha_{1}(1-y_{ij})+s^{t\leftrightarrow
m}_{ij}-s^{t\leftrightarrow m}_{ii})],$

where $N$ is the batch size, $\alpha_{1}$ is the margin value. It lets cross-
modal embeddings of positive pairs be closer than those of negative pairs.

#### Product self-distinctiveness losses (PDC)

PDC is designed to ensure that three modal embeddings of one item should have
the smaller distance than the multimodal embeddings between trigger item and
recall item. In other words, the similarity values between $t^{v}_{i}$,
$t^{t}_{i}$ and $t^{m}_{i}$ should larger than those between $t^{m}_{i}$ and
$r^{m}_{j}$.

(4) $\displaystyle
L_{PDC}=\frac{1}{N^{2}}\underset{i}{\sum}\underset{j}{\sum}$
$\displaystyle\frac{1}{2}[max(0,\alpha_{2}+s^{m\leftrightarrow
m}_{ij}-s^{v\leftrightarrow m}_{ii})$
$\displaystyle+max(0,\alpha_{2}+s^{m\leftrightarrow
m}_{ij}-s^{t\leftrightarrow m}_{ii})],$

where $\alpha_{2}$ is the margin value. This objective is similar to PPM, but
it is one effective way to reduce representation discrepancy of three modals
from another perspective.

#### Product locality consistency losses (PLC)

PLC tries to align the embedding of image, text, and image+text to reduces
interference of irrelevant attributes. That is, three normalized embeddings of
one trigger item should be as consistent as possible. It can be described as
the mean square error (MSE) of the similarity values $s^{m\leftrightarrow
m}_{ij}$, $s^{v\leftrightarrow m}_{ij}$, and $s^{t\leftrightarrow m}_{ij}$
should smaller than a certain value:

(5) $\displaystyle
L_{PLC}=\frac{1}{N^{2}}\underset{i}{\sum}\underset{j}{\sum}$
$\displaystyle\frac{1}{3}[max(0,-\alpha_{3}+(s^{v\leftrightarrow
m}_{ij}-s^{m\leftrightarrow m}_{ij})^{2})$
$\displaystyle+max(0,-\alpha_{3}+(s^{t\leftrightarrow
m}_{ij}-s^{m\leftrightarrow m}_{ij})^{2})$
$\displaystyle+max(0,-\alpha_{3}+(s^{v\leftrightarrow
m}_{ij}-s^{t\leftrightarrow m}_{ij})^{2})],$

where $\alpha_{3}$ is the margin value. This computation is relatively large,
and we can compute only top 10 pairs for each trigger item, where the rank
list is arranged according to $s^{t\leftrightarrow m}_{ij}$ from large to
small.

So the total loss is formulated as follows:

(6) $\displaystyle L_{total}=\frac{1}{3}(L_{PPM}+L_{PDC}+L_{PLC}),$

As a side note, the pairs in the same batch should keep the same sub-category
but belong to different same-style. This operation drives the model to focus
on hard samples, and learn the fine-grained difference between styles.

##  3\. Experiments

In this section, we conduct comprehensive evaluations on annotated same-style
data offline and A/B test online to verify the feasibility of proposed method.

Dataset. We collect a large scale of training samples from user click log by
the sampling skills detailed above, with the coefficient
$\boldsymbol{\lambda}$ set as (1,2,2,5,5). It contains 4 million trigger-
recall item pairs of 20 industries and 4000 sub-categories. Since there is no
public evaluation data, here we collect 5196 manually annotated pairs covering
15 industries and 973 sub-categories for the offline testing.

Implementation details. We use tiny FashionBert (L2-H128-A6) (Gao et al.,
2020) as the base pre-trained model. It is an economical and efficient model
and have been online applied in the item recall stage of many search
platforms. The image is split into $4\times 4$ patches, and passes through
ResNet-50 to generate 16 tokens. The maximum text sequence length is set to 50
with the special [CLS] on the head, so the total token length is 66. Learning
rate is set as 3e-4, the batch size is 256 and the margin values
${\alpha_{1,2,3}}$ is set as (0.3, 0.2, $0.05^{2}$). All the evaluated models
are trained within 5 epochs and we adopt the early-stopping strategy.

###  3.1. Offline evaluation.

For the fair comparison with same model structure and parameters, we fine-tune
two baseline models with only image (base v-v) and text (base t-t) as the
inputs while the corresponding objectives are set to be image-image and text-
text hinge loss, then use the outputs (visual embeddings, textual embeddings)
of trigger and recall items to compute the similarities. We abbreviate the
proposed model as eSSPR for short. eSSPR v-v / t-t / m-m denote that we fine-
tune pre-trained model with image + text as the inputs and $L_{total}$ as the
loss function, and we compute the similarities with only image, only text and
image+text as the inputs to generate corresponding embeddings. For all
evaluations we adopt Mean Reciprocal Ranking (MRR) and recall@K as the
metrics, which are widely used in search and recommendation systems. All data
presented are the average values of the metrics for all testing pairs.

Table 1. Comparison with different methods on manual annotated data. R@n is
the abbreviation of Recall@n

.  Method MRR R@1 R@5 R@10 R@20 base t-t 0.5454 0.4744 0.6407 0.6938 0.7404
eSSPR t-t 0.6428 0.5597 0.7485 0.8174 0.8761 base v-v 0.8620 0.8389 0.8905
0.9121 0.9284 eSSPR v-v 0.9027 0.8784 0.9355 0.9511 0.9675 eSSPR m-m 0.9197
0.8870 0.9619 0.9761 0.9879

Table 2. Ablation experiment results of different losses with same inputs. R@n
is the abbreviation of Recall@n

.  Loss MRR R@1 R@5 R@10 R@20 $L_{base}$(Gao et al., 2020) 0.8915 0.8582
0.9342 0.9542 0.9654 $L_{PPM}$ 0.8935 0.8543 0.9463 0.9646 0.9779
$L_{PPM+PDC}$ 0.9065 0.8730 0.9524 0.9695 0.9801 $L_{PPM+PLC}$ 0.9131 0.8784
0.9596 0.9761 0.9852 $L_{total}$ 0.9197 0.8870 0.9619 0.9761 0.9879

The results are shown in Table 1, we can see that eSSPR m-m outperforms two
baseline method with large improvements, demonstrating the effectiveness of
the proposed unified vision-language representation method. Furthermore, even
if we only use unimodal features (text or image), the model eSSPR t-t (v-v)
achieves a higher performance than base t-t (v-v), which proves that the
contrastive loss unit contributes to make the key features involved in image
and text dominate the generation of embeddings.

In order to verify the feasibility of proposed loss unit, we provide
additional ablation experiments, which are exhibited in Table 2. The
$L_{base}$ is the hinge loss of image+text embeddings for trigger items
($t^{m}_{i}$) and recall items ($r^{m}_{i}$). We find that $L_{PPM}$ have the
better performance than $L_{base}$. Combined with the results of unimodal
models in Table 1, we believe that $L_{PPM}$ can make the representation of
image, text and image+text mutually enhance each other. $L_{PPM+PLC}$ and
$L_{PPM+PDC}$ improve MRR and recall rate significantly, and $L_{total}$
achieves the best results among them. These comparisons verify the contrastive
loss unit is capable of boosting the performance substantially.

###  3.2. Online Experiments.

We lauched the proposed model in our similar products retrieval platform,
which is used in the search, homepage recommender and you-shopping-history
recommender modules in alibaba.com (shown in Figure 1). We compare the
experimental performance of SCR (same-style products coverage rate), CTR
(click through rate) and CVR (average conversion rate) before and after the
launch. As shown in Table 3, the SCR, CTR and CVR metrics increase $+64.9\%$,
$+2.51\%$, $+2.31\%$ respectively. The results verified that the proposed
unified vision-language representation method can attract more clicks and
conversions for increasing the industry revenue.

Table 3. Online results for A/B testing

.  SCR CTR CVR +64.9% +2.51% +2.31%

More cross-modal retrieval results, the annotated testing data cases,
corresponding codes and the application demo of proposed methods in
alibaba.com are attached in the appendix. All of them will be released to
public through the official repository, as well as the adaptive testings for
modern dual-stream models like CLIP (Radford et al., 2021) and Align (Li et
al., 2021).

##  4\. Conclusion

Same-style products retrieval (SSPR) provides one channel for users to compare
more same products with lower price or higher quality. To identify the same
products which may have different text descriptions and images, we proposed a
unified vision-language representation method for same-style products
retrieval. It contains a reliable training samples construction technique, and
a novel contrastive loss unit acts in the fine-tune stage to model the product
image, text, and image+text representation into one joint embedding space.
Extensive experiments on offline annotated item pairs demonstrates its
superior performance for the same products retrieval. Online testings have
verified that it can attract more clicks and conversions in the similar
product recommendation in alibaba.com. In the further, we will focus on its
extended applications likes style transfer and user-interactive search, and
also study how to make the more fine-grained alignment of multimodal features
for e-commerce platform.

## References

  * (1)
  * Conneau et al. (2019) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Unsupervised cross-lingual representation learning at scale.  _arXiv preprint arXiv:1911.02116_ (2019).
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186.
  * Dosovitskiy et al. (2020) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020\.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ (2020).
  * Gao et al. (2020) Dehong Gao, Linbo Jin, Ben Chen, Minghui Qiu, Peng Li, Yi Wei, Yi Hu, and Hao Wang. 2020.  Fashionbert: Text and image matching with adaptive loss for cross-modal retrieval. In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_. 2251–2260.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016\.  Deep residual learning for image recognition. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 770–778.
  * Li et al. (2020a) Gen Li, Nan Duan, Yuejian Fang, Ming Gong, and Daxin Jiang. 2020a.  Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 34. 11336–11344.
  * Li et al. (2022) Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022\.  Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation.  _arXiv preprint arXiv:2201.12086_ (2022).
  * Li et al. (2021) Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021.  Align before fuse: Vision and language representation learning with momentum distillation.  _Advances in Neural Information Processing Systems_ (2021).
  * Li et al. (2020b) Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, and Jianfeng Gao. 2020b.  Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks. In _Computer Vision – ECCV 2020_. Springer International Publishing, Cham, 121–137.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ (2019).
  * Lu et al. (2019) Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019\.  ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks. In _Advances in Neural Information Processing Systems_ , Vol. 32. Curran Associates, Inc.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021\.  Learning transferable visual models from natural language supervision. In _International Conference on Machine Learning_. PMLR, 8748–8763.
  * Ren et al. (2015) Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015\.  Faster r-cnn: Towards real-time object detection with region proposal networks.  _Advances in neural information processing systems_ 28 (2015).
  * Su et al. (2019) Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. 2019\.  Vl-bert: Pre-training of generic visual-linguistic representations.  _arXiv preprint arXiv:1908.08530_ (2019).
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017\.  Attention is All you Need. In _Advances in Neural Information Processing Systems_ , Vol. 30. Curran Associates, Inc.  https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
  * Yu et al. (2022) Licheng Yu, Jun Chen, Animesh Sinha, Mengjiao MJ Wang, Hugo Chen, Tamara L Berg, and Ning Zhang. 2022.  CommerceMM: Large-Scale Commerce MultiModal Representation Learning with Omni Retrieval.  _arXiv preprint arXiv:2202.07247_ (2022).
  * Yu et al. (2021) Tan Yu, Xiaokang Li, Jianwen Xie, Ruiyang Yin, Qing Xu, and Ping Li. 2021\.  MixBERT for image-ad relevance scoring in advertising. In _Proceedings of the 30th ACM International Conference on Information & Knowledge Management_. 3597–3602.