# Plasma sheath tailoring by a magnetic field for three-dimensional plasma
etching

Elia Jüngling  Chair Experimental Physics II, Ruhr University Bochum, Bochum,
Germany  Sebastian Wilczek  Chair of Applied Electrodynamics and Plasma
Technology, Ruhr University Bochum, Bochum, Germany  Thomas Mussenbrock  Chair
of Applied Electrodynamics and Plasma Technology, Ruhr University Bochum,
Bochum, Germany  Marc Böke  Chair Experimental Physics II, Ruhr University
Bochum, Bochum, Germany  Achim von Keudell  Chair Experimental Physics II,
Ruhr University Bochum, Bochum, Germany

###### Abstract

Three-dimensional (3D) etching of materials by plasmas is an ultimate
challenge in microstructuring applications. A method is proposed to reach a
controllable 3D structure by using masks in front of the surface in a plasma
etch reactor in combination with local magnetic fields to steer the incident
ions in the plasma sheath region towards the surface to reach 3D
directionality during etching and deposition. This effect can be controlled by
modifying the magnetic field and/or plasma properties to adjust the
relationship between sheath thickness and mask feature size. Since the guiding
length scale is the plasma sheath thickness, which for typical plasma
densities is at least 10s of microns or larger, controlled directional etching
and deposition target the field of microstructuring, e.g. of solids for
sensors, optics, or microfluidics. In this proof-of-concept study, it is shown
that $\vec{E}\times\vec{B}$ drifts tailor the local sheath expansion, thereby
controlling the plasma density distribution and the transport when the plasma
penetrates the mask during an RF cycle. This modified local plasma creates a
3D etch profile. This is shown experimentally as well as using 2d3v Particle-
In-Cell/Monte Carlo collisions simulation.

††preprint: XXX

Plasma processing of materials is very advanced in microelectronics to create
2D structures with very high precision by anisotropic etching in silicon or 2D
materialsSekine (2002); Radjenović and Radmilović-Radjenović (2014); Krüger
_et al._ (2019). This is complemented by various deposition techniques ranging
from standard plasma-enhanced chemical vapor deposition (CVD)Orfert and
Richter (1999) to atomic layer deposition (ALD) or plasma-ALDLim and Yun
(2004) to generate various oxide or metal layersArts  _et al._ (2022). The
directed ion flux ensures the directionality of the etch process in plasma-
based microstructuring due to the process of chemical sputtering, where the
impact of ions and reactive species simultaneously ensures a high etch rate.
This process is e.g. exploited in trench etching, where the bottom of a trench
is etched very efficiently compared to the side walls. This effect is also
amplified by side wall passivation due to the polymerization of the reactive
etching gas. This allows for creating very small structures down to 7 nm for a
gate oxide width or trenches with extremely high aspect ratios in the range of
50 to 100 (depth vs. width). Plasma processing is also well-advanced to create
microstructures with feature sizes on the micrometer scale to produce micro-
electromechanical systems (MEMS) devices, preferably in silicon for sensors or
microfluidic devices. 3D printing technologies are employed at even larger
scales using mainly polymethyl methacrylate (PMMA) or polyether ether ketone
(PEEK) as material. However, there is demand for advanced 3D structuring of
materials for optical applications such as microlensesBrunner  _et al._
(2004), gratingsAmako, Sawaki, and Fujii (2009) or for illumination Miller
_et al._ (1993). All these applications require 3D shape control with a small
surface roughness in the range of $\lambda$/100, with $\lambda$ being the
wavelength in the particular application. Another field for 3D structuring is
microfluidic fuel cells, where the turnover of an electrochemical reaction is
enhanced by replacing a membrane by a diffusion gradient in a microchannel and
designing specific 3D electrode surfacesZhou  _et al._ (2021).

Realizing complex structures requires accurate 3D control of plasmas since the
sidewalls and the bottom and top of the channel must be processed differently.
In the past, several attempts have been explored to reach 3D plasma processing
capabilities: (i) Shadowing effects in glancing angle deposition \- the
shadowing of the growing structures during glancing angle deposition of metals
may lead to micro-pillars on top of a surfaceHawkeye and Brett (2007); (ii)
Charging of 3D structures in the plasma sheath to steer the ions \- Chang et
al. Chang (2020) used structures above the wafer level that charge up and
distort the plasma sheath in front of the surface. This steers the ions, and a
tilted angle etching and manufacturing of MEMS structures is realized. Yoon et
al. Yoon  _et al._ (2022) used a similar concept to tilt the incident ions;
(iii) Alternating chemistry during trench etching: \- Ni et al. Ni  _et al._
(2020) reached 3D patterning by alternating the etching of deep holes using
the anisotropic etch characteristics of a fluorocarbon etch process, with a
subsequent isotropic chemical etch step. This sequence can be repeated several
times to reach stacked 3D structures; (iv) Affecting trench charging by
magnetic fields \- Schaepkens and Oehrlein Schaepkens and Oehrlein (1998) used
a weak magnetic field parallel to the wafer surface, which affected the
electrons to charge one side of the sidewall. This created an electric field
component, which diverted the incident ions in the very same direction inside
a trench during plasma etching; (v) Micromachining for optics \- macroscopic
3D structures can be created by various micromachining techniques such as
electrical discharge machining or laser ablation Pawar, Ballav, and Kumar
(2017); Hof and Abou Ziki (2017). These are serial processes operating more on
the macroscopic scale; (vi) Use of photosensitive materials \- the 3D aspect
of an etch process can be controlled by using photosensitive materials, where
an external illumination changes the material properties so that the etch
rates develop locally differentlyWeigel  _et al._ (2021). This requires,
however, the choice of particular photosensitive materials.

In this paper, we introduce the following idea for a flexible 3D pattering
process: a magnetic field $\vec{B}$ is applied parallel to the surface,
leading to $\vec{E}\times\vec{B}$ drifts in combination with the electric
fields $\vec{E}$ in the plasma sheaths. A mask is placed in front of the
substrate to design the $\vec{E}$ fields and thus allow for a 3D control of
the plasma density in front of a substrate by tailoring the plasma sheath with
the $\vec{E}\times\vec{B}$ drifts. This local variation of the plasma density,
in turn, leads to a well-defined 3D etching pattern.

The etching experiments are performed in an inductively coupled plasma
(ICP)Chabert, Tsankov, and Czarnetzki (2021); Chabert and Braithwaite (2011)
etching setup (see Fig. 1a) using an argon -CF4 mixture as a processing gas
for silicon etching and an argon-oxygen mixture for hydrocarbon film etching,
both in a $5:1$ ratio. A sample assembly is placed on a substrate holder
opposite the ICP dielectric window. At the substrate holder, an external RF
bias voltage can be applied. The distance between the RF substrate holder and
the ICP dielectric window is 13.5 cm.

The sample assembly consists of a silicon wafer (or a silicon wafer coated
with a hydrocarbon coating) mounted below a metal grid (thickness 1 mm) with
four linear slits and two adjacent permanent magnets that generate a magnetic
field parallel to the grid openings in the order of 70 mT. The width of the
slit is 1mm and the distance in between two slits is 2 mm. The distance
between the grid and the wafer is 0.5 mm. A schematic illustration is shown in
Fig. 1b.

For a typical experiment, the applied ICP power is set to 500 W for
hydrocarbon film etching and 600 W for silicon etching, respectively. All
experiments are performed at a pressure of 2 Pa. The RF bias power is
automatically adjusted to reach a given DC self-bias. Here, a DC self-bias of
-100 V and -150 V is used for hydrocarbon film and silicon etching,
respectively. Both the ICP power and the RF bias power are operated at a
frequency of 13.56 MHz.

After the sample has been processed for a given time, the assembly is removed
from the setup and dismounted to analyze the etch profiles on the wafer below
the slits.

Figure 1: Schematic of the plasma etching setup consisting of an ICP plasma at
the bottom and RF-biased substrate holder at the top (a). 3D sketch (left) and
cross-section (right) of the sample assembly consisting of a wafer with a mask
in front and attached magnets (b). Schematic of the simulation domain,
including the magnetic field configuration in the simulation (c). (a) and (b)
are not to scale.

The geometry of the etch profiles is analyzed using profilometry (Veeco 6M,
horizontal resolution: 5 µm, vertical resolution: 2 nm). Measurements are
taken exclusively at the center of the two middle slits because the magnetic
field of the sample assembly is uniform only in that location. Ex-situ XPS
measurements were performed to determine the stoichiometry of the samples. The
measurements were realized with a PHI5000 x-ray photoelectron spectrometer
using Al K radiation at 1,486.6 eV. A beam diameter of 200 µm and a pass
energy of $187.85\text{\,}\mathrm{eV}$ were used. The step width was
$1\text{\,}\mathrm{eV}$, resulting in a resolution of
$2.82\text{\,}\mathrm{eV}$.

To analyze the manipulation of the plasma transport in front of the surface by
the mask and the magnetic field, a Particle-In-Cell/Monte Carlo collisions
(PIC/MCC) simulation is used. The 2d3v PIC/MCC code had been designed
following the concept of the 1D code eduPICDonkó  _et al._ (2021) and has been
validated against benchmarks in the literatureDonkó  _et al._ (2021); Turner
_et al._ (2013). An excellent agreement has been found. This 2d3v PIC/MCC code
simulates a capacitively coupled plasma (CCP)Chabert and Braithwaite (2011);
Lieberman and Lichtenberg (1994); Wilczek  _et al._ (2020) in pure argon to
quantify the dynamic asymmetry in the incident ion flux at the wafer surface.
The full Lorentz force law $\vec{F}=±q(\vec{E}+\vec{v}\times\vec{B})$ is
considered for both electrons and ions, although the ions are predominantly
non-magnetized due to their large massTrieschmann  _et al._ (2013). We use an
explicit push scheme based on Boris’s approachBoris (1970). While the
simulation considers multiple collision processes in argon using cross
sections from literaturePhelps and Petrovic (1999), any complex plasma
chemistry regarding reactive processing gases has been neglected. Furthermore,
the code includes glancing angle reflection of ions at the inside walls of the
mask in front of the wafer. For this, a simple model has been developed using
reflection probabilities calculated with SRIMZiegler, Ziegler, and Biersack
(2010).

The simulation domain is defined as a mask in front of a wafer at a distance
of 0.5 mm with a single 1 mm slit; the thickness of the mask is 1 mm. The
whole 2D simulation area is 4 mm x 15 mm. The magnetic field is adjusted
parallel to the wafer surface pointing out of the plane. A schematic
illustration of the geometry is shown in Fig.1c. The driving sinusoidal
voltage waveform with an amplitude of 100 V and a frequency of 13.56 MHz is
applied to the lower boundary and the mask elements, while the upper electrode
is grounded. Both surfaces and the mask structure, except for the inside walls
of the mask where the reflection model is used, fully absorb incident
particles. Secondary electron emission at all surfaces is neglected. Periodic
boundary conditions are used at the sides of the simulation domain. The
computational grid is Cartesian and equidistant with a cell width of 0.1 mm
and a time step of 35 ps is used. The initial particle configuration consists
of $10^{5}$ electrons and ions with an initial temperature of 3 eV and 300 K,
respectively, and a particle weight of $8\cdot 10^{6}$. Consequently, the
conditions for stability and accuracy are fulfilled with these parameters
Donkó  _et al._ (2021); Turner  _et al._ (2013). The background gas is set to
a pressure of 2 Pa. The time- and space-resolved results are acquired after 50
µs of simulation time when convergence is reached. The ion flux is time-
integrated from 50 µs to 100 µs simulation time. The magnetic field is
designed to affect only a small region near the slit (extending only $\pm$ 0.5
mm from both sides of the slit). This local restriction of the magnetic field
is artificial, and therefore only the modeling results in the vicinity of the
slit region are reliable. At first, a homogeneous magnetic field in the
complete simulation area was tested, which led to excellent confinement of the
electrons because the magnetic field parallel to the surface reduced the
cross-magnetic field transport. Still, the code did not reach a stationary
solution. Such perfect electron confinement is also unrealistic given the
topology of the magnetic fields of the small magnets in the experiment, where
the field lines also end at surfaces, leading to a loss of electrons. This is
mimicked in the code by restricting the magnetic field only to the vicinity of
the slits so that electron losses to the surface can occur outside the slit
regions.

In the first experiment, the etching of a hydrocarbon film (C:H) in an argon-
oxygen plasma at an RF self-bias of -100 V is analyzed. Two experiments with
(blue solid line) and without (orange dashed line) a magnetic field are
compared and shown as an etch profile across the slit in Fig. 2a. The grey
area indicates the position of the mask above the wafer, and the direction of
the magnetic field is out of the plane, as indicated.

With a magnetic field, one can observe that the etch rate on the right-hand
side is much larger than on the left-hand side (blue solid line in Fig. 2a).
This seemingly contradicts what one would expect, given that the Lorentz force
should deflect the incident ion towards the left-hand side. Additionally, the
$\vec{E}\times\vec{B}$ drift of the electrons, for an electric field
perpendicular to the substrate, should also lead to a deflection of the plasma
to the left-hand side.

Figure 2: 3D etching profiles of C:H in an argon-oxygen plasma with -100 V
self-bias (a) and of silicon in a C:F plasma with -150 V self-bias (b) as well
as the integrated ion flux from the simulation at 100 V applied voltage (c).

In a second experiment, the asymmetry of Si etching in a CF4 plasma at an RF
self-bias of -150 V is analyzed. Two experiments with and without the magnetic
field are compared and shown as an etch profile across the slit in Fig. 2b.
One can see an asymmetry in the etch profile similar to the experiments on the
C:H film etching. Additionally, a local maximum in the center of the trench
can be observed. XPS analysis (not shown) revealed the deposition of a C:F
polymer in the center of the trench.

The integrated ion flux below the mask structure from the PIC/MCC simulation
for an applied voltage of 100 V is shown in Fig. 2c. The integrated flux is
also higher on the right-hand side of the slit. This is in agreement with the
experiment. The peaks at the edges of the profile are caused by the glancing
angle reflection of ions on the inside walls of the mask. However, any
quantitative comparison of the profiles from simulation and experiment is
limited, since the simulation investigates only the ion flux and does not yet
include the chemical sputtering process to obtain an etch rate. Furthermore,
the space- and time-resolved electric field and electron dynamics in the mask
region are investigated. Fig. 3 shows the space-resolved electric field (a)
and electron densities (b) at different phases relative to the RF cycle. The
mask structure modifies the electric field, resulting in the electric field
vectors inside the slit pointing towards the interior of the mask structure
parallel to the substrate below. The plasma penetrates the slit at the
different phases of the RF cycle, predominantly at $\phi=\pi/2$ when the
sheath collapses, and the sheath voltage becomes minimal. At $\phi=3/2\pi$,
the plasma is expelled from the slit due to maximum sheath expansion. It is
interesting to regard the phase of maximum plasma penetration at $\phi=\pi/2$.
Here, one can see that the plasma penetrates asymmetrically into the slit
opening. One could also note that the plasma density shows some structure
above the mask. By inspecting the time development of the density maps, one
can observe the propagation of waves along the interface. Based on the
geometry of the magnetic field and density gradient, these waves are
presumably gradient drift waves.

Figure 3: The space-resolved electric field (a) and the electron density (b)
from the PIC simulation for an RF amplitude of 100 V at different phases of
the RF cycle, as indicated. The red rectangles represent the mask.

The experiment revealed asymmetry in the etching process due to a magnetic
field and mask in front of a wafer. Based on the comparison between the
experiment and modeling, two processes overlap, as illustrated in Fig. 4:

Figure 4: Mechanisms to create a 3D deposition or etching profile: (a) varying
penetration of the plasma into the mask structure depending on the
$\vec{E}\times\vec{B}$ drifts, (b) ion focusing and re-deposition from the
mask.

  * •

Plasma insertion: The plasma penetrates the metal slit structure, and an
electric field develops between the plasma and the mask sidewalls. This
electric field in the plasma sheath is parallel to the wafer surface and
causes an $\vec{E}\times\vec{B}$ drift on the electrons either towards the
wafer or away from the wafer. As a result, the plasma penetrates further into
the slit of the mask on one side of the slit compared to the other (Fig. 4a).

This plasma insertion should scale with the sheath thickness in relation to
the slit width of the mask. If we take the simple Child-Langmuir sheath model
and an electron temperature of 3 eV and a plasma density of 1017 m3, we obtain
a sheath thickness of 0.6 mm at a bias voltage of -150 V. Then the plasma may
not be able to penetrate the slit structure completely. However, the RF sheath
is modulated, and within an RF cycle, the plasma partially enters the slit
structure when the sheath collapses.

  * •

Ion scattering, re-deposition: The etch profile is further modified by
glancing angle scattering of ions at the inside wall of the metal mask. In
addition, the re-deposition of any deposits from the inside walls of the mask
causes re-deposition on the substrate surface (Fig. 4b). A typical example is
C:F polymer re-deposition during CF4 etching of silicon.

This effect can be controlled by changing the magnetic field or any electric
field by biasing these masks, but also by changing the plasma properties,
which affects the relation between the sheath thickness and mask feature sizes
since the guiding length scale is the plasma sheath thickness which ranges for
typical plasma densities of at least 10s of microns or larger. The magnetic
field’s impact is crucial for the $\vec{E}\times\vec{B}$ drift but also
modifies the sheath thickness. Since the magnetic field directs parallel to
the wafer surface, the confinement of electrons is improved so that the
effective bias voltage is smaller than in the non-magnetized case.

In conclusion, it can be stated that adding a magnetic field to an etch mask
in front of a wafer induces an asymmetry in the incident ion flux onto the
wafer. The effect is caused by asymmetric penetration of the plasma into the
mask structure, due to the $\vec{E}\times\vec{B}$ drift, which increases the
penetration of the plasma on one side of the mask and decreases it on the
other side. This effect depends sensitively on the relation between the mask’s
length scale and the plasma sheath’s length scale to allow plasma penetration.
The exploitation as a tool to also asymmetrically etch depends on the etch
chemistry of chemical sputtering, which was very strong in the case of C:H
etching, leading to very asymmetric etch profiles but only small in the case
of silicon etching.
In the future, it will be necessary to further explore this effect
experimentally to optimize the etch asymmetry and complementary develop the
PIC/MCC model by including the chemistry directly and a realistic surface
model for the different materials which includes the reflection of particles
and the emission of secondary electrons.

###### Acknowledgements.

The authors thank Martin Hoffmann (Microsystem Technology, Ruhr University
Bochum, Germany) and Stefan Sinzinger (Technical Optics, Technical University
Ilmenau, Germany) for helpful discussions.

## Conflict of interest statement

The authors have no conflicts to disclose.

## Data Availability Statement

Data are available on request from the authors.

## References

  * Sekine (2002) M. Sekine, “Dielectric film etching in semiconductor device manufacturing: Development of sio2 etching and the next generation plasma reactor,” Applied Surface Science 192, 270–298 (2002).
  * Radjenović and Radmilović-Radjenović (2014) B. Radjenović and M. Radmilović-Radjenović, “The implementation of the surface charging effects in three-dimensional simulations of sio 2 etching profile evolution,” Engineering 2014 (2014).
  * Krüger  _et al._ (2019) F. Krüger, S. Wilczek, T. Mussenbrock, and J. Schulze, “Voltage waveform tailoring in radio frequency plasmas for surface charge neutralization inside etch trenches,” Plasma Sources Science and Technology 28, 075017 (2019).
  * Orfert and Richter (1999) M. Orfert and K. Richter, “Plasma enhanced chemical vapor deposition of sin-films for passivation of three-dimensional substrates,” Surface and Coatings Technology 116, 622–628 (1999).
  * Lim and Yun (2004) J.-w. Lim and S.-J. Yun, “Method for forming nitrogen-containing oxide thin film using plasma enhanced atomic layer deposition,” (2004), uS Patent 6,723,642.
  * Arts  _et al._ (2022) K. Arts, S. Hamaguchi, T. Ito, K. Karahashi, H. C. M. Knoops, A. J. M. Mackus, and W. M. M. (Erwin) Kessels, “Foundations of atomic-level plasma processing in nanoelectronics,” Plasma Sources Science and Technology 31, 103002 (2022).
  * Brunner  _et al._ (2004) R. Brunner, M. Ferstl, S. Hohng, J. O. White, M. Burkhardt, A. Pesch, and O. Sandfuchs, “Diffraction-based solid immersion lens,” Journal of the Optical Society of America A 21, 1186 (2004).
  * Amako, Sawaki, and Fujii (2009) J. Amako, D. Sawaki, and E. Fujii, “High-efficiency diffractive beam splitters surface-structured on submicrometer scale using deep-UV interference lithography,” Applied Optics 48, 5105 (2009).
  * Miller  _et al._ (1993) J. M. Miller, M. R. Taghizadeh, J. Turunen, and N. Ross, “Multilevel-grating array generators: Fabrication error analysis and experiments,” Applied Optics 32, 2519 (1993).
  * Zhou  _et al._ (2021) Y. Zhou, X. Zhu, Y. Yang, D. Ye, R. Chen, and Q. Liao, “Route towards high-performance microfluidic fuel cells: A review,” Sustainable Energy & Fuels 5, 2840–2859 (2021).
  * Hawkeye and Brett (2007) M. M. Hawkeye and M. J. Brett, “Glancing angle deposition: Fabrication, properties, and applications of micro- and nanostructured thin films,” Journal of Vacuum Science & Technology A 25, 1317–1335 (2007), https://pubs.aip.org/avs/jva/article-pdf/25/5/1317/13577268/1317_1_online.pdf .
  * Chang (2020) B. Chang, “Oblique angled plasma etching for 3D silicon structures with wiggling geometries,” Nanotechnology 31, 085301 (2020).
  * Yoon  _et al._ (2022) M. Y. Yoon, H. Yeom, J. H. Kim, J.-R. Jeong, and H.-C. Lee, “Plasma etching of the trench pattern with high aspect ratio mask under ion tilting,” Applied Surface Science 595, 153462 (2022).
  * Ni  _et al._ (2020) S. Ni, E. J. W. Berenschot, P. J. Westerik, M. J. De Boer, R. Wolf, H. Le-The, H. J. G. E. Gardeniers, and N. R. Tas, “Wafer-scale 3D shaping of high aspect ratio structures by multistep plasma etching and corner lithography,” Microsystems & Nanoengineering 6, 25 (2020).
  * Schaepkens and Oehrlein (1998) M. Schaepkens and G. S. Oehrlein, “Asymmetric microtrenching during inductively coupled plasma oxide etching in the presence of a weak magnetic field,” Applied Physics Letters 72, 1293–1295 (1998).
  * Pawar, Ballav, and Kumar (2017) P. Pawar, R. Ballav, and A. Kumar, “Micromachining of Borosilicate Glass: A State of Art Review,” Materials Today: Proceedings 4, 2813–2821 (2017).
  * Hof and Abou Ziki (2017) L. Hof and J. Abou Ziki, “Micro-Hole Drilling on Glass Substrates—A Review,” Micromachines 8, 53 (2017).
  * Weigel  _et al._ (2021) C. Weigel, U. Brokmann, M. Hofmann, A. Behrens, E. Rädlein, M. Hoffmann, S. Strehle, and S. Sinzinger, “Perspectives of reactive ion etching of silicate glasses for optical microsystems,” Journal of Optical Microsystems 1 (2021), 10.1117/1.JOM.1.4.040901.
  * Chabert, Tsankov, and Czarnetzki (2021) P. Chabert, T. V. Tsankov, and U. Czarnetzki, “Foundations of capacitive and inductive radio-frequency discharges,” Plasma Sources Science and Technology 30, 024001 (2021).
  * Chabert and Braithwaite (2011) P. Chabert and N. Braithwaite,  _Physics of radio-frequency plasmas_ (Cambridge University Press, 2011).
  * Donkó  _et al._ (2021) Z. Donkó, A. Derzsi, M. Vass, B. Horváth, S. Wilczek, B. Hartmann, and P. Hartmann, “edupic: an introductory particle based code for radio-frequency plasma simulation,” Plasma Sources Science and Technology 30, 095017 (2021).
  * Turner  _et al._ (2013) M. M. Turner, A. Derzsi, Z. Donkó, D. Eremin, S. J. Kelly, T. Lafleur, and T. Mussenbrock, “Simulation benchmarks for low-pressure plasmas: Capacitive discharges,” Physics of Plasmas 20 (2013), 10.1063/1.4775084.
  * Lieberman and Lichtenberg (1994) M. A. Lieberman and A. J. Lichtenberg, “Principles of plasma discharges and materials processing,” MRS Bulletin 30, 899–901 (1994).
  * Wilczek  _et al._ (2020) S. Wilczek, J. Schulze, R. P. Brinkmann, Z. Donkó, J. Trieschmann, and T. Mussenbrock, “Electron dynamics in low pressure capacitively coupled radio frequency discharges,” Journal of Applied Physics 127 (2020).
  * Trieschmann  _et al._ (2013) J. Trieschmann, M. Shihab, D. Szeremley, A. E. Elgendy, S. Gallian, D. Eremin, R. P. Brinkmann, and T. Mussenbrock, “Ion energy distribution functions behind the sheaths of magnetized and non-magnetized radio frequency discharges,” Journal of Physics D: Applied Physics 46, 084016 (2013).
  * Boris (1970) J. P. Boris, “Relativistic plasma simulation-optimization of a hybrid code,” Proceeding of Fourth Conference on Numerical Simulations of Plasmas (1970).
  * Phelps and Petrovic (1999) A. V. Phelps and Z. L. Petrovic, “Cold-cathode discharges and breakdown in argon: surface and gas phase production of secondary electrons,” Plasma Sources Science and Technology 8, R21 (1999).
  * Ziegler, Ziegler, and Biersack (2010) J. F. Ziegler, M. Ziegler, and J. Biersack, “Srim – the stopping and range of ions in matter (2010),” Nuclear Instruments and Methods in Physics Research Section B: Beam Interactions with Materials and Atoms 268, 1818–1823 (2010), 19th International Conference on Ion Beam Analysis.


# Language-Oriented Communication with Semantic Coding and Knowledge
Distillation for Text-to-Image Generation

###### Abstract

By integrating recent advances in large language models (LLMs) and generative
models into the emerging semantic communication (SC) paradigm, in this article
we put forward to a novel framework of language-oriented semantic
communication (LSC). In LSC, machines communicate using human language
messages that can be interpreted and manipulated via natural language
processing (NLP) techniques for SC efficiency. To demonstrate LSC’s potential,
we introduce three innovative algorithms: 1) semantic source coding (SSC)
which compresses a text prompt into its key head words capturing the prompt’s
syntactic essence while maintaining their appearance order to keep the
prompt’s context; 2) semantic channel coding (SCC) that improves robustness
against errors by substituting head words with their lenghthier synonyms; and
3) semantic knowledge distillation (SKD) that produces listener-customized
prompts via in-context learning the listener’s language style. In a
communication task for progressive text-to-image generation, the proposed
methods achieve higher perceptual similarities with fewer transmissions while
enhancing robustness in noisy communication channels.

Index Terms—  Semantic communication (SC), large language model (LLM),
generative model.

##  1 Introduction

Semantic communication (SC) is an emerging research paradigm that focuses on
the meanings (i.e., semantics) and effectiveness of communicating bits [1, 2,
3, 4, 5]. Deep joint source and channel coding (DeepJSCC) is a prime example
wherein an encoder-decoder structured neural network (NN) acts as a
transceiver, within which task-effective features are extracted from input
data and made into communication messages. These _neural messages_ are the
NN’s hidden-layer activations trained and tailored for a specific task, which
greatly improves communication efficiency [1, 2, 3, 4].

However, neural messages constraint the full potential of SC. First, NN
activations are not always universal messages, as they are influenced by their
training data and communication environment. Indeed, DeepJSCC transceivers
that have been trained separately are hardly interoperable without fine-tuning
[6]. Furthermore, the semantics of these NN activations are nothing but what
remains after achieving effective communication. It is therefore difficult to
interpret and manipulate these semantics as intended.

Fig. 1: An illustration of language-oriented semantic communication (LSC) for
progressive text-to-image generation, empowered by semantic source coding
(SSC), semantic channel coding (SCC), and semantic knowledge distillation
(SKD).

Human language, by contrast, is universal and versatile to describe a broad
range of tasks, owing to its evolution throughout diverse human experiences in
history. Moreover, with recent advances in natural language processing (NLP)
and generative models, machines are now capable of interpreting and
manipulating human language. Motivated by this, in this paper we propose a
novel _language-oriented SC (LSC)_ framework, which facilitates SC through
human _language messages_. The operation of LSC transceivers is threefold:

  1. 1)

Data-to-Language Translation: Text-based cross-modal models transform input
data into language messages to be transmitted (e.g., via CLIP for image-to-
text (I2T) or Whisper for speech-to-text translation).

  2. 2)

Language Analysis & Manipulation: Large language models (LLMs) and other NLP
algorithms (e.g., GPT-4[7], Llama 2[8], and CoreNLP [9]) are utilized for
analyzing the syntax, semantics, and context in language messages and
manipulating these messages for improving communication efficiency.

  3. 3)

Language-to-Data Generation: Text-conditioned generative models produce
intended data using the received message seman (e.g., via Stable Diffusion for
text-to-image (T2I) or Zeroscope for text-to-video generation).

To showcase the potential of LSC, this paper we consider a point-to-point LSC
scenario, where Alice sends a text prompt describing an intended image, word
by word, while Bob progressively generates an image based on the accumulated
received text prompt. The LSC accuracy between Alice and Bob is assessed using
the learned perceptual image patch similarity (LPIPS) that measures the
distance between intended and generated images. To improve the communication
efficiency of LSC, as visualized in Fig. 1, we focus on Step 2), and develop
the following novel algorithms:

  * •

Head-based Semantic Source Coding (SSC) is a lossy compression of the original
prompt by pruning non-head words, inspired from our empirical findings that
sending all words in a prompt does not always achieve the lowest LPIPS. The
heads of a prompt are the words determining the syntactic category of the
prompt, which can be identified, for example, using CoreNLP [9, 10].

  * •

Synonym-based Semantic Channel Coding (SCC) adds redundancy into the prompt by
replacing original head words with their longer synonyms, increasing the
robustness to channel noise perturbing each character of the words. Only the
synonyms ensuring the same semantics of the prompt are of interest, which can
be found by, for instance, using GPT-4 [7].

  * •

In-Context Learning-based Semantic Knowledge Distillation (SKD) aims to
address the out-of-distribution (OOD) prompts due to different language
knowledge between Alice and Bob, and enables Alice to emulate Bob’s prompts by
assimilating Bob’s language knowledge. This can be achieved without re-
training NN model parameters, by harnessing LLM’s unique capability of in-
context learning, i.e., few-shot learning via demonstration [11].

Simulation results reveal SSC compresses transmitted messages by up to
$42.6$%, while surprisingly reducing LPIPS by $0.015$ compared to full
prompts. Applying SCC and SKD further cuts LPIPS by up to $0.007$ and $0.009$
by addressing channel noise and heterogeneous language knowledge.

Related Works: Recent research [3] employs generator models in SC but with
neural messages, distinct from LSC’s language messages. The ts_zip [12]
algorithm exploits LLM-based synonyms for compression, differing from our
synonym-based SCC for robustness and from our head-based SSC. LSC stands apart
from other language-based SC studies that mainly focus on I2T compression or
on the entropy of text truthfulness [13], in contrast to LSC harnessing LLM
and NLP techniques to analyze and manipulate language messages for SC.

Table 1: Message compression ratios of SSC with SSC and/or SKD.

Compression | w.o.SSC | SSC | SSC+SKD | SSC | SSC+SCC | SSC+SCC+SKD | SSC | SSC+SCC | SSC+SCC+SKD
---|---|---|---|---|---|---|---|---|---
Ratio | w.o. channel noise | SNR = $6.25$ dB | SNR = $8.75$ dB
w.r.t. Word | $1$ | 0.641 | $0.654$ | 0.641 | 0.641 | 0.641 | 0.654 | 0.654 | 0.654
w.r.t. Character | $1$ | 0.426 | $0.468$ | 0.426 | $0.531$ | $0.525$ | 0.468 | $0.579$ | $0.565$
LPIPS | $0.718$ | $0.703$ | 0.697 | $0.736$ | $0.730$ | 0.721 | $0.726$ | $0.719$ | 0.715

##  2 Semantic Source Coding for Progressive Text-to-Image Generation

In this section, we propose SSC for a progressive text-to-image (T2I)
generation task in a point-to-point communication scenario, as elaborated
next.

1) Image-to-Text Translation: Alice has an intended image $\mathbf{v}$ to
send, and translates it into a text prompt $\mathbf{x}$, a sequence containing
a set $\mathbf{X}$ of words presented in a specific order, given as:

$\displaystyle\mathbf{x}=\texttt{I2T}(\mathbf{v})=(\mathbf{x}_{1},\mathbf{x}_{2},\cdots,\mathbf{x}_{|\mathbf{X}|}),$
(1)

where $\mathbf{x}_{i}$ is the $i$th word comprising $|\mathbf{x}|$ characters.
The function $\texttt{I2T}(\cdot)$ represents an image-to-text (I2T) encoder
such as BLIP [14] or CLIP [15].

2) Head-based Semantic Source Coding (SSC): Alice aims to compress and
transmit text characters of $\mathbf{x}$ while maintaining the semantics of
$\mathbf{x}$. The semantics can be maintained when the key words of
$\mathbf{x}$ are presented without loosing their syntax and context. To this
end, SSC first identifies a set $\mathbf{H}$ of $\mathbf{x}$’s head words that
determine the prompt’s syntactic category in linguistic analysis. While
keeping head words’ order of appearance in $\mathbf{x}$, SSC produces a
compressed sequence
$\mathbf{h}=(\mathbf{h}_{1},\mathbf{h}_{2},\cdots,\mathbf{h}_{|\mathbf{X}|})$
in which:

$\displaystyle\mathbf{h}_{t}=\begin{cases}\mathbf{x}_{i},&\text{if
}\mathbf{x}_{i}\in\mathbf{H}\\\ \emptyset,&\text{otherwise}.\end{cases}$ (2)

The head words in $\mathbf{H}$ can be identified using the CoreNLP algorithm
[10], i.e., $\mathbf{H}=\texttt{CoreNLP}(\mathbf{x})$. Consequently, SSC
yields the compression ratio $|\mathbf{H}|/|\mathbf{X}|\leq 1$ in terms of
words, and
$\sum_{\mathbf{x}_{i}\in\mathbf{H}}|\mathbf{x}_{i}|/\sum_{\mathbf{x}_{i}\in\mathbf{X}}|\mathbf{x}_{i}|$
in terms of characters.

3) Text-to-Image Generation: Bob receives the head words of $\mathbf{h}_{i}$
in order, and progressively generates an image using a T2I generator such as
Stable Diffusion [16] and DALL$\cdot$E. At the $i$-th head word reception with
$i\in\\{1,2,\cdots,|\mathbf{H}|\\}$, the received prompt is $\mathbf{h}(t)$,
and the generated image is:

$\displaystyle\hat{s}(t)=\texttt{T2I}(\mathbf{h}(t))=\texttt{T2I}((\mathbf{h}_{1},\mathbf{h}_{2},\cdots,\mathbf{h}_{t})).$
(3)

The perceptual similarity between Bob’s generated $\mathbf{\hat{v}}(t)$ and
Alice’s intended image $\mathbf{v}$ is measured by the learned perceptual
image patch similariy (LPIPS) score [17] that calculates the distance at
hidden layers of pre-trained AlexNet, given as:

$\displaystyle\texttt{LPIPS}(\mathbf{v},\mathbf{\hat{v}}(t))=\sum_{l}\frac{1}{H_{l}W_{l}}\sum_{h,w}\|f(\mathbf{v})-f(\mathbf{\hat{v}}(t)\|^{2}_{2}.$
(4)

The term $l$ identifies the $l$-th layer having its width $w$, height $h$, and
dimension $H_{l}\times W_{l}$ with an activation function $f(\cdot)$.

##  3 Semantic Channel Coding in Noisy Communication Channels

In the previous section, we presume that Alice’s transmitted head words are
perfectly received at Bob. In this section, we consider a noisy channel, and
propose SCC to address noisy head word receptions at Bob.

1) Noisy Channel Model: Alice individually transmits a set
$C_{\mathbf{h}_{t}}$ of characters in the head word $\mathbf{h}_{t}$.
Following a discrete memoryless channel (DMC) model, Bob receives the head
word $\mathbf{\hat{h}}_{t}$ containing a set $\hat{C}_{\mathbf{h}_{t}}$ of
characters, each of which is perturbed as a different character with a cross-
over probability $\epsilon>0$ and is otherwise successfully received.

2) Synonym-based Semantic Channel Coding (SCC): In this noisy channel, SCC
aims to enhance $\mathbf{h}_{t}$’s robustness by increasing
$|\mathbf{C}_{\mathbf{h}_{t}}|$ while maintaining the same semantics of the
prompt $\mathbf{h}(t)$. In the aforementioned channel, Bob encounters the same
error in each characters following a geometric distribution. This does not
allow communicating short words like ”cat” that changes its semantics even
with a single-character variation (e.g., bat, cut, and car), motivating SCC.
In SCC, we consider a set $\mathbf{S}_{\mathbf{h}_{t}}$ of candidate synonyms
of $\mathbf{h}_{t}$, given as:

$\displaystyle\mathbf{S}_{\mathbf{h}_{t}}=\\{\mathbf{s}_{1},\mathbf{s}_{2},...\mathbf{s}_{|\mathbf{S}_{\mathbf{h}_{t}}|}\\},$
(5)

where $\mathbf{s}_{t}$ contains a set $\mathbf{C}_{\mathbf{s}_{t}}$ of
characters. Although $\mathbf{S}_{\mathbf{h}_{t}}$ can be found using a
dictionary, it ignores the context of $\mathbf{h}(t)$,and does not guarantee
the intended semantics. To solve this, SCC utilizes an LLM such as GPT-4 and
Llama 2, a decoder-only autoregressive model that can predict the most in-
context appropriate synonym $\mathbf{s}_{t}^{*}$ of $\mathbf{h}_{t}$ in
$\mathbf{h}(t)$ by masking $\mathbf{h}_{t}$ and maximizing the following
conditional unmasking probability:

$\displaystyle\mathbf{s}_{t}^{*}=\max_{\mathbf{s}_{j}\in\mathcal{W}}p_{(}\mathbf{s}_{j})=\Pr(\mathbf{s}_{j}|\mathbf{h}(t)\backslash\mathbf{h}_{t}),$
(6)

where $\mathcal{W}$ is a set of total characters, e.g., $128$ characters in
ASCII. By relaxing this LLM, we obtain a set
$\mathbf{\hat{S}}_{\mathbf{h}_{t}}$ of in-context synonyms associated with
their unmasking probabilities exceeding a threshold $p_{c}>0$:

$\displaystyle\mathbf{\hat{S}}_{\mathbf{h}_{t}}=\\{\mathbf{\hat{s}}_{1},\mathbf{\hat{s}}_{2},\cdots,\mathbf{\hat{s}}_{.}\\}=\\{\mathbf{s}_{j}|p_{\mathbf{s}_{j}}\geq
p_{c}\\}.$ (7)

Consequently, SSC can increase noise robustness of $\mathbf{h}_{t}$ within a
set $L_{\mathbf{h}_{t}}$ of the levels in terms of characters, given as

$\displaystyle
L_{\mathbf{h}_{t}}=\\{|\mathbf{\hat{s}}_{j}|\in\mathbf{\hat{S}}_{\mathbf{h}_{t}}||\mathbf{h}_{t}|\leq|\mathbf{\hat{s}}_{j}|\leq
L_{c}\\},$ (8)

where $L_{c}$ is the number of characters of the lengthiest synonym of
$\mathbf{h}_{t}$, i.e.,
$L_{c}=\lfloor\max_{\mathbf{\hat{s}}_{j}\in\mathbf{\hat{S}}_{\mathbf{h}_{t}}}|\mathbf{C}_{\mathbf{\hat{s}}_{j}}|\rfloor$.

##  4 Semantic Knowledge Distillation in Heterogeneous Language Knowledge

In this section, we aim to address the problem when Alice and Bob have
different knowledge on text-image relations by proposing SKD that enables
Alice to produce Bob-customized text prompts via in-context learning.

1) Heterogeneous Knowledge Model: BLIP and CLIP are encoder-decoder NN models
that store knowledge on image-text relations through cross-attention weights,
enabling T2I and I2T conversions. Suppose that Alice has BLIP for T2I while
Bob utilizes CLIP for I2T. This incurs OOD generation in both, decreasing
LPIPS.

2) In-Context Learning-based Semantic Knowledge Distillation (SKD): A pre-
trained LLM has excessive knowledge containing spurious correlations, and
conditioning its knowledge within a specific context can improve task
performance. In-context learning enables this by feeding few exemplary input-
output pairs to teach the LLM a desired context, i.e., few-shot learning via
demonstration [11]. Meanwhile, knowledge distillation (KD) is a method to
transfer a target (teacher) model’s knowledge into another (student) model by
minimizing their output differences for common inputs [18]. Inspired from in-
context learning and KD, SKD shows $K$ input images to Alice and Bob that
generate $K$ output text prompts, using BLIP and CLIP, respectively, that are
fed into an LLM for demonstration. This in-context learned LLM becomes a text-
to-text (T2T) translator that can produce Bob-customized prompt
$\hat{\mathbf{x}}_{b}$ for a given Alice’s prompt $\mathbf{x}_{a}$, given as:

$\displaystyle\hat{\mathbf{x}}_{b}=\texttt{T2T}\left(\mathbf{x}_{a};\\{\mathbf{v}^{(i)},\mathbf{x}_{a}^{(i)},\mathbf{x}_{b}^{(i)}\\}_{i=1}^{K}\right),$
(9)

where $\mathbf{x}_{a}^{(i)}$ and $\mathbf{x}_{b}^{(i)}$ are the $i$-th output
prompts at Alice and Bob, respectively. After SKD, $\hat{\mathbf{x}}_{b}$ is
fed into SSC and/or SCC. Note that SKD is applicable before SSC and/or after
SCC. We focus on the former for simplicity.

##  5 Numerical Results

Simulation Settings: We consider that Alice’s I2T encoder is BLIP [19], and
Bob’s T2I generator is Stable Diffusion v1.5 [16] that generates each image
from a prompt with $50$ denoising steps. This diffusion process is conditioned
by the text prompt encoded using CLIP [15]. SKD and SCC that require LLMs are
based on GPT-4 [7], while SSC is run by CoreNLP [9, 10]. For image data, the
Flicker8k dataset is considered, containing 8,092 samples, each with 256x256
pixels [20]. For text prompts, each character is 8-bit ASCII coded, and
modulated using 16QAM. During SCC, $p_{c}$ is set as $0.72$. All LPIPS values
are averaged over 100 simulation runs, except for snapshot visualizations in
Fig. 2.

Fig. 2: Alice’s image and prompts (left) and Bob’s generated images and LPIPS
(right), with SSC, SCC, and SKD. Fig. 3: SSC with or without SKD w.r.t.
transmitted characters. Fig. 4: SSC with or without SCC and SKD w.r.t. SNR.

Impact of SSC: Tab. 1 reveals SSC achieves 64.1% word compression and 42.6% in
character. Surprisingly, mean LPIPS improves by $0.015$, suggesting SSC’s
roles not only in compression but also in prompt engineering. Fig. 3
highlights SSC’s dual benefits (solid red), head extraction and appearance-
based ordering. To dissect their LPIPS reduction contributions, we introduce
two baselines: _Random SC_ (dotted gray), which maintains appearance order
with the same compression ratio as SSC but transmits random words from
$\mathbf{x}$ instead of head words; and _Random sequence SSC_ (dotted red),
sending head words from $\mathbf{h_{t}}$ like SSC but in a shuffled order.
Results indicate head extraction reduces mean LPIPS by up to $0.04$, and
appearance-based ordering contributes up to a $0.012$ reduction, as seen when
comparing SSC with Random SC and Random sequence SSC, respectively.

Impact of SCC: In Fig. 4, we observe that mean LPIPS decreases with SNR.
Comparing SSC+SCC (dotted red) to SSC (solid red), SCC contributes to a
reduction in mean LPIPS by up to $0.007$. This reduction diminishes with SNR.
However, SCC compromises compression, increasing it by up to 10.5% as shown in
Tab. 1. In these simulations the increase in characters is capped at $4$. In
certain instances, as illustrated in Fig. 2 at $\text{SNR}=7.5$dB, the LPIPS
reduction from SCC can be as much as 7.57x its average. This suggests
potential benefits in optimizing SSC level based on given channel conditions
for future research.

Impact of SKD: As illustrated in Figs. 3 and 4, SKD contributes to a reduction
in mean LPIPS by up to $0.006$ and $0.009$, respectively. Notably, the latter
reduction surpasses even the contribution of SCC to LPIPS reduction. However,
SKD may extend the prompt length, e.g., by an average of $5$ characters in
Fig. 3, highlighting a trade-off between compression and LPIPS.

##  6 Conclusion

In this article we proposed LSC, and developed SSC, SCC, and SKD that leverage
NLP and LLM techniques to improve LSC’s SC efficiency under noisy channels and
heterogeneous T2I/I2T knowledge. Future research might explore various tasks
such as I2T-based control and compare LSC’s performance with its DeepJSCC
counterpart. It could also be interesting to exploit other LLM capabilities
such as reasoning and interactions with humans.

## References

  * [1] D. Gündüz, Z. Qin, I. E. Aguerri, H. S. Dhillon, Z. Yang, A. Yener, K. K. Wong, and C.-B. Chae, “Beyond transmitting bits: Context, semantics, and task-oriented communications,” _IEEE Journal on Selected Areas in Communications_ , vol. 41, no. 1, pp. 5–41, 2022.
  * [2] Z. Qin, X. Tao, J. Lu, W. Tong, and G. Y. Li, “Semantic communications: Principles and challenges,” __[Online]_. ArXiv preprint: arXiv.2201.01389_, 2022.
  * [3] S. Barbarossa, D. Comminiello, E. Grassucci, F. Pezone, S. Sardellitti, and P. Di Lorenzo, “Semantic communications based on adaptive generative models and information bottleneck,” __[Online]_. ArXiv preprint: arXiv:2309.02387_, 2023.
  * [4] S. Seo, J. Park, S.-W. Ko, J. Choi, M. Bennis, and S.-L. Kim, “Towards semantic communication protocols: A probabilistic logic perspective,” _IEEE Journal on Selected Areas in Communications_ , 2023.
  * [5] H. Seo, J. Park, M. Bennis, and M. Debbah, “Semantics-native communication via contextual reasoning,” _IEEE Transactions on Cognitive Communications and Networking_ , vol. 9, no. 3, pp. 604–617, 2023.
  * [6] J. Choi, J. H. Park, S.-W. Ko, J. Choi, M. Bennis, and S.-L. Kim, “Semantics alignment via split learning for resilient multi-user semantic communication,” _submitted to IEEE Transactions on Vehicular Technology_ , 2023.
  * [7] OpenAI, “GPT-4 technical report,” 2023.
  * [8] H. Touvron, L. Martin, K. Stone _et al._ , “Llama 2: Open foundation and fine-tuned chat models,” 2023.
  * [9] C. D. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, and D. McClosky, “The Stanford CoreNLP natural language processing toolkit,” in _Association for Computational Linguistics (ACL) System Demonstrations_ , 2014, pp. 55–60. [Online]. Available: http://www.aclweb.org/anthology/P/P14/P14-5010
  * [10] D. Chen and C. D. Manning, “A fast and accurate dependency parser using neural networks,” in _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_ , 2014, pp. 740–750.
  * [11] T. Brown, B. Mann, N. Ryder _et al._ , “Language models are few-shot learners,” _Advances in neural information processing systems_ , vol. 33, pp. 1877–1901, 2020.
  * [12] F. Bellard. ts_zip. [Online]. Available: https://bellard.org/ts_server/ts_zip.html
  * [13] R. Carnap, Y. Bar-Hillel _et al._ , “An outline of a theory of semantic information,” _Research Laboratory of Electronics, Massachusetts Institute of Technology_ , 1952.
  * [14] J. Li, D. Li, C. Xiong, and S. Hoi, “BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” in _International Conference on Machine Learning_. PMLR, 2022, pp. 12 888–12 900.
  * [15] A. Radford, J. W. Kim, Hallacy _et al._ , “Learning transferable visual models from natural language supervision,” in _International conference on machine learning_. PMLR, 2021, pp. 8748–8763.
  * [16] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 10 684–10 695.
  * [17] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 586–595.
  * [18] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” _NeurIPS Workshop on Deep Learning _[Online]_. ArXiv preprint: arXiv:1503.02531_, 2014.
  * [19] J. Li, D. Li, C. Xiong, and S. Hoi, “BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” 2022. [Online]. Available: https://arxiv.org/abs/2201.12086
  * [20] M. Hodosh, P. Young, and J. Hockenmaier, “Framing image description as a ranking task: Data, models and evaluation metrics,” _Journal of Artificial Intelligence Research_ , vol. 47, pp. 853–899, 2013.