# Hybrid integrated near UV lasers using the deep-UV $\mathrm{Al_{2}O_{3}}$
platform

C.A.A. Franken  Laser Physics and Nonlinear Optics Group, Department of
Science and Technology, MESA+ Institute of Nanotechnology, University of
Twente, Enschede, The Netherlands  W.A.P.M. Hendriks  Integrated Optical
Systems, Department of Science and Technology, MESA+ Institute of
Nanotechnology, University of Twente, Enschede, The Netherlands  L.V. Winkler
Laser Physics and Nonlinear Optics Group, Department of Science and
Technology, MESA+ Institute of Nanotechnology, University of Twente, Enschede,
The Netherlands  TOPTICA Photonics, Gräfelfing, Germany  M. Dijkstra
Integrated Optical Systems, Department of Science and Technology, MESA+
Institute of Nanotechnology, University of Twente, Enschede, The Netherlands
A.R. do Nascimento Jr  PHIX B.V., Enschede, The Netherlands  A. van Rees
Laser Physics and Nonlinear Optics Group, Department of Science and
Technology, MESA+ Institute of Nanotechnology, University of Twente, Enschede,
The Netherlands  M.R.S. Mardani  Integrated Optical Systems, Department of
Science and Technology, MESA+ Institute of Nanotechnology, University of
Twente, Enschede, The Netherlands  R. Dekker  LioniX International B.V.,
Enschede, The Netherlands  J. van Kerkhof  PHIX B.V., Enschede, The
Netherlands  P.J.M. van der Slot  Laser Physics and Nonlinear Optics Group,
Department of Science and Technology, MESA+ Institute of Nanotechnology,
University of Twente, Enschede, The Netherlands  S.M. García-Blanco
Integrated Optical Systems, Department of Science and Technology, MESA+
Institute of Nanotechnology, University of Twente, Enschede, The Netherlands
K.-J. Boller  Laser Physics and Nonlinear Optics Group, Department of Science
and Technology, MESA+ Institute of Nanotechnology, University of Twente,
Enschede, The Netherlands

(February 22, 2023)

Hybrid integrated diode lasers have so far been realized using silicon,
polymer, and silicon nitride ($\mathbf{Si_{3}N_{4}}$) waveguide platforms for
extending on-chip tunable light engines from the infrared throughout the
visible range. Here we demonstrate the first hybrid integrated laser using the
aluminum oxide ($\mathbf{Al_{2}O_{3}}$) deep-UV capable waveguide platform. By
permanently coupling low-loss $\mathbf{Al_{2}O_{3}}$ frequency-tunable Vernier
feedback circuits with GaN double-pass amplifiers in a hermetically sealed
housing, we demonstrate the first extended cavity diode laser (ECDL) in the
near UV. The laser shows a maximum fiber-coupled output power of 0.74 mW,
corresponding to about 3.5 mW on chip, and tunes more than 4.4 nm in
wavelength from 408.1 nm to 403.7 nm. Integrating stable, single-mode and
tunable lasers into a deep-UV platform opens a new path for chip-integrated
photonic applications.

A wide range of emerging photonic applications in the ultraviolet requires
chip-sized integrated laser sources in the ultraviolet with wide tunability
and high coherence. Specifically, such lasers would unlock increased
integration density and upscaling in integrated quantum photonics [1, 2, 3],
UV spectroscopy [4, 5], UV biophotonics [6], and multiple UV optical clock
transitions [7].

Heterogeneous and hybrid integrated lasers provide a wide tuning range and
high coherence at longer wavelengths, while maintaining small size and high
efficiency. In these devices, photonic integrated circuits (PICs) are used to
spectrally filter and couple back light from III-V semiconductor optical
amplifiers to impose tunability and improve coherence. In the infrared telecom
range, such lasers are now providing tunability across the entire gain
bandwidth, and allow for coherence levels which are becoming comparable to
advanced bulk laser systems. This evolution in performance became possible by
introducing a change of paragdigm in chosing the optical materials for PIC
fabrication. The initially used semiconductor feedback circuits, intended to
address the infrared telecom range, offered only small material bandgaps, such
as in Si waveguides (1.1 eV) or InP waveguides (1.3 eV). The situation changed
dramatically when low-loss waveguides with a much wider bandgap became
available. An example of such a waveguide platform is silicon nitride
($\mathrm{Si_{3}N_{4}}$) embedded in silicon oxide ($\mathrm{SiO_{2}}$) [8, 9]
which provides a bandgap as high as 3.3 eV [10]. This reduced both linear and
nonlinear material losses, enabled highly frequency selective waveguide
circuits and allowed to maximize laser coherence with extended on-chip photon
lifetimes. Recently, the wide bandgap also enabled the first realization of
hybrid integrated lasers in the visible range with mW-level fiber-coupled
output [11, 12]. Employing separate feedback chips for spectral narrowing and
frequency pulling with Fabry-Perot lasers resulted in visible tunable output
as well. Milliwatt level fiber-coupled powers were obtained in the blue range
(450-460 nm), about 500 µW in the violet [13], however, at the very end of the
visible the power was restricted to the 100 and 10-µW regimes [14, 15],
respectively.

For reaching out to shorter wavelengths in the UV range, it has recently been
discussed whether employing silicon nitride would still be appropriate [16].
However, a straightforward extension to the UV range using
$\mathrm{Si_{3}N_{4}}$ feedback circuits, especially combined with 5-eV GaN
diodes at 250 nm can be excluded [17]. Moving into these ranges would increase
the waveguide propagation losses strongly, as the photon energy approaches,
matches and supersedes the 3.3-eV silicon nitride bandgap (corresponding to
380 nm) [18, 19]. Ultimately, if losses become too high, the feedback circuit
loses its filter and feedback function, and control over tuning and coherence
fails. Even below the bandgap energy, $\mathrm{Si_{3}N_{4}}$ has shown to be
susceptible to nonlinear excitation [20, 21], while UV irradiation creates
defects that introduce absorption also at below-bandgap wavelengths [22].

Clearly, these considerations and observations require to introduce a deep-UV
capable platform to extend the complex functionalities of laser feedback
circuits into the UV, such that the material bandgap remains much wider than
the targeted UV wavelength. The most promising materials for this task are
aluminum nitride (AlN) and aluminum oxide ($\mathrm{Al_{2}O_{3}}$) because of
their huge bandgaps of 6 eV (200 nm) and 7.6 eV (165 nm) respectively, [23,
24]. As the $\mathrm{Al_{2}O_{3}}$ platform does not suffer from anisotropy
and high sidewall roughness due to AlN crystallinity and provides a wider
spectral coverage it seems more suitable for deep-UV applications. Further
indicators of suitability as a laser feedback platform are that simple low-
loss waveguide components have already been fabricated [25, 23, 6], extending
the established range of fabrication tools for $\mathrm{Al_{2}O_{3}}$-based
infrared waveguide lasers [26, 27]. Tight guiding in $\mathrm{Al_{2}O_{3}}$
high-Q ridge waveguide resonators has been demonstrated so far only in the
near-UV, showing intrinsic qualitity factors of Q $>$ 470.000 [23]. Comparably
low straight propagation losses were found in the ultraviolet as well (3 dB/cm
at 360 nm [6]).

These features underline the potential of $\mathrm{Al_{2}O_{3}}$ waveguides
for spectral control of chip-integrated UV lasers, however, in this wavelength
range there are additional challenges. Due to the high photon energy, losses
and damage can occur at all UV-exposed surfaces and interfaces such as the
facets of semiconductor optical amplifiers, i.e., GaN. The reasons are photo-
induced chemical processes with atmospheric moisture and gases that form
surface defects or absorptive layers [28, 29]. This degradation needs to be
counteracted, such as by hermetic sealing for shielding from ambient humidity.
In the infrared, hybrid integration with transparent epoxy for permanent chip-
to-chip bonding, heterogeneous integration, and packaging solely aim on long-
term frequency stability, robustness and electro-optical connectivity. With UV
lasers, however, chip-to-chip bonding, chip-to-fiber bonding, and photonic
packaging [30] has to be UV compatible as well. Similarly, photonic
wirebonding [31] and heterogeneous integration via direct bonding [32] or
transfer printing [33] would have to involve UV-suitable materials, such as
AlN as buffer layers [34]. Otherwise, stable long-term laser operation can not
be achieved.

Here we present a chip-scale hybrid integrated laser, where for the first time
the deep-UV capable $\mathrm{Al_{2}O_{3}}$ platform is employed (see in Fig.
1)[35]. Using CMOS compatible technology, $\mathrm{Al_{2}O_{3}}$ waveguides
embedded in a $\mathrm{SiO_{2}}$ cladding are fabricated for tightly guided
single-mode propagation, minimized side-wall scattering, and minimized bend
radiation loss. For obtaining single wavelength laser oscillation, two
sequentially coupled $\mathrm{Al_{2}O_{3}}$ microring resonators are
fabricated and connected in Vernier configuration [36], which provides
frequency selective feedback to a near UV InGaN double-pass semiconductor
amplifier. Thin-film micro-heaters enable thermo-optical laser tuning and
provide tunable output coupling. Hybrid integration serves for permanent
optical coupling of amplifier, feedback chip and output fibers. The packaged
chip assembly is based on hermetic sealing to yield protection against
degradation for long-term operation. The laser generates a fiber-coupled
output power of 0.74 mW, which corresponds to an on-chip power of 3.5 mW, and
is tunable in wavelength more than 4.4 nm from 408.1 nm to 403.7 nm.

Figure 1: a) Schematic showing the functional design of the laser, with the
InGaN amplifier edgecoupled to the $Al_{2}O_{3}$ feedback chip. b) Cross-
section of the $Al_{2}O_{3}$ waveguide embedded in $SiO_{2}$ showing the
calculated intensity profile of the optical mode (at 405 nm), with a
horizontal and vertical modefield diameter (MFD) of 0.28 $\mu m$ and 0.14 $\mu
m$ respectively. c) Transmission function (solid line) for light resonating
with both rings in the Vernier filter. A smoothed curve for the measured
amplified spontanous emission (ASE) of the amplifier driven at 55 mA (dashed
line) shows the Vernier filter can select a single wavelength in the amplifier
emission bandwidth. d) Light at 405 nm injected in an 11.3-cm spiral
structure. A series of spirals and paths of different length were used to
measure the straight propagation loss of $\mathrm{\alpha}$ = 2.8 $\pm$ 0.3
dB/cm. e) Top-down microscopic picture of the hybrid laser, showing the
waveguides, heaters, wirebonds and location of the InGaN amplifier and fiber
array in detail. f) The hybrid integrated and packaged laser in a hermetically
sealed nitrogen environment. Here, the laser is driven with a pump current
above threshold. The white substance on the bottom of the glass lid is a
getter material which absorbs any remaining volatile gasses and moisture in
the package.

### Waveguide cross-section and fabrication

We chose to make use of $\mathrm{Al_{2}O_{3}}$ cores embedded in a
$\mathrm{SiO_{2}}$ cladding to provide a maximum electronic bandgap (7.3 eV
[23] and 9.3 eV [37], respectively). To identify a suitable core cross-section
for fabrication, preceding infrared and UV measurements were consulted and
used for scaling. With 400 nm thick and 2000 nm wide straight
$\mathrm{Al_{2}O_{3}}$ waveguides we measured losses of 0.05 dB/cm at a
wavelength of 1550 nm. In the UV at 377 nm, for a 170 nm thick
$\mathrm{Al_{2}O_{3}}$ film deposited with a reactive sputtering process and
followed by chemical mechanical polishing, we measured slab propagation losses
of 0.6 $\pm$ 0.3 dB/cm [38]. Choosing a conservative approach for this first-
time $\mathrm{Al_{2}O_{3}}$-based hybrid integrated diode laser, we decided
for a near UV target wavelength of 405 nm. In that range, suitable double pass
gallium nitride amplifiers, as well as equipment for laser output
characterization, was available. Scaling the measured UV and IR losses to the
target wavelength range predicts losses between 1 and 4 dB/cm for 400 nm thick
waveguides. Selecting a thinner waveguide would reduce propagation loss via
reduced sidewall scattering while maintaining tight guiding for UV
wavelengths.

Choosing a proper waveguide cross-section is of central importance to design
low-loss and fabrication tolerant waveguide components. This applies to curved
waveguides in small ring resonators for spectral filtering with wide free
spectral range, where the cross-section and ring radius determines the bending
loss and thus the filter $Q$-factor. Also the design of directional couplers
depends on the cross-section, as the coupling strength is influenced by the
coupler length and by the fabrication tolerance of the gap between the
waveguide cores. The cross-section to be selected also needs to restrict
propagation to a single transverse mode with a polarization matching that of
the amplifier ($\mathrm{TE_{00}}$), while efficient coupling to the diode
amplifier and output fibers should be enabled by modematching through inverse
tapering at the selected waveguide thickness. Using measured losses, scaling
and numerical simulations (see Methods) a waveguide cross section of a 400 nm
wide by 100 nm thick $\mathrm{Al_{2}O_{3}}$ core embedded in a
$\mathrm{SiO_{2}}$ cladding is chosen (see Fig. 1b).

The fabrication employs common CMOS compatible techniques on a wafer scale
(see Methods for details). With a mask design for a 10-cm wafer, 127 chips are
fabricated carrying an extensive range of test structures from waveguide
spirals (Fig. 1d) for loss measurements to Mach-Zehnder interferometers (MZI)
to test the heater functionality. Various Vernier feedback circuits as shown
in Fig. 1a are fabricated with variation in ring radii, tapers and coupling
coefficients between ring resonators and bus waveguides. Characterizing the
individual test structures and considering the measured loss at the nominal
wavelength allows us to fabricate and select suitable feedback circuits for
hybrid integration with amplifiers, while requiring only a single wafer run.

### Design of $\mathbf{Al_{2}O_{3}}$ feedback chip

Employing $\mathrm{Al_{2}O_{3}}$ waveguides for novel near UV laser feedback
circuits requires to adapt the design of crucial circuit functions. One
function is providing tunable optical filtering to the amplifier with the goal
to impose wavelength tunable laser output with high side-mode suppression.
Tuning reliably across the entire gain bandwidth is easily facilitated if the
free spectral range of the Vernier filter is wider than or equal to the gain
bandwidth. To obtain single-frequency output with high side mode suppression
the side peaks of the Vernier filter need to be sufficiently suppressed and
the main filter peak should be narrowband. A main condition always remains,
that the feedback of the $\mathrm{Al_{2}O_{3}}$ circuit at the main filter
peak is high enough for reaching laser threshold with the available small-
signal gain. Fulfilling all of these conditions requires a functional circuit
design where all circuit parameters are properly set. Such conditions are the
free spectral ranges of the micro-ring resonators used in the Vernier filter,
the power coupling coefficients of the ring resonators
($\mathrm{\kappa}^{2}$), the mode matching between the amplifier and feedback
chips, and the variable strength of the optical output coupling from the laser
resonator. The amplifier’s gain bandwidth determines the Vernier filter’s free
spectral range and the overall allowed feedback loss, while the amplifier’s
mode profile determines what minimum loss can be achieved in edge coupling and
hybrid integration of the chips. Measuring the propagation loss of fabricated
test structures determines the optimal coupling strength to the ring
resonators that would maximize the peak filter transmission.

The selected amplifier is an InGaN/GaN superluminescent diode (SLED) with a
center wavelength of 405 nm. Experimental data for the amplifiers are
available from the manufacturer (Exalos AG). To work as a double pass
amplifier, the diode is highly-reflective coated (vs. air $>$95%) on its back
facet and anti-reflective coated (vs. air $<$0.1%) on the facet facing the
$\mathrm{Al_{2}O_{3}}$ chip. When operated without feedback and driven with a
current of 78 mA the output is 10 mW of amplified spontaneous emission (ASE)
with a 3-dB emission bandwidth of approximately 3.4 nm. The manufacturer
specified mode field diameter is 1.87$\times$0.6 $\mathrm{\mu m}$.

For quantifying the propagation loss in the fabricated waveguides, we perform
single-pass transmission measurements , using a set of waveguide spirals with
weak curvature and different lengths (see Methods). The measurements yield a
propagation loss of $\mathrm{\alpha}$ = 2.8 $\pm$ 0.3 dB/cm at a wavelength of
405 nm for the cross-section depicted in Fig. 1b. The ASE emission bandwidth
was used to set the radii of the Vernier resonators to (radius
$\mathrm{R_{1}}$ = 150 $\mu$m and radius $\mathrm{R_{2}}$ = 153 $\mu$m), to
obtain a Vernier spectral range of 4.84 nm (see Fig. 1c). As a direct
transmission measurement for fabricated Vernier filters is difficult, we
measure instead the fabricated cross sections and tunnel gaps of the couplers
using a scanning electron microscope (SEM) and use the retrieved geometry with
index data to simulate the strength of couplers and Q-factors of ring
resonators. The Vernier filter that was selected for most of the laser
characterization experiments we obtain a power coupling of
$\mathrm{\kappa^{2}}$ = 5.9%. This Vernier filter would result in the
narrowest spectral filtering, while still providing sufficient feedback to the
amplifier to bring the laser above threshold. Using both parameters, $\alpha$
and $\kappa$, we conclude that both resonators are near-critically coupled
with intrinsic and loaded Q-factors of 427.000 and 143.000, respectively,
yielding a feedback filter resolution of approximately 3.3 GHz (full-width
half maximum of Vernier resonance). The transmission function of this Vernier-
filter is plotted in Fig. 1c. It can be seen that the individual resonators
with a free spectral range of about 97 pm open a Vernier free spectral range
that is much wider, about 4.84 nm. This ensures that the filter passes only a
single wavelength within the amplifier emission spectrum, qualitatively
represented by the dashed outline of a measured ASE spectrum in Fig. 1c. It
can also be seen that the next-adjacent Vernier filter side mode (VFSM, see
inset in 1c) is well suppressed (3.3 dB below main peak), to maximize the side
mode suppression during laser operation. For light passing resonantly through
the Vernier filter, we calculate an optical roundtrip length of the laser
cavity of 55.9 mm, which means the nearest cavity mode is at 5.36 GHz from the
central lasing mode.

For optimum coupling between modes in the InGaN amplifier chip and
$\mathrm{Al_{2}O_{3}}$ feedback chip, lateral inverse tapers are designed near
the facets of the feedback chip. Limited by the elliptical shape of the
amplifier output mode, the maximum theoretical coupling between both chip
modes is calculated to be 91%. Fresnel reflections from the interface of the
chips into the amplifier are suppressed by letting the waveguides form a
slight, index-matched angle with regard to the facet normal. For efficient
coupling to output fibers, inverse tapers were designed at the output facet of
the feedback chip. Other functionalities included in the design of the
feedback chips are an adjustable phase section (PS), phase shifters for fine-
tuning the optical length of the ring resonators ($\mathrm{R_{1}}$ and
$\mathrm{R_{2}}$), an adjustable Mach-Zehnder interferometer (MZI) outcoupler
(OC) formed by two 50% directional couplers, and a phase section behind the
outcoupler. Tuning of these circuit components is realized via the thermo-
optic effect by placing thin-film resistive microheaters on the chip (as
indicated in Fig. 1a). Tuning the phase section allows for spectrally aligning
a laser cavity mode with a resonance of the Vernier filter. Tuning of ring
resonators ($\mathrm{R_{1}}$ and $\mathrm{R_{2}}$) selects a particular
feedback wavelength. The main purpose of variable outcoupling is to enable
maximum output power at each drive current of the laser diode and heaters of
the Vernier filter. Varying the coupling is achieved by applying a current to
one of the heaters at the arms of the MZI in the outcoupler. Characterization
of test MZI structures showed that the outcoupling of the laser can be
adjusted between approximately 10 and 90%. To reduce ASE noise in the output,
the output is taken from the filtered light from the Vernier loop mirror that
is directed back to the laser amplifier. The phase section behind the
outcoupler may be used to prevent injection locking of the laser frequency via
unwanted reflections such as as from chip and fiber facets.

### Hybrid integration and packaging

Characterization of separate waveguide circuits can be carried out with
manually aligned stages, such as in waveguide loss measurements using free-
space fiber-to-chip coupling. Manual alignment of edge coupled feedback chips
for spectral narrowing via self-injection locking can reveal ultra-low, Hertz-
level intrinsic linewidths, however, vibration and drift of aligment stages
typically limits stability to time scales shorter than milliseconds [39]. In
contrast, long-term frequency and power stability, and inherent robustness to
external perturbations, require mutual bonding of chips, hermetic sealing and
temperature stabilization [30]. In infrared lasers, the laser cavity mode is
allowed to propagate through bonding materials, and hermetic sealing is not
essential. This is demonstrated with hybrid integration [40] and heterogeneous
integration [16]. However, at the much higher UV photon energies, bonding
materials need to be kept out of the laser mode and hermetic sealing is
instrumental for long-term stable output.

In total, three lasers were hybrid integrated and packaged with hermetic
sealing in a standard 14-pin butterfly housing with electrical wirebonding and
with feed-throughs for the output fibers and electrical signals (see Methods).
The package contains a thermistor and Peltier element for temperature control
of the laser. One packaged variant is sealed in a nitrogen atmosphere and
equipped with a glass lid. This allows for visual inspection of the circuit in
operation, such as seen in Fig. 1e, 1f and 2a. For the other variants,
improved hermetical sealing is realized by sealing the butterfly housing with
a seamwelded metal lid which results in a significantly better hermetic
sealing. On top of the improved sealing this variant uses an argon atmosphere,
which might give an extended lifetime over a nitrogen buffer gas [29].
Feedback chips with two different coupling strengths between bus waveguides
and and microring resonators were packaged ($\kappa^{2}$ = 5.9% and 2.9%).

Figure 2: a) Microscopic picture of the $\mathrm{Al_{2}O_{3}}$-based hybrid
laser in operation, showing scattered light from intracavity waveguides. Some
parts of the waveguides are less bright due to the metal heater layers
obscuring parts of the waveguide circuit. b) Fiber-coupled output power versus
drive current, showing a laser threshold and slope efficiency of 55 mA and
20.7 $\pm$ 1 $\mu$W/mA, respectively. The fiber-coupled output power reaches a
maximum of 0.74 $\pm$ 0.04 mW, correcting for fiber-to-chip coupling losses
(6.7 $\pm$ 2.1 dB, see Methods) this corresponds to about 3.5 mW ($\pm$ 50%)
on-chip power in the output waveguide. c) Optical spectrum showing single
wavelength operation with a sidemode suppression ratios (SMSR) of 43 dB
($\kappa^{2}$ = 5.9%) and 42 dB ($\kappa^{2}$ = 2.9%). d) Superimposed laser
spectra shown for various heater settings of ring $R_{1}$, here, the fiber-
coupled output power varies between 0.31 and 0.52 mW. The laser tunes in
wavelength more than 4.4 nm from 408.1 nm to 403.7 nm, shown by center
wavelength versus heating power plotted in the inset. Each tuning step is
followed by automated optimization for the output power. The slope of the
linear fit is -21.5 $\pm$ 0.5 pm/mW. e) High-resolution wavelength meter
recording the laser wavelength over time. Recording begins right after start-
up, when the laser is still far from thermal equilibrium. After an initial
drift over 1.5 GHz, the frequency becomes constant with small residual
fluctuations in the order of a few tens of MHz. No mode-hops are observed and
the drift remains well below the cavity FSR. We attribute the high stability
to hybrid integration of the chips and hermetic sealing. f) RF beat signal
from a delayed self-heterodyne set-up for our laser, measured at 405 nm
wavelength using a 7.9 m fiber delay (purple). The noise floor is recorded by
blocking the photodiode from any input (blue). From the fringes in the RF beat
signal we can extract a laser linewidth in the order of 25 MHz or lower.

### Laser characterization

When the InGaN-$\mathrm{Al_{2}O_{3}}$ laser is succesfully brought in
operation, the intracavity waveguides in the circuit light up brightly, see
Fig. 1f and 2a. To record the fiber-coupled output power, for each measurement
we make small readjustments of the heaters on the micro-resonators
($\mathrm{R_{1}}$, $\mathrm{R_{2}}$), the phase section (PS) and MZI-based
outcoupler (OC). This optimization process is recursive and usually requires a
run of one to three times for all parameters to find the maximum output power
while maintaining single wavelength operation with high sidemode suppression.
All heater adjustments can be carried out automatically via multichannel
computer control of the heaters. The laser temperature control, a PID control
loop using the thermistor and Peltier element in the package, is set to 20 °C
for all measurements.

The fiber-coupled output power is shown in Fig. 2b, we find a laser threshold
current of 55.0 mA, above which the output increases linearly with pump
current. The fiber-coupled, optical power reaches a value 0.74 $\pm$ 0.04 mW
for the maximum drive current that we apply (90.0 mA). Correcting for losses
at the chip-to-fiber coupling interface (estimated at 6.7 $\pm$ 2.1 dB, see
Methods), the laser generates about 3.5 mW ($\pm$ 50%) on-chip power in its
output waveguide. At maximum output power the monitored wavelength is 405.5
nm. We note that the fiber-coupled, near UV output levels are up to two orders
of magnitude higher than what has been reported recently with self-injection
of stage-coupled Fabry-Perot lasers to silicon nitride feedback chips [15].
Such large difference in output can have various reasons, however we expect
that the main contributors to high output power are permanent bonding and much
better mode matching between amplifier and feedback chip.

Wide-range near UV output spectra are recorded with an optical spectrum
analyzer (OSA). Typical emission spectra as in Fig. 2c display single-
wavelength operation with high sidemode suppression ratios (SMSR) of about 42
dB and 43 dB, which is two orders of magnitude more than previously reported
around this wavelength range [14, 15]. These recordings indicate a full width
at half maximum laser linewidth below the resolution limit of the optical
spectrum analyzer (about 90 GHz). We perform coarse tuning of the laser by
varying the heater power for one of the microring resonators
($\mathrm{R_{1}}$). Figure 2d shows superimposed spectra of the laser at 14
coarse steps along its entire tuning range, with the inset showing the
tunability of the center wavelength from 408.1 nm to 403.7 nm. The data shows
a wavelength coverage of 4.4 nm, not exceeding the designed free spectral
range of the Vernier (4.84 nm, Fig. 1c).

To verify the promise of long term passive stability inherent to hybrid
integration of lasers [41], we characterized the passive wavelength stability
of the laser. Next to residual thermal drift, an important question is whether
the laser would exhibit longitudinal mode hops. Such mode hops can impose
limitations in electronic frequency locking to reference cavities or to
absolute reference absorption lines. A mode hop changes the laser frequency by
at least one free spectral range of the laser cavity, which is about 5.36 GHz
for this laser. Immunity from environmental perturbations can thus be judged
by comparing the laser longer-term frequency drift with one cavity free
spectral range. To measure the laser stability the wavelength was recorded
over time using a high resolution wavelength meter. The result of such a
measurement, at an average sampling time of 12 ms, is shown in Fig. 2e. The
figure shows that the laser was mode-hop-free for at least 84 minutes, which
was also the entire duration of recording, and drifting less than 1.6 GHz. In
the final 24 minutes the laser has reached thermal equilibrium with the
environment, here a standard optical laboratory, and showed a drift of less
than 30 MHz. Recording longer time traces does not appear problematic,
however, the degradation times (inherent to all GaN-type diode lasers) are not
known and presently not subject to systematic investigations.

In order to determine the laser linewidth with higher resolution than 90 GHz
(from optical spectrum analyzer measurements), we use a delayed self-
heterodyne experiment with a 7.9 meter long optical fiber as delay line (see
Methods). A typical RF spectrum is displayed in Fig. 2f (purple trace). The
presence of fringes proves that the laser coherence time is approximately
similar to or longer than delay time provided by the fiber. However, the
fringe contrast in the line wings is strongly masked by the noise floor (blue
trace). This prevents an evaluation for Lorentzian line shape components [42]
and thus requires that the signal-to-noise ratio is increased in further
experiments. Nevertheless, the fringe period can be extracted reliably as 40
ns, which corresponds to the 7.9 m fiber delay. This gives a lower bound for
the coherence time, i.e., the full width of the laser spectrum is in the order
of 25 MHz or lower.

### Summary

In this work we have shown the design, fabrication and operation of the first
near UV hybrid integrated diode laser using the deep-UV capable
$\mathrm{Al_{2}O_{3}}$ material platform. The fabricated
$\mathrm{Al_{2}O_{3}}$ chips show a propagation loss of 2.8 $\pm$ 0.3 dB/cm.
For the first time a 405 nm InGaN amplifier was hybrid integrated with an
$\mathrm{Al_{2}O_{3}}$ waveguide circuit, ensuring a UV-compatible bonding
between both chips in a hermetically sealed environment. This approach
maximizes the frequency stability and durability of the device. The laser
shows a maximum output power of 0.74 mW, on-chip about 3.5 mW, and tunes more
than 4.4 nm from 408.1 nm to 403.7 nm. The laser shows high passive frequency
stability, operating without mode hops over more than an hour, settling to
residual frequency deviations of a few ten MHz per 20 minutes. Delayed self-
heterodyne measurement indicates that the full laser linewidth is in the order
of the inverse delay time (25 MHz) or below, presently a low signal-to-noise
ratio in heterodyne detection excludes a closer quantification. Continuing
this work we aim on improved linewidth measurements via higher signal-to-noise
detection. Our experiments show that the $\mathrm{Al_{2}O_{3}}$ platform newly
engaged here for near UV, complex and tunable feedback circuits opens the path
for chip integrated UV applications based on fully functional hybrid
integrated diode lasers.

## Methods

### Design and fabrication

Simulation of optical modes (2D finite element methods, Lumerical), sidewall
scattering integrals, and mode overlap integrals were performed to select the
core cross-section as shown in Fig. 1b. With this cross-section the expected
propagation loss from scaling is $\mathrm{<}$4 dB/cm and calculations show
tight bend radii down to 80 $\mu$m result in negligible bend radiation loss.
The named cross-section was used to design directional couplers with nominal
values $\mathrm{\kappa^{2}}$ of 1, 2, 5 and 10, 20 and 50%. Manufacturers
specifications on the the modefield diameter for the diode amplifier and UV
fibers were consulted to design linear inverse tapers.

The $\mathrm{Al_{2}O_{3}}$ waveguides, fabricated by the Integrated Optical
Systems group in the MESA+ cleanroom (University of Twente), starts by
depositing a 110 nm $\mathrm{Al_{2}O_{3}}$ layer using an optimized RF
reactive sputter deposition process [43], onto an 8 $\mathrm{\mu}$m thick
thermally oxidized 10 cm diameter silicon wafer. A chemical mechanical
polishing step is used to reduce the surface roughness of the deposited
$\mathrm{Al_{2}O_{3}}$ layer, reducing the layer thickness to the targeted 100
nm. Next, the substrate is coated with negative e-beam resist (AR-N 7520).
Using a Raith EBPG5150 e-beam lithography system the waveguide layer is
written in the resist. After the e-beam write, the pattern is developed using
AR-300-47 developer. The resulting patterns are etched into the
$\mathrm{Al_{2}O_{3}}$ using an Oxford PlasmaPro 100 Cobra (reactive ion
etching). Afterwards the resist is stripped by oxygen plasma using a TEPLA
300. The resulting waveguides are fully buried by an 8 $\mathrm{\mu}$m thick
$\mathrm{SiO_{2}}$ cladding. To implement thermo-optic tuning at various
locations on the feedback chips, resistive heaters are fabricated, by
deposition and structuring of a 10/10 nm Cr/Pt layer topped with a 300 nm Au
layer. All three layers are patterned using a lift-off process. To create the
heaters, the gold layer is etched away leaving only the two thin and high
electrically resistive Cr and Pt layers. For the $\mathrm{Al_{2}O_{3}}$ thin
film the material index was measured using an ellipsometer (Woollam M-2000UI)
in the wavelength range from 600 - 1600 nm. Cauchy’s equation,
$\mathrm{n(\lambda)=A+B/\lambda^{2}}$ with $\mathrm{\lambda}$ in
$\mathrm{\mu}$m, was fitted to the ellipsometer data with A = 1.6848 $\pm$
0.009 and B = 0.0119 $\pm$ 0.002. The function was extrapolated to obtain the
material index in the near UV spectral range.

### Characterization of $\mathbf{Al_{2}O_{3}}$ feedback chip

Several chips were fabricated with test structures to characterize individual
building blocks of the waveguide feedback circuit. First, to characterize the
straight propagation loss, several spirals and straight waveguides were
investigated. These structures used a minimum bend radius of 150 $\mathrm{\mu
m}$, which ensures negligible bend radiation loss while maintaining a small
footprint circuit. Using a 405 nm Fabry-Perot diode laser (QPhotonics) light
was fiber coupled (PM-S405-XP) into the chip, passing through a spiral or path
(Fig. 1d). At the output side of the chip the transmitted power was measured
(Thorlabs, S150C photodiode). The transmittance ($\mathrm{P_{out}/P_{in}}$) as
a function of the on-chip propagation length is shown in Fig. 3 and yields an
average propagation loss of 2.8 $\pm$ 0.3 dB/cm and a fiber-to-chip coupling
loss of 10.2 $\pm$ 0.8 dB/facet. To estimate the loss of the inverse tapers,
test structures were investigated where light propagates sequentially through
a number of adiabatic tapers lined up as a waveguide and showed a loss of the
inverse tapers below the measurement accuracy. To determine the phase tuning
of light using the fabricated heaters, another test structure with an MZI and
a thermal heater on each arm revealed that about 350 mW of electrical power is
needed for a phase shift of 2$\mathrm{\pi}$ in the guided light. Scanning
electron microscope (SEM) images allowed verification of the cross-section,
gaps at directional couplers and tapers on various parts of the photonic chip.
These cross-sectional images also verified that the waveguide sidewall is at
the designed right angle with the base (approximately 90$\degree$).

Figure 3: Transmittance ($P_{out}/P_{in}$) of 405 nm light through various
paths and spirals of different propagation length. The linear slope indicates
a propagation loss of 2.8 $\pm$ 0.3 dB/cm and from the y-axis crossing a 10.2
$\pm$ 0.8 dB fiber-to-chip coupling loss is obtained.

### Hybrid integration and packaging

The hybrid integration and packaging process was carried out in conjunction
with PHIX B.V. and with support from Lionix International B.V. First the
individual $\mathrm{Al_{2}O_{3}}$ chips were diced from the wafer and
polished. The right side of the feedback chip (Fig. 1d) was polished at an
8$\degree$ vertical angle to match the angle of the fiber array, minimizing
any back reflections into the laser mode from this interface. Also here, the
waveguide width is tapered down to maximize the mode matching with the large
and circular 3.3 $\mu$m diameter mode of the PM-S405-XP fiber, showing a
theoretical coupling efficiency of 59% limited by the elliptical modeshape in
the waveguide. Here, a fiber array with five of these fibers was used. For the
active temperature control a 10 k$\ohm$ thermistor was added to the SLED
submount.

After preparing both chips, the SLED is aligned and butt-coupled to the
$\mathrm{Al_{2}O_{3}}$ feedback chip. When optimum alignment is reached, the
chips are hybrid integrated by bonding using a UV curable epoxy. The epoxy is
applied such that the near UV optical mode remains free from epoxy. On the
output side of the feedback chip the fiber array is aligned for maximum
coupling and bonded using an epoxy, again ensuring a free optical path between
the feedback chip and fibers. Subsequently a Peltier element is mounted to the
bottom of a shared substrate, to be used with the thermistor for the thermal
control of the laser. Afterwards the full assembly is placed in a standard
14-pin butterfly package. The electrical connections for the cathode and anode
of the SLED and heaters on the $\mathrm{Al_{2}O_{3}}$ feedback chip are
wirebonded to the pins of the butterfly package. The Peltier connectors are
soldered to the remaining two pins of the package.

The final step is hermetic sealing of the butterfly package. For each package
a getter material is attached to the inside of the lid before hermetically
sealing the package. The getter material absorbs any residual gas (from
outgassing of the epoxy, like volatile organic compounds and moisture after
sealing). The laser in the pictures (Fig. 1e, 1f, 2a) is a package with a
glass lid in a nitrogen environment ($\mathrm{\kappa^{2}}$ = 5.9%). The
measurements shown in this work are carried out with a seamwelded, argon
atmosphere laser ($\mathrm{\kappa^{2}}$ = 5.9%), except the measurement for
one of the optical spectra shown in Fig. 2c (seamwelded, argon atmosphere,
$\mathrm{\kappa^{2}}$ = 2.9%).

### Experimental methods

Waveguide heaters are driven with a high-precision, low-noise, multichannel
power supply (Chilas B.V., Tunable Laser Controller, TLC). The TLC also
contains a PID control loop which controls the temperature of the laser with
the thermistor and Peltier element in the butterfly housing. The TLC is
equipped with a USB interface to receive serial commands from a PC. Together
with in-house software, photodiodes and other lab equipment, optimization and
control of the laser can be fully automated. Depending on availability several
current sources were used: Toptica DLC Pro, ILX Lightwave LDX-3620a and
Thorlabs LDC205B.

Output powers are measured by connecting the output fiber to a calibrated
photodiode (Thorlabs, S150C photodiode). The calibration accuracy of the
photodiode is 5% in this spectral range. A 405-nm 90/10 custom fiber splitter
(Thorlabs with S405-XP fiber) is used for simultaneous monitoring of spectral
properties and output power. To find the conversion for the fiber-coupled
output power to the on-chip power, we first note the measured fiber-to-chip
coupling loss, which is 10.2 $\pm$ 0.8 dB (Fig. 3). This value is measured
with an unpolished chip facet, whereas the integrated laser facets are finely
polished which should reduce the fiber-to-chip coupling loss by an estimated
3.5 $\pm$ 1.1 dB [44]. Finally, the 5% ($\sim$ 0.2 dB) measurement error for
the photodiode can be included in this conversion error. These estimates bring
the total conversion factor from fiber-coupled output to on-chip power to +6.7
$\pm$ 2.1 dB. Therefore, we find an on-chip power of about 3.5 mW ($\pm$ 50%).

The measurements for the spectra, wavelength stability and spectral linewidth
are all recorded at the maximum current of 90 mA. Laser output spectra are
recorded with an optical spectrum analyzer (Ando AQ6315A, approximately 50 pm
or 90 GHz resolution at 405 nm). To measure the optical spectra the 90/10
fiber splitter is used to simultaneously monitor the output power of the laser
with a photodiode, which enables to use our optimization software for
controlling the laser. The nominally 90% arm of the fiber splitter is
connected to the spectrum analyzer and the spectra are corrected (due to the
splitting and loss in fiber splitter) in post-processing for the photodiode
measured, fiber-coupled output power of the laser. Long-term measurements of
passive laser frequency stability rely on a high-resolution wavelength meter
(HighFinesse WS-U 1645, wavelength deviation sensitivity of 0.5 MHz ).

Delayed self-heterodyne measurements, using existing methods [42], are carried
out with a fiber delay length of 7.9 m (S405-XP fiber), an acousto-optic
modulator (at 200 MHz, G$\mathrm{\&}$H FiberQ) and a balanced photodiode
(Thorlabs PD435A-AC), with signals being recorded with an electrical spectrum
analyzer (Keysight CXA N9000B). To match the polarization of the signal from
both arms of the set-up, a fiber polarization controller is used (Thorlabs
FPC560 with an S405-XP fiber). The recorded RF spectrum of the beatnote is
shown as the purple trace in Fig. 2f. The measurement time of an RF frequency
sweep is 67 ms and the raw data is averaged 20 times. The fringe period is
extracted from the RF signal as 40 ns, which corresponds well with a 7.9 m
fiber delay length. The noise floor is recorded when the light to the sensor
is blocked (blue trace, Fig. 2f).

## References

  * [1] Galan Moody et al.  “2022 Roadmap on integrated quantum photonics”  In _Journal of Physics: Photonics_ 4, 2022  DOI: 10.1088/2515-7647/ac1ef4
  * [2] Karan K. Mehta et al.  “Integrated optical multi-ion quantum logic”  In _Nature_ 586, 2020, pp. 533–537  DOI: 10.1038/s41586-020-2823-6
  * [3] R.. Niffenegger et al.  “Integrated multi-wavelength control of an ion qubit”  In _Nature_ 586 Springer US, 2020, pp. 538–542  DOI: 10.1038/s41586-020-2811-x
  * [4] A. Zybin et al.  “Diode laser atomic absorption spectrometry”  In _Spectrochimica Acta - Part B Atomic Spectroscopy_ 60, 2005, pp. 1–11  DOI: 10.1016/j.sab.2004.10.001
  * [5] Pengcheng Liu et al.  “ Tunable microwave frequency comb generation using a $\mathrm{Si_{3}N_{4}}$-MDR based actively mode-locked OEO ”  In _Optics Express_ 30, 2022, pp. 25380  DOI: 10.1364/oe.463312
  * [6] Chupao Lin et al.  “UV photonic integrated circuits for far-field structured illumination autofluorescence microscopy”  In _Nature Communications_ , 2022, pp. article number: 4360  DOI: 10.1038/s41467-022-31989-8
  * [7] Andrew D. Ludlow et al.  “Optical atomic clocks”  In _Reviews of Modern Physics_ 87, 2015, pp. 637–701  DOI: 10.1103/RevModPhys.87.637
  * [8] Chris G.H. Roeloffzen et al.  “Low-loss $\mathrm{Si_{3}N_{4}}$ TriPleX optical waveguides: Technology and applications overview”  In _IEEE Journal of Selected Topics in Quantum Electronics_ 24.4, 2018  DOI: 10.1109/JSTQE.2018.2793945
  * [9] Daniel J. Blumenthal et al.  “Silicon Nitride in Silicon Photonics”  In _Proceedings of the IEEE_ 106, 2018, pp. 2209–2231  DOI: 10.1109/JPROC.2018.2861576
  * [10] Clemens J. Krückel, Attila Fülöp, Peter A. Andrekson and Víctor Torres-Company  “Bandgap engineering in nonlinear silicon nitride waveguides”  Optical Fiber Communications ConferenceExhibition, 2017, pp. 770–776
  * [11] Cornelis A.. Franken et al.  “Hybrid-integrated diode laser in the visible spectral range”  In _Optics Letters_ 46, 2021, pp. 4904  DOI: 10.1364/OL.433636
  * [12] Lisa V. Winkler et al.  “Silicon nitride hybrid-integrated diode laser at 637 nm”  SPIE Photonics West, 2023, pp. Integrated Optics: DevicesMaterialsand Technologies XXVII
  * [13] Thomas Wunderer et al.  “Single-Frequency Violet and Blue Laser Emission from AlGaInN Photonic Integrated Circuit Chips”  In _Optica Open, Preprint_ , 2023, pp. 1–4  DOI: 10.1364/opticaopen.21984815.v1
  * [14] Anat Siddharth et al.  “Near ultraviolet photonic integrated lasers based on silicon nitride”  In _APL Photonics_ 7 AIP Publishing, LLC, 2022, pp. article number: 046108  DOI: 10.1063/5.0081660
  * [15] Mateus Corato-Zanarella et al.  “Widely tunable and narrow-linewidth chip-scale lasers from near-ultraviolet to near-infrared wavelengths”  In _Nature Photonics_ 17 Springer US, 2022, pp. 157–164  DOI: 10.1038/s41566-022-01120-w
  * [16] Minh A. Tran et al.  “Extending the spectrum of fully integrated photonics to submicrometre wavelengths”  In _Nature_ 610 Springer US, 2022, pp. 54–60  DOI: 10.1038/s41586-022-05119-9
  * [17] Dong Liu et al.  “226 nm AlGaN/AlN UV LEDs using p-type Si for hole injection and UV reflection”  In _Applied Physics Letters_ 113, 2018  DOI: 10.1063/1.5038044
  * [18] Cheryl Sorace-Agaskar et al.  “Versatile Silicon Nitride and Alumina Integrated Photonic Platforms for the Ultraviolet to Short-Wave Infrared”  In _IEEE Journal of Selected Topics in Quantum Electronics_ 25.5, 2019, pp. 1–15  DOI: 10.1109/JSTQE.2019.2904443
  * [19] Theodore J Morin et al.  “CMOS-foundry-based blue and violet photonics”  In _Optica_ 8, 2021, pp. 755–756  DOI: 10.1364/OPTICA.426065
  * [20] Marco A G Porcel et al.  “Photo-induced second-order nonlinearity in stoichiometric silicon nitride waveguides”  In _Optics Express_ 25, 2017, pp. 33143–33159  DOI: 10.1364/OE.25.033143
  * [21] Adrien Billat et al.  “Large second harmonic generation enhancement in $\mathrm{Si_{3}N_{4}}$ waveguides by all-optically induced quasi-phase-matching”  In _Nature Communications_ 8, 2017, pp. article number: 1016  DOI: 10.1038/s41467-017-01110-5
  * [22] Pieter Neutens et al.  “Mitigation of UV-Induced Propagation Loss in PECVD Silicon Nitride Photonic Waveguides”  In _ACS Photonics_ 5, 2018, pp. 2145–2150  DOI: 10.1021/acsphotonics.8b00014
  * [23] Gavin N. West et al.  “Low-loss integrated photonics for the blue and ultraviolet regime”  In _APL Photonics_ 4, 2019  DOI: 10.1063/1.5052502
  * [24] Tsung-Ju Lu et al.  “Aluminum nitride integrated photonics platform for the ultraviolet to visible spectrum”  In _Optics Express_ 26, 2018, pp. 11147  DOI: 10.1364/OE.26.011147
  * [25] Ward Hendriks et al.  “Poly-crystalline low-loss aluminium oxide waveguides”  In _Integrated Optics: Devices, Materials, and Technologies XXV_ SPIE, 2021, pp. 32  DOI: 10.1117/12.2578141
  * [26] Ward A… Hendriks et al.  “Rare-earth ion doped $\mathrm{Al_{2}O_{3}}$ for active integrated photonics”  In _Advances in Physics: X_ 6 Taylor & Francis, 2021  DOI: 10.1080/23746149.2020.1833753
  * [27] Dawson B. Bonneville, Henry C. Frankis, Renjie Wang and Jonathan D.. Bradley  “Erbium-ytterbium co-doped aluminium oxide waveguide amplifiers fabricated by reactive co-sputtering and wet chemical etching”  In _Optics Express_ 28.20, 2020, pp. 30130–30140  DOI: 10.1364/oe.402802
  * [28] A. Uddin, A.. Wei and T.. Andersson  “Study of degradation mechanism of blue light emitting diodes”  In _Thin Solid Films_ 483 Elsevier, 2005, pp. 378–381  DOI: 10.1016/j.tsf.2005.01.018
  * [29] Thomas Schoedl et al.  “Facet degradation of GaN heterostructure laser diodes”  In _Journal of Applied Physics_ 97, 2005  DOI: 10.1063/1.1929851
  * [30] Luigi Ranno et al.  “Integrated Photonics Packaging: Challenges and Opportunities”  In _ACS Photonics_ , 2022  DOI: 10.1021/acsphotonics.2c00891
  * [31] Mingyuan Xue and Juanning Zhao  “Laser linewidth measurement based on long and short delay fiber combination”  In _Optics Express_ 29, 2021, pp. 27118–27126  DOI: 10.1364/oe.428787
  * [32] Di Liang, Gunther Roelkens, Roel Baets and John E. Bowers  “Hybrid integrated platforms for silicon photonics”  In _Materials_ 3, 2010, pp. 1782–1802  DOI: 10.3390/ma3031782
  * [33] Zhechao Wang et al.  “Novel Light Source Integration Approaches for Silicon Photonics”  In _Laser and Photonics Reviews_ 11, 2017, pp. 1–21  DOI: 10.1002/lpor.201700063
  * [34] Yi Sun et al.  “Room-temperature continuous-wave electrically injected InGaN-based laser directly grown on Si”  In _Nature Photonics_ 10, 2016, pp. 595–599  DOI: 10.1038/nphoton.2016.158
  * [35] Cornelis A.. Franken et al.  “First near-UV hybrid integrated laser in the $\mathrm{Al_{2}O_{3}}$ platform”  SPIE Photonics West, 2023, pp. Integrated Optics: DevicesMaterialsand Technologies XXVII
  * [36] Larry A. Coldren, Scott W. Corzine and Milan L. Mašanović  “Diode Lasers and Photonic Integrated Circuits”  John Wiley & Sons, Inc., 2012
  * [37] Z.. Weinberg, G.. Rubloff and E. Bassous  “Transmission, photoconductivity, and the experimental band gap of thermally grown $\mathrm{SiO_{2}}$ films”  In _Physical Review B_ 19, 1979, pp. 3107–3117  DOI: 10.1103/PhysRevB.19.3107
  * [38] S. Mardani et al.  “Low-loss chemical mechanically polished $\mathrm{Al_{2}O_{3}}$ thin films for UV integrated photonics”  23rd European Conference on Integrated Optics, 2022, pp. 387–389
  * [39] Warren Jin et al.  “Hertz-linewidth semiconductor lasers using CMOS-ready ultra-high-Q microresonators”  In _Nature Photonics_ 15, 2021, pp. 346–353  DOI: 10.1038/s41566-021-00761-7
  * [40] Albert Rees et al.  “Ring resonator enhanced mode-hop-free wavelength tuning of an integrated extended-cavity laser”  In _Optics Express_ 28, 2020, pp. 5669  DOI: 10.1364/OE.386356
  * [41] Jörn P. Epping et al.  “Hybrid integrated silicon nitride lasers”  Proc. SPIE 11274, 2020, pp. Physics and Simulation of Optoelectronic Devices XXVIII  DOI: 10.1117/12.2552264
  * [42] L.. Richter, H.. Mandelberg, M.. Kruger and P.. McGrath  “Linewidth Determination From Self-Heterodyne Measurements With Subcoherence Delay Times.”  In _IEEE Journal of Quantum Electronics_ 22, 1986, pp. 2070–2074  DOI: 10.1109/jqe.1986.1072909
  * [43] Carlijn I. Emmerik et al.  “ Relative oxidation state of the target as guideline for depositing optical quality RF reactive magnetron sputtered $\mathrm{Al_{2}O_{3}}$ layers ”  In _Optical Materials Express_ 10, 2020, pp. 1451–1462  DOI: 10.1364/ome.393058
  * [44] Stéphanie Jétte-Charbonneau et al.  “End-facet polishing of surface plasmon waveguides in lithium niobate”  In _IEEE Transactions on Advanced Packaging_ 31, 2008, pp. 479–483  DOI: 10.1109/TADVP.2008.924236


# A Large-Scale Comparative Study of Accurate COVID-19 Information versus
Misinformation

Yida Mu,1 Ye Jiang, 2 Freddy Heppell, 1 Iknoor Singh, 1 Carolina Scarton, 1
Kalina Bontcheva, 1 Xingyi Song 1

###### Abstract

The COVID-19 pandemic led to an infodemic where an overwhelming amount of
COVID-19 related content was being disseminated at high velocity through
social media. This made it challenging for citizens to differentiate between
accurate and inaccurate information about COVID-19. This motivated us to carry
out a comparative study of the characteristics of COVID-19 misinformation
versus those of accurate COVID-19 information through a large-scale
computational analysis of over 242 million tweets. The study makes comparisons
alongside four key aspects: 1) the distribution of topics, 2) the live status
of tweets, 3) language analysis and 4) the spreading power over time. An added
contribution of this study is the creation of a COVID-19 misinformation
classification dataset. Finally, we demonstrate that this new dataset helps
improve misinformation classification by more than 9% based on average F1
measure.

## Introduction

The COVID-19 pandemic was the first pandemic where social networks and mobile
devices played a major role as sources of information in a rapidly evolving,
dynamic context. Due to the large number of users and posts made daily on
social platforms, a COVID-19 infodemic was created (WHO 2020; WHO et al. 2020)
where huge volumes of content (both accurate and mis/disinformation) were
being published online daily. This made it challenging for citizens to find
reliable information, as many fell victim to misinformation and turned to
false treatments (Mehrpour and Sadeghi 2020; Caceres et al. 2022; Zhao et al.
2023) instead, or even started attacking medical workers (Orellana 2023; van
Stekelenburg et al. 2023).

In order to further our understanding of the infodemic, this paper undertakes
a large-scale study of the statistical characteristics of accurate COVID-19
information compared to COVID-19 misinformation on Twitter. The paper aims to
answer the following research questions:

  * •

Q1: What are the differences in topics and languages between accurate
information and misinformation?

  * •

Q2: What types of misinformation have social media platforms addressed?

  * •

Q3: What is the spreading power of the different types of misinformation?

To help address these research questions, we developed an evidence-based
COVID-19 misinformation classifier (Jiang et al. 2021) based on a newly
developed training set. Our experiments show that this new training data
resulted in a significantly improved model that outperforms a state-of-the-art
baseline model (Jiang et al. 2021) by almost 0.2 in F1 measure score (0.51 vs
0.70) under leave-claim-out cross-validation111This evaluation method splits
the training and testing sets based on the claim/topic, which is a realistic
testing approach for evaluating the model’s performance on unseen
misinformation (Jiang et al. 2021).

Next, we collected over 240 million COVID-19 related tweets and applied our
best performing classifier to identify the misinformation posts automatically.
This then enabled us to analyse the differences between misinformation and
accurate posts. The statistical analysis makes comparisons alongside five
dimensions: i) topical distribution, ii) spreading power, iii) deletion rate,
iv) Bag-of-Words, and v) Linguistic Inquiry and Word Count Analysis.

## Related Work

In this section we categorise COVID-19 misinformation statistical
characteristic studies into three groups. The first category is the general
COVID-19 information characteristic study, which investigates the
characteristics of both non-mis- and mis-information but doesn’t split the
results along these categories. The second category is the COVID-19
misinformation characteristic study, where machine learning, external
knowledge, or manual annotation is used to identify COVID-19 misinformation,
and the statistical analysis is based only on misinformation. The third
category is the COVID-19 non-mis- and mis-information statistical comparison
study, which is the primary focus of this paper. This category investigates
the statistical characteristics of both non-mis- and mis-information
simultaneously, enabling a direct comparison between them.

### General COVID-19 information analysis

Singh et al. (2020) conducted a study on COVID-19 related tweets from January
16 to March 15, 2020. The study reports the volume of tweets over various
dimensions, including time, language, and geolocation. Additionally, the study
investigates the statistics of word frequency, themes, popular myths, and
URLs. Although Singh et al. did not focus specifically on misinformation
tweets, the study reports statistics on tweets containing high-quality health
sources and low-quality misinformation sources, based on
NewsGuard222https://www.newsguardtech.com/. Previous studies (Abdul-Mageed et
al. 2020; Photiou, Nicolaides, and Dhillon 2021; Iwendi et al. 2022; Waheeb,
Khan, and Shang 2022) have performed sentiment analysis (Medford et al. 2020;
Dashtian and Murthy 2021; Chen et al. 2020b; Lamsal 2021; Gupta, Vishwanath,
and Yang 2021; Shi et al. 2020), hashtag analysis (Lamsal 2021; Chen et al.
2020a; Suarez-Lledo et al. 2022), engagement (Cinelli et al. 2020), and
clustering analysis using topic modelling (Dashtian and Murthy 2021; Gupta,
Vishwanath, and Yang 2021; John and Keikhosrokiani 2022) or pre-trained
embeddings (Cinelli et al. 2020; Biradar, Saumya, and Chauhan 2022) on general
COVID-19 related posts without specifically focusing on COVID-19
misinformation.

### COVID-19 misinformation analysis

The statistical analyses on COVID-19 misinformation encompass sentiment
(Sharma et al. 2020; Cheng et al. 2021), geolocation (Sharma et al. 2020),
topics modelling (Sharma et al. 2020; Shi et al. 2020), sources (Sharma et al.
2020), user analysis (Jain and Sethi 2021), engagement level (Jain and Sethi
2021), and hashtag and word frequency (Sharma et al. 2020; Dharawat et al.
2022; Jain and Sethi 2021; Cheng et al. 2021) have also been conducted in
previous studies. In addition, Brennen et al. (2020) conducted a
misinformation topic analysis that can be applied to prioritise the fact-check
resources to debunk the most harmful kind of misinformation topics. Brennen et
al. defined nine different types of misinformation topics and found that the
most spread misinformation is related to public authority actions in the early
stages of the COVID-19 pandemic. Song et al. (2021) extend this study on a
larger scale, and report the topic evolution through different time periods.
In this paper, we continue the topic study (Brennen et al. 2020; Song et al.
2021) but in different dimensions which include its spread, live tweet status
and sentiment analysis.

Next, we discuss some of the approaches used for identifying misinformation
from abundant COVID-19 related information. Some of the previous methods
include, 1) Manual annotation (Jiang et al. 2021; Brennen et al. 2020): the
most reliable but also the most expensive approach. Therefore, analysis based
on this approach is often limited to a small scale. 2) Credibility based
classification (Sharma et al. 2020; Cheng et al. 2021): which identifies
misinformation based on the credibility of the author and/or the source (e.g.
shared URLs). Compared to manual annotation, the credibility approach is a
much cheaper alternative to adapt misinformation analysis to a larger scale;
however, the major drawback of this approach is identifying misinformation
indirectly. Jiang et al. (2021) notice that misinformation is often shared
from highly credible sources. 3) Style based machine learning classification
(Cheng et al. 2021; Abdelminaam et al. 2021; Kar et al. 2020): style based
classifiers identify misinformation based on the writing styles of the text.
This approach is effective and often performs well with the misinformation
(theme/topics) already covered in the training data. On the other hand, the
style-based classifier is less effective in identifying misinformation that is
not covered in the training data (Jiang et al. 2021). 4) Evidence-based
machine learning classification (Jiang et al. 2021; Hossain et al. 2020a; Pan
et al. 2018; Hu et al. 2021): in comparison with the style based
classification solely relying on the input text, evidence-based classification
also requires external knowledge as evidence to aid the misinformation
classification. The external knowledge could be a professional debunked
misinformation database (Jiang et al. 2021; Hossain et al. 2020a) or knowledge
graph built from true (Pan et al. 2018; Hu et al. 2021) or misinformation (Pan
et al. 2018). This approach often provides more reliable misinformation
detection results and the evidence is also provided with the results. In this
paper, we also apply an evidence-based classification approach for
misinformation identification.

### Misinformation Comparision Analysis

Cui and Lee (2020) analysed sentiment and hashtag frequency of mis- and non-
mis- information and find that COVID-19 misinformation leans more towards
negative sentiment compared to non-misinformation, and also hashtag
distributions are significantly different. Memon and Carley (2020) studied the
Twitter user’s network density, bot ratio and sociolinguistics in the post and
demonstrate that misinformed users have a higher bot ratio and a denser
network (i.e. accordance opinions are more easily accepted and it is also more
difficult to have opposing opinions accepted in a denser network). The study
also finds some sociolinguistic differences between mis and non-mis-
information, but the results are not significant and are inconclusive.
Micallef et al. (2020) conducted a case study to investigate the spread of
misinformation and debunks related to some pre-defined COVID-19 topics (e.g.,
5G conspiracy theories), which may lead to hesitation towards COVID-19
vaccination (Mu et al. 2023). Burel et al. (2020) did a post-hoc causality
analysis to study the spread of COVID-19 misinformation and fact-checks on
Twitter, while Recuero et al. (2022) used Crowdtangle to study the spread of
COVID-19 disinformation and fact-checks on Facebook. Silva et al. (2020) and
Kouzy et al. (2020) both study the engagement level between mis and non-mis-
information, however, the conclusions from these two studies are different.
Kouzy et al. (2020) observed that there is no difference in the engagement
level between the two groups, but Silva et al. (2020) notice that
misinformation often has a higher engagement level. The difference is possibly
due to the different data collection methods.

## Data and Misinformation Classification

The data in this work is collected from Twitter via Twitter Stream API333
https://developer.twitter.com/en/docs/twitter-api using keywords associated
with COVID-19 (e.g. covid and covid-19, the complete list is in Table 5). The
data was collected from March 2020 to July 2021, and we collected a total of
242,823,999 tweets. However, we only considered the source tweets for analysis
and therefore excluded retweets and replies, resulting in 6,691,181 remaining
source tweets that were used in this study.

Our approach for detecting misinformation is based on evidence-based
classification, adapted from Jiang et al. (2021). We used a coarse-grained
BERT pairwise classification model that takes two inputs: 1) a verified
misinformation claim from a professional fact-checker and 2) the tweet text to
be classified. The classifier predicts whether the tweet text belongs to one
of three classes: a) Misinformation, b) Debunk, or c) Irrelevant. We have
enhanced the original model by adding two additional steps: 1) Data enrichment
and 2) Results filtering to improve classification accuracy. We describe these
steps in detail in Section ‘Data Enrichment’ and Section ‘Misinformation
Detection’.

### Data Enrichment

In addition to the training data outlined in Jiang et al. (2021), we expanded
our dataset by collecting training data from three additional sources: 1) the
CovidLies (Hossain et al. 2020b) dataset, 2) our own collection of tweets with
external knowledge, and 3) the IFCN Poynter website. The process of enriching
the dataset with data from these sources will be discussed in this section.

The CovidLies dataset provides a manually annotated evidence-based dataset for
misinformation classification. We converted its label set directly into the
labels used in Jiang et al. (2021): Misinformation, Debunk, and Irrelevant.
However, many tweets were deleted by the time we reconstructed the CovidLies
data using Twitter API, leaving us with only 87 Misinformation, 87 Debunk, and
3,142 Irrelevant tweets.

For our in-house tweet collection of 6.6 million tweets, we utilised external
knowledge to create a ‘silver standard’ training dataset. First, we classified
all the source tweets using the original (Jiang et al. 2021) trained
classifier and selected all Misinformation and Debunk tweets based on the
classifier output label as candidate tweets for the next step. Then, we
applied a confidence score filter to the candidates to further select them,
with a threshold of 0.7.

In the third step, we used Twitter’s COVID-19 misleading information policy to
identify deleted tweets likely to be Misinformation. Any candidate
Misinformation tweet that had been removed from Twitter was selected as a
final Misinformation sample.

For Debunk samples, we noticed that debunking tweets often reference articles
published on professional fact-checker websites. We utilised the IFCN Poynter
debunking data as a knowledge source to select any remaining Debunk candidate
tweets containing URLs linked to the IFCN Poynter debunks as enriched Debunk
training samples.

Finally, we selected Irrelevant training samples to correct false positives
caused by the classifier. These samples were extracted from the Misinformation
candidates but posted from highly credible accounts, such as WHO. The full
list of credible accounts is provided in Table 6.

Figure 1: (a): Screenshot of an IFCN debunk post. The post includes 1) fact
checking organisation, 2) misinformation claim, 3) explanation of why the
claim is false and 4) the link to full debunk article. (b): Partial screenshot
of full debunk article of the IFCN debunk post.

The IFCN Poynter website provides a valuable source for enriching our training
samples. As shown in Figure 1, each debunk on the website includes a
misinformation claim, an explanation of why it is false, and a link to the
full debunk article. For example:

To extract training samples from this source, we collect all the explanations
and use them as Debunk samples. For Misinformation samples, we need to access
the full article of the debunk, as shown in Figure 1. The original source
misinformation claims are often referenced in the article (highlighted in blue
in Figure 1). We use regular expressions to extract this text and label it as
Misinformation in our enriched dataset.

DataSet | IRRELEVANT | DEBUNK | MISINFO
---|---|---|---
Jiang et al. | 1066 | 194 | 522
CovidLies | 3142 | 87 | 123
Tweet Collect | 5757 | 3692 | 670
IFCN | 0 | 1892 | 96
Total | 9965 | 5865 | 1411
Table 1: The statistic of enriched COVID-19 misinformation classification
training dataset.

#### Enrichment Data Cleaning

Since the classifier used in this study is trained on pairwise input (claim
and tweet text pair), tweet text can only be accurately labelled as
Misinformation if it matches the paired claim. However, in the data enrichment
process using in-house and IFCN data, there is no guarantee that the pairing
process will be accurate. To improve the quality of the enriched data and
reduce the number of mismatches between claim and text pairs, a pair cleaning
process is conducted. Specifically, the Stanford OpenIE SVO parser (Angeli,
Premkumar, and Manning 2015) is used to extract the Subject and Object in both
the claim and the tweet text. Samples are discarded if neither the claim
Subject nor the claim Object is mentioned in the tweet text. Table 1 shows the
statistics of training samples after the cleaning process.

#### Model Training and Performance

The training process of the model is based on Jiang et al. (2021), and we will
briefly explain the key settings in this section. For more detailed training
settings, please refer to the paper.

The classification model is a fine tuned COVID-Twitter Pre-Trained (Müller,
Salathé, and Kummervold 2020) BERT Large (Devlin et al. 2019). It contains 24
transformer layers, and in our experiment, only the parameters in the last
transformer encoding layer are unlocked for fine-tuning, while the remaining
BERT weights remain frozen. Similar to the BERT next sentence prediction task,
the input to the model is constructed as ‘[CLS] + Claim + [SEP] + Tweet_Text +
[SEP]’, and the Softmax classifier predicts the probability of labels based on
the pairwise [CLS] representation.

We conduct five-folds ‘Leave-Claim-Out Cross Validation’ described in Jiang et
al. (2021) to compare enrichment performance. Note that the five-folds are
only conducted on the original Jiang et al. (2021) annotated dataset. The
enriched data is only added as supplemental training data, therefore, the
total number of test samples is the same as in the experiment described in
Jiang et al. (2021). The purpose of Leave-Claim-Out Cross Validation is to
test the classification performance for unseen misinformation. To ensure a
fair comparison with the non-enriched model, we manually removed claims from
the enriched dataset that were already present in the original annotated
dataset in Jiang et al. (2021), so that the test claims were unseen during
training.

| Non-enriched | Enriched
---|---|---
Accuracy | 0.64 | 0.70
Avg. F1 | 0.57 | 0.66
Debunk F1 | 0.47 | 0.56
IRRELEVANT F1 | 0.72 | 0.72
Misinformation F1 | 0.51 | 0.70
Table 2: The macro average of five fold Leave-Claim-Out Cross Validation
results. The number of Non-enriched Classifier is directly borrowed from Jiang
et al. (2021).

Compared to the non-enriched classifier, our enriched classifier yields an
overall F1 measure of 0.66, which is more than 9% higher than the non-enriched
classifier’s overall F1 measure of 0.57. Furthermore, the F1 measure of the
Misinformation class is significantly increased from 0.51 to 0.70 by our
enriched classifier.

### Misinformation Detection

In the misinformation detection process, we used 12,748 claims collected from
the IFCN Poynter website as input for our pairwise classifier. The claim
collection process followed the same procedure as in previous studies (Song et
al. 2021; Jiang et al. 2021).

However, since our classifier requires pairwise inputs, classifying the 6
million source tweets against the 12,748 IFCN claims would result in 72
billion pairs, which is computationally expensive. To reduce the computational
cost and speed up the classification process, we conducted an additional
candidate pair selection step. Here’s an overview of our misinformation
extraction procedure:

  1. 1.

We index the 6 million source tweets using Elastic Search.

  2. 2.

The 12,748 IFCN claims are used as queries to search for relevant tweets in
the index. We use the BM25 ranking algorithm (Robertson et al. 1995) to rank
the tweets and select the top 20,000 tweets for each query based on their BM25
score.

  3. 3.

After retrieving the top 20,000 tweets using BM25, we conduct reranking based
on the cosine similarity between tinyBERT embeddings of the queries and
tweets. This allows us to select the top 1,000 tweets that are semantically
similar to the IFCN claim, which serves as candidate pairs for the
classification process.

  4. 4.

Misinformation may also be time-sensitive. The claim may become true or false
in different time periods. For example, in early March 2020, there was
misinformation claiming that Trump was infected with
COVID-19444https://www.poynter.org/?ifcn˙misinformation=trump-faints-is-
infected-with-coronavirus, which remained misinformation until October 2020
when the White House COVID-19 outbreak meant Donald Trump actually tested
positive for
COVID-19555https://www.forbes.com/sites/elanagross/2020/10/02/white-house-
outbreak-here-are-the-people-who-have-tested-positive-for-covid-near-
president-trump/?sh=43ab1d76799a. Therefore, we only consider tweets that were
posted within a date range of ten weeks before and two weeks after the IFCN
claim debunk date. Candidate tweets that fall outside of this period will be
filtered out.

  5. 5.

The misinformation classifier (described in Section ‘Model Training and
Performance’) is applied to the remaining candidates. A post-process step is
conducted to ensure the precision of the misinformation extraction. The final
set of misinformation tweets is selected if the classifier predicted label is
‘misinformation’ and satisfies one of the following conditions: 1) the
classifier confidence score is greater than, or equal to 0.95, or 2) the tweet
text shares the same subject or object with the paired IFCN claim. The subject
and object are extracted using the Stanford OpenIE SVO parser. In total, we
collected 14,058 misinformation tweets.

## Misinformation Analysis

This section presents a statistical analysis of 14,058 source misinformation
tweets, including their topical distribution, emotional categories, and
distributional statistics. For comparison, we also conduct the same analysis
for non-misinformation tweets, which consist of 20,000 randomly selected
source tweets that were not classified as ‘misinformation’ by the classifier.

### Topical Distribution

Figure 2: Misinformation tweets topic distribution Figure 3: Non-
misinformation tweets topic distribution Figure 4: Misinformation tweets topic
distribution Figure 5: Non-misinformation tweets live state Figure 6:
Spreading power of misinformation tweets. Figure 7: Spreading power of non-
misinformation tweets. Average number of spreads of source misinformation and
non-misinformation tweets (retweets or quotes) in 4 hour intervals. The
coloured columns are the average number of spreads in each topic, and the
purple stepped line shows the average number of spreads including all topics.

Figure 2 and Figure 3 are the topic distribution of source misinformation and
non-misinformation tweets.

The topic distribution of source misinformation tweets is notably different
from that of non-misinformation tweets. The most frequently mentioned topic in
the misinformation tweets is ‘conspiracy theory’, accounting for almost one
third (33.1%) of all the misinformation tweets. The second-largest proportion
of the misinformation tweets pertains to ‘general medical advice’ (18%).
Conversely, the most commonly mentioned topics in non-misinformation tweets
are ‘prominent actors’ (22.7%), ‘community spread and impact’ (22.6%) and
‘public authority action’ (20.1%).

Figure 4 and Figure 5 display bar charts that show the number of tweets in
each topic, with colours indicating the live status of the tweet as of October
28, 2021. The live status of the tweets was determined by revisiting them via
the Twitter API. The colours used in the charts represent the following live
statuses: blue indicates that the tweet was still available on the day of the
revisit, red indicates that the account used to post the tweet has been
suspended by Twitter, Account deleted indicates that the account used to post
the tweet has been deleted for an unknown reason, ‘Tweet deleted’ indicates
that the tweet has been deleted for an unknown reason, and ‘Other’ indicates
that the tweet was not accessible on the day of the revisit due to privacy
settings.

The live statuses of misinformation and non-misinformation tweets exhibit
significant differences. More than 40% of the misinformation tweets were not
accessible during revisit, and the primary reason was account suspension, with
a suspension rate of 33.1%. In contrast, only 8.8% of non-misinformation
tweets were inaccessible, and the account suspension rate was much lower at
3.7%.

The spread of conspiracy theory misinformation seems to have garnered
significant attention from Twitter, with approximately half of the conspiracy
theory misinformation tweets being removed from the platform. In particular,
the account suspension rate for tweets about conspiracy theories is 37%. On
the other hand, misinformation tweets about the ‘virus origin’ seem to have
received the least attention, with almost 70% of such tweets still being
available on the platform and the account suspension rate being only 23.7%.

Figure 8: Bag-of-Words features associated with Non-misinformation (Red) and
Misinformation (Blue). Note that bigger font size denotes higher Pearson’s
Correlation R. Bag-Of-Words
---
Non-misinformation | r | Misinformation | r
Our | 0.138 | Gates | 0.204
Cases | 0.127 | Bill | 0.177
During | 0.125 | China | 0.172
Today | 0.121 | hydroxychloroquine | 0.171
We | 0.117 | Wuhan | 0.164
At | 0.107 | Coronavirus | 0.159
On | 0.101 | Lab | 0.149
And | 0.098 | Fauci | 0.144
Support | 0.098 | Vitamin | 0.143
Help | 0.093 | Virus | 0.143
Table 3: N-grams associated with Non-mis- and Mis- information sorted by
Pearson’s correlation ($r$) between the TF-IDF frequency and the labels
($p<.05$). LIWC
---
Non-misinformation | r | Misinformation | r
Affiliation | 0.200 | Health | 0.164
Time | 0.186 | Biological Processes | 0.142
Emotional Tone | 0.170 | Negative Emotion | 0.131
Relativity | 0.170 | Anger | 0.126
Prepositions | 0.148 | Quotation Marks | 0.123
Authentic | 0.136 | Past Focus | 0.113
Drives | 0.133 | Death | 0.109
Positive Emotion | 0.126 | Semicolons | 0.105
We | 0.126 | Causation | 0.093
Social | 0.124 | Dashes | 0.076
Table 4: LIWC categories associated with Non-mis- and Mis- information sorted
by Pearson’s correlation ($r$) between the normalised frequency and the labels
($p<.05$).

Figure 6 and Figure 7 show the average number of spreads (i.e. retweets or
quotes) of source misinformation and non-misinformation tweets in 4-hour
intervals. Overall, the analysis reveals that misinformation spreads more
quickly than non-misinformation. Specifically, the average spread power of a
misinformation tweet during the first 36 hours is 64.5, which means that the
tweet is retweeted or quoted an average of 64.5 times in that period. By
contrast, non-misinformation tweets have a much lower average spread power of
34.8 during the first 36 hours.

During the first 4 hours, the spread power of misinformation tweets is
highest, with a value of 30.8. The topics of ‘prominent actor’ and ‘public
authority action’ have notably higher spread power than other topics, with
values of 37.6 and 39.6 spreads, respectively. In contrast, the spread of non-
misinformation is more evenly distributed across different topics, and the
average spread power in the first 4 hours is 19.4.

After 4 hours, the number of spreads reduces significantly. In the next 4-hour
period, the average spread power of misinformation drops to 10.2, and that of
non-misinformation drops to 5.09. During this period, ‘public authority
action’ related misinformation still receives significantly higher spreads
(15.3 spreads) than other topics. In the period between 12 and 28 hours,
‘public preparation’ misinformation remains a hot topic, where an average
tweet on this topic has 32.8 spreads. Moreover, ‘conspiracy theory’ has a
longer spreading period than other topics, as it is the only topic that still
has more than two spreads even after 32 hours.

### Linguistic Analysis

In this section, we compare linguistic features between misinformation and
non-misinformation tweets. We follow a similar approach to a previous study by
Schwartz et al. (2013) and extract the top 10 correlated features using Bag-
Of-Words and Linguistic Inquiry and Word Count (LIWC) (Pennebaker, Francis,
and Booth 2001), based on Pearson’s correlations. The reported linguistic
features are statistically significant and show correlations with both
misinformation and non-misinformation categories (with a p-value of less than
0.05).

##### Bag-Of-Words

We established specific criteria to select the Bag-Of-Words (BOW) features,
which are as follows: i) the word’s document frequency must be between three
and 40% of the total tweets (i.e. $3<df<0.4*t$ where $df$ is the document
frequency, $t=32,748$ is the total number of tweets used for comparison
study). ii) The words are then scored based on the TF-IDF score, and we keep
the top 5,000 words as the BOW features for this analysis.

The BOW analysis results are presented in Table 3, where Pearson’s
correlations are reported in column r. The top 10 BOWs for misinformation
tweets align with the findings from the topical distribution analysis
discussed in Section ‘Topical Distribution’. The top 10 BOWs are mainly
associated with ‘conspiracy theories’ (e.g. _Gates_ , _Bill_
666https://www.politifact.com/factchecks/2020/may/20/facebook-posts/no-gates-
foundation-isnt-pushing-microchips-all-me/, _Lab_ , _China_
777https://www.poynter.org/?ifcn˙misinformation=coronavirus-has-been-
originated-in-a-laboratory-linked-to-chinas-biowarfare-program-2) and ‘general
medical advice’ (e.g. _hydroxychloroquine_
888https://www.poynter.org/?ifcn˙misinformation=hydroxychloroquine-completely-
cures-people-infected-with-covid-19, _Vitamin_). Compared to non-
misinformation, the top 10 BOWs all related to common words (e.g. _Our, We,
On, And_).

#blackfungus | #covid19vaccination | #socialdistancing | #chinesebioterrorism | #greatreset
---|---|---|---|---
#chinesevirus | #covid19vaccine | #stayathome | #chinesevirus | #hydroxichloroquine
#corona | #covid2019 | #stayhomestaysafe | #ciavirus | #hydroxychloroquine
#corona19 | #covid2019uk | #staysafe | #coronabollocks | #idonotconsent
#corona2019 | #covid_19 | #unite2fightcorona | #coronacon | #israelvirus
#coronapandemic | #covid_19_uk | #wearamask | #coronafacts | #kungflu
#coronasecondwave | #covid_2019_uk | #workfromhome | #coronafakenews | #mildsymptoms
#coronaupdate | #covidemergency | #wuhanvirus | #coronafraud | #nwo
#coronav | #covidemergency2021 | #5g | #coronahoax | #nwoevilelites
#coronavaccine | #covidhelp | #5gcoronavirus | #coronasymptoms | #nwoevilplans
#coronavirus | #covidresources | #americavirus | #coronavillains | #nwovirus
#coronavirus19 | #covidsecondwave | #astrazeneca | #coronavirus5g | #obamagate
#coronavirus2019 | #covidsos | #ccpvirus | #coronaviruscoverup | #oxfordvaccine
#coronavirusoutbreak | #coviduk | #chinaliedandpeopledied | #coronavirusfacts | #plandemic
#coronaviruspandemic | #covidvaccination | #chinaliedpeopledied | #covid19symptoms | #preventnwo
#coronavirustruth | #covidvaccine | #chinaliespeopledied | #covidiots | #remdesivir
#coronavirusupdates | #covid?19 | #chinesebioterrorism | #covidsymptoms | #reopenbritain
#covid | #lockdown | #chinesevirus | #cronyvirus | #resistthegreatreset
#covid-19 | #lockdown2021 | #ciavirus | #deepstatevirus | #scamdemic
#covid-19-uk | #pandemic | #chinaliedandpeopledied | #depopulation | #sorosvirus
#covid19 | #remotework | #chinaliedpeopledied | #endthelockdown | #wholiedpeopledied
#covid19uk | #sarscov2 | #chinaliespeopledied | #endthelockdownuk | #wuhanvirus
Table 5: The full list keywords used for Twitter collection though Twitter API Account Name | Display Name
---|---
ANI | ANI
GT | Global Times
roinnslainte | Department of Health
WHO | World Health Organization (WHO)
wef | World Economic Forum
GHS | Global Health Strategies
AFP | AFP News Agency
UN | United Nations
EMRO | WHO Eastern Mediterranean Regional Office
Table 6: The full list of credible accounts applied for the IRRELEVANT
training sample enrichment.

##### LIWC

The Linguistic Inquiry and Word Count (LIWC) (Pennebaker, Francis, and Booth
2001) is a dictionary of around 6,400 words that are mapped to 93 manually
created lexical categories. LIWC has been widely used in computational social
science. For this study, we used version 2015 of LIWC (Pennebaker, Francis,
and Booth 2001). Examples of words for each category can be found in the user
manual999https://repositories.lib.utexas.edu/bitstream/handle/2152/31333/LIWC2015˙LanguageManual.pdf.
The top 10 LIWC categories, sorted by Pearson’s Correlation (r) between the
normalised frequency and the labels (i.e., Non-mis vs. Misinformation), are
shown in Table 4. We observed that LIWC categories such as _Health_ ,
_Biological Processes_ , and _Death_ are highly correlated with
misinformation.

Furthermore, Table 4 displays the top 10 LIWC categories sorted by Pearson’s
Correlation (r) between the normalised frequency and the tweet categories
(i.e., non-misinformation vs. misinformation). Misinformation is highly
correlated with categories such as ‘Anger’, ‘Negative Emotion’, and ‘Death’.
In contrast, non-misinformation is more correlated with opposite categories,
such as ‘Positive Emotion’, ‘Authentic’, and social categories (‘Social’,
‘We’, ‘Affiliation’).

## Conclusion

This paper presents a comprehensive comparison study between non-mis- and mis-
COVID-19 information, using a machine learned classifier to extract
misinformation. To ensure the accuracy of misinformation classification, we
introduced a data enrichment process and post-process steps.

The results of our comparison study show that there are clear differences
between COVID-19 misinformation and non-misinformation tweets. Specifically,
we found that non-misinformation tweets predominantly focus on prominent
actors and community spread, whereas approximately one-third of misinformation
source tweets are related to conspiracy theory.

Our linguistic analysis supports the findings from the topical analysis. The
top 10 Bag-Of-Words features associated with misinformation are related to
conspiracy theory. Moreover, our LIWC analysis indicates that misinformation
is frequently associated with ‘negative emotion’, ‘anger’, and ‘death’, while
non-misinformation is associated with ‘positive emotion’, ‘authenticity’, and
‘social’ categories.

Addressing misinformation related to conspiracy theory appears to be the
primary concern of social media platforms, as evidenced by the removal of over
40% of related misinformation tweets on Twitter. Conversely, tweets related to
the topic of virus origin have received the least attention, with nearly 70%
of such tweets still accessible.

Misinformation tweets have a spreading power 158% higher than non-
misinformation tweets. Notably, tweets related to conspiracy theory have a
longer spreading period than other topics, with more than two spreads even
after 32 hours.

## References

  * Abdelminaam et al. (2021) Abdelminaam, D. S.; Ismail, F. H.; Taha, M.; Taha, A.; Houssein, E. H.; and Nabil, A. 2021.  Coaid-deep: An optimized intelligent framework for automated detecting covid-19 misleading information on twitter.  _Ieee Access_ , 9: 27840–27867.
  * Abdul-Mageed et al. (2020) Abdul-Mageed, M.; Elmadany, A.; Pabbi, D.; Verma, K.; and Lin, R. 2020.  Mega-cov: A billion-scale dataset of 65 languages for covid-19.  _arXiv preprint arXiv:2005.06012_ , 3(1).
  * Angeli, Premkumar, and Manning (2015) Angeli, G.; Premkumar, M. J. J.; and Manning, C. D. 2015.  Leveraging linguistic structure for open domain information extraction.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , 344–354.
  * Biradar, Saumya, and Chauhan (2022) Biradar, S.; Saumya, S.; and Chauhan, A. 2022.  Combating the infodemic: COVID-19 induced fake news recognition in social media networks.  _Complex & Intelligent Systems_, 1–13.
  * Brennen et al. (2020) Brennen, J. S.; Simon, F. M.; Howard, P. N.; and Nielsen, R. K. 2020.  _Types, sources, and claims of COVID-19 misinformation_.  Ph.D. thesis, University of Oxford.
  * Burel et al. (2020) Burel, G.; Farrell, T.; Mensio, M.; Khare, P.; and Alani, H. 2020.  Co-spread of misinformation and fact-checking content during the COVID-19 pandemic.  In _International Conference on Social Informatics_ , 28–42. Springer.
  * Caceres et al. (2022) Caceres, M. M. F.; Sosa, J. P.; Lawrence, J. A.; Sestacovschi, C.; Tidd-Johnson, A.; Rasool, M. H. U.; Gadamidi, V. K.; Ozair, S.; Pandav, K.; Cuevas-Lou, C.; et al. 2022.  The impact of misinformation on the COVID-19 pandemic.  _AIMS public health_ , 9(2): 262.
  * Chen et al. (2020a) Chen, E.; Lerman, K.; Ferrara, E.; et al. 2020a.  Tracking social media discourse about the covid-19 pandemic: Development of a public coronavirus twitter data set.  _JMIR public health and surveillance_ , 6(2): e19273.
  * Chen et al. (2020b) Chen, L.; Lyu, H.; Yang, T.; Wang, Y.; and Luo, J. 2020b.  In the eyes of the beholder: analyzing social media use of neutral and controversial terms for COVID-19.  _arXiv preprint arXiv:2004.10225_.
  * Cheng et al. (2021) Cheng, M.; Wang, S.; Yan, X.; Yang, T.; Wang, W.; Huang, Z.; Xiao, X.; Nazarian, S.; and Bogdan, P. 2021.  A COVID-19 Rumor Dataset.  _Frontiers in Psychology_ , 12.
  * Cinelli et al. (2020) Cinelli, M.; Quattrociocchi, W.; Galeazzi, A.; Valensise, C. M.; Brugnoli, E.; Schmidt, A. L.; Zola, P.; Zollo, F.; and Scala, A. 2020.  The COVID-19 social media infodemic.  _Scientific Reports_ , 10(1): 1–10.
  * Cui and Lee (2020) Cui, L.; and Lee, D. 2020.  CoAID: COVID-19 Healthcare Misinformation Dataset.  arXiv:2006.00885.
  * Dashtian and Murthy (2021) Dashtian, H.; and Murthy, D. 2021.  Cml-covid: A large-scale covid-19 twitter dataset with latent topics, sentiment and location information.  _arXiv preprint arXiv:2101.12202_.
  * Devlin et al. (2019) Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , 4171–4186.
  * Dharawat et al. (2022) Dharawat, A.; Lourentzou, I.; Morales, A.; and Zhai, C. 2022.  Drink Bleach or Do What Now? Covid-HeRA: A Study of Risk-Informed Health Decision Making in the Presence of COVID-19 Misinformation.  In _Proceedings of the International AAAI Conference on Web and Social Media_ , volume 16, 1218–1227.
  * Gupta, Vishwanath, and Yang (2021) Gupta, R.; Vishwanath, A.; and Yang, Y. 2021.  Global reactions to COVID-19 on Twitter: a labelled dataset with latent topic, sentiment and emotion attributes.
  * Hossain et al. (2020a) Hossain, T.; Logan IV, R. L.; Ugarte, A.; Matsubara, Y.; Young, S.; and Singh, S. 2020a.  COVIDLies: Detecting COVID-19 Misinformation on Social Media.  In _Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020_. Online: Association for Computational Linguistics.
  * Hossain et al. (2020b) Hossain, T.; Logan IV, R. L.; Ugarte, A.; Matsubara, Y.; Young, S.; and Singh, S. 2020b.  COVIDLIES: Detecting COVID-19 Misinformation on Social Media.  In _Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020_.
  * Hu et al. (2021) Hu, L.; Yang, T.; Zhang, L.; Zhong, W.; Tang, D.; Shi, C.; Duan, N.; and Zhou, M. 2021.  Compare to the knowledge: Graph neural fake news detection with external knowledge.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , 754–763.
  * Iwendi et al. (2022) Iwendi, C.; Mohan, S.; Ibeke, E.; Ahmadian, A.; Ciano, T.; et al. 2022.  Covid-19 fake news sentiment analysis.  _Computers and electrical engineering_ , 101: 107967.
  * Jain and Sethi (2021) Jain, D.; and Sethi, T. 2021.  The State of Infodemic on Twitter.  _arXiv preprint arXiv:2105.07730_.
  * Jiang et al. (2021) Jiang, Y.; Song, X.; Scarton, C.; Aker, A.; and Bontcheva, K. 2021.  Categorising Fine-to-Coarse Grained Misinformation: An Empirical Study of COVID-19 Infodemic.  _arXiv preprint arXiv:2106.11702_.
  * John and Keikhosrokiani (2022) John, S. A.; and Keikhosrokiani, P. 2022.  COVID-19 fake news analytics from social media using topic modeling and clustering.  In _Big Data Analytics for Healthcare_ , 221–232. Elsevier.
  * Kar et al. (2020) Kar, D.; Bhardwaj, M.; Samanta, S.; and Azad, A. P. 2020.  No rumours please! A multi-indic-lingual approach for COVID fake-tweet detection.  In _2021 Grace Hopper Celebration India (GHCI)_ , 1–5. IEEE.
  * Kouzy et al. (2020) Kouzy, R.; Abi Jaoude, J.; Kraitem, A.; El Alam, M. B.; Karam, B.; Adib, E.; Zarka, J.; Traboulsi, C.; Akl, E. W.; and Baddour, K. 2020.  Coronavirus goes viral: quantifying the COVID-19 misinformation epidemic on Twitter.  _Cureus_ , 12(3).
  * Lamsal (2021) Lamsal, R. 2021.  Design and analysis of a large-scale COVID-19 tweets dataset.  _Applied Intelligence_ , 51(5): 2790–2804.
  * Medford et al. (2020) Medford, R. J.; Saleh, S. N.; Sumarsono, A.; Perl, T. M.; and Lehmann, C. U. 2020\.  An “infodemic”: leveraging high-volume Twitter data to understand early public sentiment for the coronavirus disease 2019 outbreak.  In _Open Forum Infectious Diseases_ , volume 7, ofaa258. Oxford University Press US.
  * Mehrpour and Sadeghi (2020) Mehrpour, O.; and Sadeghi, M. 2020.  Toll of acute methanol poisoning for preventing COVID-19.  _Archives of toxicology_ , 94(6): 2259–2260.
  * Memon and Carley (2020) Memon, S. A.; and Carley, K. M. 2020.  Characterizing covid-19 misinformation communities using a novel twitter dataset.  _arXiv preprint arXiv:2008.00791_.
  * Micallef et al. (2020) Micallef, N.; He, B.; Kumar, S.; Ahamad, M.; and Memon, N. 2020.  The role of the crowd in countering misinformation: A case study of the COVID-19 infodemic.  In _2020 IEEE International Conference on Big Data (Big Data)_ , 748–757. IEEE.
  * Mu et al. (2023) Mu, Y.; Jin, M.; Grimshaw, C.; Scarton, C.; Bontcheva, K.; and Song, X. 2023.  VaxxHesitancy: A Dataset for Studying Hesitancy Towards COVID-19 Vaccination on Twitter.  _arXiv preprint arXiv:2301.06660_.
  * Müller, Salathé, and Kummervold (2020) Müller, M.; Salathé, M.; and Kummervold, P. E. 2020.  Covid-twitter-bert: A natural language processing model to analyse covid-19 content on twitter.  _arXiv preprint arXiv:2005.07503_.
  * Orellana (2023) Orellana, C. I. 2023.  Health workers as hate crimes targets during COVID-19 outbreak in the Americas.  _Revista de Salud Pública_ , 22: 253–257.
  * Pan et al. (2018) Pan, J. Z.; Pavlova, S.; Li, C.; Li, N.; Li, Y.; and Liu, J. 2018.  Content based fake news detection using knowledge graphs.  In _International semantic web conference_ , 669–683. Springer.
  * Pennebaker, Francis, and Booth (2001) Pennebaker, J. W.; Francis, M. E.; and Booth, R. J. 2001.  Linguistic inquiry and word count: LIWC 2001.  _Mahway: Lawrence Erlbaum Associates_ , 71(2001): 2001.
  * Photiou, Nicolaides, and Dhillon (2021) Photiou, A.; Nicolaides, C.; and Dhillon, P. S. 2021.  Social status and novelty drove the spread of online information during the early stages of COVID-19.  _Scientific reports_ , 11(1): 1–7.
  * Recuero et al. (2022) Recuero, R.; Soares, F. B.; Vinhas, O.; Volcan, T.; Hüttner, L. R. G.; and Silva, V. 2022.  Bolsonaro and the Far Right: How Disinformation About COVID-19 Circulates on Facebook in Brazil.  _International Journal of Communication_ , 16: 24.
  * Robertson et al. (1995) Robertson, S. E.; Walker, S.; Jones, S.; Hancock-Beaulieu, M. M.; Gatford, M.; et al. 1995.  Okapi at TREC-3.  _Nist Special Publication Sp_ , 109: 109.
  * Schwartz et al. (2013) Schwartz, H. A.; Eichstaedt, J. C.; Kern, M. L.; Dziurzynski, L.; Ramones, S. M.; Agrawal, M.; Shah, A.; Kosinski, M.; Stillwell, D.; and Seligman, M. E. 2013.  Personality, Gender, and Age in the Language of Social Media: The Open-vocabulary Approach.  _PloS ONE_ , 8(9).
  * Sharma et al. (2020) Sharma, K.; Seo, S.; Meng, C.; Rambhatla, S.; and Liu, Y. 2020.  Covid-19 on social media: Analyzing misinformation in twitter conversations.  _arXiv preprint arXiv:2003.12309_.
  * Shi et al. (2020) Shi, W.; Liu, D.; Yang, J.; Zhang, J.; Wen, S.; and Su, J. 2020.  Social bots’ sentiment engagement in health emergencies: A topic-based analysis of the covid-19 pandemic discussions on twitter.  _International Journal of Environmental Research and Public Health_ , 17(22): 8701.
  * Silva et al. (2020) Silva, M.; Ceschin, F.; Shrestha, P.; Brant, C.; Fernandes, J.; Silva, C. S.; Grégio, A.; Oliveira, D.; and Giovanini, L. 2020.  Predicting misinformation and engagement in covid-19 twitter discourse in the first months of the outbreak.  _arXiv preprint arXiv:2012.02164_.
  * Singh et al. (2020) Singh, L.; Bansal, S.; Bode, L.; Budak, C.; Chi, G.; Kawintiranon, K.; Padden, C.; Vanarsdall, R.; Vraga, E.; and Wang, Y. 2020.  A first look at COVID-19 information and misinformation sharing on Twitter.  _arXiv preprint arXiv:2003.13907_.
  * Song et al. (2021) Song, X.; Petrak, J.; Jiang, Y.; Singh, I.; Maynard, D.; and Bontcheva, K. 2021\.  Classification aware neural topic model for COVID-19 disinformation categorisation.  _PloS one_ , 16(2): e0247086.
  * Suarez-Lledo et al. (2022) Suarez-Lledo, V.; Ramos-Fiol, B.; Ortega-Martin, M.; Carretero-Bravo, J.; and Alvarez-Galvez, J. 2022.  Use of Hashtags related to Covid-19 infodemics by bot accounts: Victor Suarez-Lledo.  _European Journal of Public Health_ , 32(Supplement_3): ckac131–172.
  * van Stekelenburg et al. (2023) van Stekelenburg, B. C.; De Cauwer, H.; Barten, D. G.; and Mortelmans, L. J. 2023\.  Attacks on health care workers in historical pandemics and COVID-19.  _Disaster medicine and public health preparedness_ , 17: e309.
  * Waheeb, Khan, and Shang (2022) Waheeb, S. A.; Khan, N. A.; and Shang, X. 2022.  Topic Modeling and Sentiment Analysis of Online Education in the COVID-19 Era Using Social Networks Based Datasets.  _Electronics_ , 11(5): 715.
  * WHO (2020) WHO. 2020.  Novel Coronavirus(2019-nCoV) Situation Report—13.  World Health Organization.
  * WHO et al. (2020) WHO; UN; UNICEF; UNDP; UNESCO; UNAIDS; ITU; Pulse, U. G.; and IFRC. 2020.  Managing the COVID-19 infodemic: Promoting healthy behaviours and mitigating the harm from misinformation and disinformation.  https://www.who.int/news/item/23-09-2020-managing-the-covid-19-infodemic-promoting-healthy-behaviours-and-mitigating-the-harm-from-misinformation-and-disinformation.  Accessed: 1 Dec 2020.
  * Zhao et al. (2023) Zhao, S.; Hu, S.; Zhou, X.; Song, S.; Wang, Q.; Zheng, H.; Zhang, Y.; Hou, Z.; et al. 2023.  The Prevalence, Features, Influencing Factors, and Solutions for COVID-19 Vaccine Misinformation: Systematic Review.  _JMIR Public Health and Surveillance_ , 9(1): e40201.