# Self-Induced Superradiant Masing

Wenzel Kersten<EMAIL_ADDRESS>Vienna Center for Quantum Science and
Technology, Atominstitut, TU Wien, Stadionallee 2, A-1020 Vienna, Austria
Nikolaus de Zordo  Vienna Center for Quantum Science and Technology,
Atominstitut, TU Wien, Stadionallee 2, A-1020 Vienna, Austria  Elena S.
Redchenko  Vienna Center for Quantum Science and Technology, Atominstitut, TU
Wien, Stadionallee 2, A-1020 Vienna, Austria  Nikolaos Lagos  Vienna Center
for Quantum Science and Technology, Atominstitut, TU Wien, Stadionallee 2,
A-1020 Vienna, Austria  Andrew N. Kanagin  Vienna Center for Quantum Science
and Technology, Atominstitut, TU Wien, Stadionallee 2, A-1020 Vienna, Austria
Andreas Angerer  Vienna Center for Quantum Science and Technology,
Atominstitut, TU Wien, Stadionallee 2, A-1020 Vienna, Austria  William J.
Munro  Okinawa Institute of Science and Technology Graduate University, 1919-1
Tancha, Onna-son, Kunigami-gun, Okinawa 904-0495, Japan  Kae Nemoto  Okinawa
Institute of Science and Technology Graduate University, 1919-1 Tancha, Onna-
son, Kunigami-gun, Okinawa 904-0495, Japan  Igor E. Mazets  Vienna Center for
Quantum Science and Technology, Atominstitut, TU Wien, Stadionallee 2, A-1020
Vienna, Austria  Jörg Schmiedmayer<EMAIL_ADDRESS>Vienna Center
for Quantum Science and Technology, Atominstitut, TU Wien, Stadionallee 2,
A-1020 Vienna, Austria

###### Abstract

We study superradiant masing in a hybrid system composed of nitrogen-vacancy
center spins in diamond coupled to a superconducting microwave cavity. After
the first fast superradiant decay we observe transient pulsed and then quasi-
continuous masing. This emission dynamics can be described by a
phenomenological model incorporating the transfer of inverted spin excitations
into the superradiant window of spins resonant with the cavity. After
experimentally excluding cQED effects associated with the pumping of the
masing transition we conjecture that direct higher-order spin-spin
interactions are responsible for creating the dynamics and the transition to
the sustained masing. Our experiment thus opens up a novel way to explore
many-body physics in disordered systems through cQED and superradiance.

††preprint: APS/123-QED

In cavity quantum electrodynamics (cQED) [1] and particularly superradiance
[2, 3], the emitters are generally assumed to be independent and non-
interacting [4]. Interactions only come about through the emitted light itself
[5]. These mediated interactions lead to an extremely rich zoo of phases and
phase transitions in the emitter-light system [6, 7, 8, 9]. The role of
genuine many-body physics coming from direct interactions between the emitters
in these systems, however, remains elusive.

Superradiance – enhanced collective emission exhibiting high coherence and a
nonlinear scaling of the intensity with the number of emitters [4] – has been
observed in many systems [10, 11, 12, 13, 14] including solid-state
realizations with quantum dots [15, 16] and the negatively charged nitrogen-
vacancy centers ($\mathrm{NV^{\text{-}}}$) in diamond [17, 18]. In this solid-
state context where billions of emitters are confined on a
$\text{\,}\mathrm{\SIUnitSymbolMicro m}$ scale, the direct dipole-dipole
interactions start to play a larger role.

In our experiment, we investigate a dense ensemble of $\mathrm{NV^{\text{-}}}$
spins strongly and homogeneously coupled to a MW cavity which can be tuned in
and out of resonance much faster than the timescales of the spin ensemble. The
setup details are outlined in Ref. [19], where we demonstrated the triggered
superradiant emission from an inverted spin ensemble. Our system, illustrated
in Fig. 1a, allows for the application of strong microwave inversion pulses
and readout using a heterodyne detection scheme. The cavity is made of two
parallel sapphire chips, each featuring a superconducting split-ring
structure. The diamond sample, a cube with side length of around
$200\text{\,}\mathrm{\SIUnitSymbolMicro m}$, is positioned between the chips.
A detuning loop is wrapped directly around the chips and used for rapid
switching of the external magnetic field. This assembly is then placed in a
copper box and located inside a dilution refrigerator operating at
$25\text{\,}\mathrm{mK}$. The cavity has a resonance frequency of around
$\omega_{c}/2\pi=$3.1\text{\,}\mathrm{GHz}$$ and linewidth
$\kappa/2\pi=$418\text{\,}\mathrm{kHz}$$ (half-width at half-maximum). The
$N\approx$5.3\text{\times}{10}^{12}$$ $\mathrm{NV^{\text{-}}}$ spins are
coupled to the cavity with a collective coupling strength of
$g_{\mathrm{coll}}/2\pi=$4.6\text{\,}\mathrm{MHz}$$, where the single spin
coupling of $g_{0}/2\pi\approx$2\text{\,}\mathrm{Hz}$$ is uniform over the
sample volume – a necessary requirement for the superradiant dynamics we
observe. The spin system has an effective ensemble linewidth of
$\Gamma/2\pi=$3.6\text{\,}\mathrm{MHz}$$ due to the inhomogeneously broadened
spin frequency distribution $\rho(\omega)$ [20], with a width of
$W/2\pi=$9.2\text{\,}\mathrm{MHz}$$ (full-width at half-maximum), and
individual spin linewidths of
$\gamma_{\perp}/2\pi\approx$0.2\text{\,}\mathrm{MHz}$$. Next, the typical
nearest-neighbor distance between $\mathrm{NV^{\text{-}}}$ spins in our
diamond sample with roughly 6 ppm concentration is
$r\approx$10\text{\,}\mathrm{nm}$$. That results in a spin-spin coupling
strength between neighboring $\mathrm{NV^{\text{-}}}$s of
$J_{n}/2\pi\approx$52\text{\,}\mathrm{kHz}$$. Further details are provided in
the Supplementary Methods section and in Supplementary Fig. 1.

Initial superradiant decay
For all experiments, we employ the previously established protocol [19] for
generating a uniformly inverted spin state. All $\mathrm{NV^{\text{-}}}$ spins
are tuned into resonance with the cavity, using a static magnetic field with
equal projections along the four diamond axes. We apply a microwave inversion
pulse to homogeneously invert all spins from a relaxed initial state of the
effective two-level systems. Subsequently, the spin ensemble is rapidly
detuned from the cavity resonance and stored for a set hold time. This
procedure allows for the preparation of states with uniform initial spin
inversion $p_{0}=\langle\sigma_{z}^{i}\rangle$ and almost zero transversal
spin components $\langle\sigma_{-}^{i}\rangle\approx 0$. The values for
$p_{0}$, bounded by $\pm 1$, are tunable within the range of $0.1$ to $0.4$ by
modifying the hold time on the order of milliseconds.

Figure 1: Experimental setup and superradiant masing dynamics. a, Schematic of
the superconducting microwave cavity strongly coupled to the
$\mathrm{NV^{\text{-}}}$ diamond. The cavity is made of two parallel chips
with split-ring structures inside a copper box. A superconducting wire loop
surrounding the chips facilitates rapid spin detuning. b, Zoomed-in plot of
the cavity amplitude $|a|$ during the initial superradiant (SR) decay, which
is triggered upon tuning the inverted spin ensemble back into resonance and is
well described by a semi-classical model. c, After the initial superradiant
decay we observe a series of narrow masing pulses evolving into a quasi-
continuous cavity emission, slowly decreasing over a long timescale of half a
millisecond (see inset). Note the different y-axis scaling in b and c.

Upon tuning the spins back into resonance with the cavity using the detuning
loop, the inverted spin state is free to interact with the cavity mode. If the
stored inversion is above the threshold given by $p_{0}C>1$ [20], where
$C=g_{\mathrm{coll}}^{2}/\kappa\Gamma\approx 14$ is the cooperativity of the
coupled system, it enters a metastable state. In this state, any fluctuation
will stimulate a collective emission process known as a superradiant decay
[21]. Conversely, if the inversion is below the instability threshold,
dephasing due to inhomogeneous broadening becomes dominant and prevents the
avalanche process. In our case, the superradiant decay is triggered by noise
photons from the input line. As the spin decay accelerates, the cavity
amplitude $|a|$ increases. It reaches its maximum when the collective spin
vector points towards the equator of the Bloch sphere. Subsequently, the
energy is coherently exchanged between cavity and spins, visible as damped
Rabi oscillations in Fig. 1b. This coherent exchange is eventually stopped due
to loss processes in the system, with the main contribution being the
dephasing of the inhomogeneously broadened spins. The superradiant decay
dynamics is well-captured by the Maxwell-Bloch equations [22], a standard
semi-classical description involving non-interacting spins coupled to the
cavity.

Figure 2: Spectral analysis of quasi-continuous masing emission. a, Long tail
of the quasi-continuous masing emission, showing the quadratures $I$ and $Q$
of the cavity amplitude $|a|$, digitally demodulated in the rotating frame of
the cavity resonance frequency for visual clarity. The gray shaded area marks
the interval for the Fourier analysis, which is plotted in b, where the
emission has a linewidth much smaller than the cavity, see inset. The
frequency difference $\Delta\omega/2\pi$ is relative to the cavity frequency
of $3.1\text{\,}\mathrm{GHz}$. c, The frequency and linewidth of the emission
changes over time when the window of the Fourier analysis is shifted.

Pulsed revivals and quasi-continuous masing
Surprisingly, following a pause of $\Delta
t\approx$15\text{\,}\mathrm{\SIUnitSymbolMicro s}$$ after the initial
superradiant decay, a sequence of revival pulses emerges, as shown in Fig. 1c,
which goes beyond the standard description. The revival dynamics timescale is
unexpected, as it is much longer than the characteristic timescales of the
cavity loss rate, the ensemble dephasing, and the collective interaction of
the coupled system, $\Delta t\gg\\{\kappa,\Gamma,g_{\mathrm{coll}}\\}^{-1}$.
It also significantly exceeds the individual spin linewidth $\Delta
t\gg\gamma_{\perp}^{-1}$. This excludes any Rabi oscillations or spin-echo
behavior as a possible explanation for the revivals. The revival pulses
manifest as distinct peaks in the cavity amplitude, progressively spreading
over time. These masing pulses are approximately Gaussian, featuring full-
widths at half-maximum ranging from around
$1.2\text{\,}\mathrm{\SIUnitSymbolMicro s}$ to
$2.9\text{\,}\mathrm{\SIUnitSymbolMicro s}$. Each pulse has a random but
nearly constant phase as determined from the quadratures $I$ and $Q$ of the
cavity amplitude, corresponding to coherent pulsed masing with bandwidths
ranging from around $120\text{\,}\mathrm{kHz}$ to $350\text{\,}\mathrm{kHz}$.
After the early sequence of discrete pulses, subsequent ones start to merge,
transforming into a quasi-continuous and sustained cavity emission that
persists for more than $500\text{\,}\mathrm{\SIUnitSymbolMicro s}$.

In Fig. 2, we focus on the spectral characteristics of the quasi-continuous
masing emission. The extended cavity dynamics, shown in Fig. 2a, is measured
with the same system parameters as in Fig. 1c but at half the digitizer sample
rate in the heterodyne detection chain. The recorded quadratures $I$ and $Q$
of the cavity amplitude are demodulated at an intermediate frequency of
$5\text{\,}\mathrm{MHz}$ detuned from the cavity resonance. We extract the
emission’s linewidth by employing fast Fourier transform (FFT) analysis and
Lorentzian profile fitting within the integration window of
$200\text{\,}\mathrm{\SIUnitSymbolMicro s}$ (see Fig. 2b). Shifting the start
of the integration window $t_{\text{F}}$, we analyze in Fig. 2c how the
central frequency of the masing emission drifts over time within
$\pm$25\text{\,}\mathrm{kHz}$$. The observed linewidth, varying from
$5\text{\,}\mathrm{kHz}$ to $20\text{\,}\mathrm{kHz}$, is two orders of
magnitude below the cavity linewidth $\kappa$ and the individual spin
linewidth $\gamma_{\perp}$, highlighting the crucial role of collective
enhancement – an indicative trait of superradiance – in achieving high
coherence. We attribute the linewidth variation to the emission frequency
drift within the integration window, likely caused by magnetic field
fluctuations. These fluctuations are three orders of magnitude smaller than
the full magnetic field created by the detuning loop.

To investigate the cause of the unexpected revival pulses and quasi-continuous
masing, we conducted a set of auxiliary measurements. Rapidly detuning the
spins right after the initial superradiant decay inhibits the spin system’s
interaction with the cavity mode, effectively introducing a second hold time.
Extending this second hold time leads to an increase in the amplitude of the
first superradiant pulse afterward, see Fig. 3a. The amplitude increase has a
characteristic timescale of $T=$13.7\text{\,}\mathrm{\SIUnitSymbolMicro s}$$,
as shown in Fig. 3b. This result suggests that the cavity plays no role in the
mechanism driving the pulsed revivals. They persist even when the spins are
off-resonant with the cavity. Over time, this mechanism leads to an increase
in spin inversion, which gets converted into cavity photons upon release of
the superradiant decay. The amplitude decrease over very long times has been
discussed previously in Ref. [19].

Figure 3: Influence of second hold time on revival dynamics. a, Stacked cavity
signals of the superradiant dynamics with a second stabilization sequence
(hold time), represented by light green shading. Increasing the duration of
this second hold time extends the spectral hole-filling process, influencing
the amplitude of the superradiant masing pulse revival. b, Revival amplitude
for varying second hold times, showing a sharp initial increase, and an
exponential decrease for very long timescales. Figure 4:  Simulation of
superradiant dynamics with effective relaxation rate model. a, Initial spin
distribution showing a homogeneously inverted ensemble (green), a deep
spectral hole after superradiant decay (red), a partially refilled hole before
the first revival pulse (yellow). The y-axis scaling represents a weighted
spin inversion $\rho_{j}\langle\sigma_{z}^{j}\rangle$, rescaled to accurately
represent $\langle\sigma_{z}^{j}\rangle$ of the center frequency bin. For
comparison, we show the cavity resonance (black) together with the spin
packet’s cooperativities $C_{j}$ (pink). b, Color plot illustrating the
simulated spin inversion dynamics for spin packets with different detunings.
Vertical lines indicate three crucial time steps highlighted in a. c, Measured
and simulated cavity amplitude $|a|$ using the spectral hole-filling model. We
also plot the weighted instability threshold $\overline{pC}$ over time.

Effective model
To gain deeper insights into the underlying dynamical processes, we take a
closer look at the semi-classical model. In our simulations, we discretize the
spin frequency distribution into a number $N_{\rho}=501$ of numerical spin
packets with index $j$ and different weights $\rho_{j}$, as plotted in Fig.
4a. For simulating the initial superradiant decay, we initialize all spins
with uniform inversion $p_{0}$ and a small non-zero
$\langle\sigma_{-}^{j}\rangle$ component, to account for the triggering of the
superradiant decay. We adjust only the value of $p_{0}$ and a time offset to
simulate the measured signals as in Fig. 1b. The simulation illustrates that
during the superradiant decay, only spins close to the cavity resonance are
selectively de-excited, creating a spectral hole in the spin distribution, as
shown in Fig. 4a for $t=$7\text{\,}\mathrm{\SIUnitSymbolMicro s}$$ (red mark
in Fig. 4b). After the superradiant decay has occurred, the instability
threshold, now evaluated by summing over the product of each spin packet’s
weighted inversion and cooperativity, represented graphically in Fig. 4a, and
denoted as $\overline{pC}=\sum_{j}\rho_{j}N_{\rho}\langle\sigma_{z}^{j}\rangle
C_{j}$, drops below $1$. This indicates stabilization of the ensemble,
preventing superradiant emission at later times.

To model the pulsed revivals while leaving the initial superradiant decay
virtually unaffected, we introduce a new term to the Maxwell-Bloch equations
that induces a gradual filling of the spectral hole. This additional term
allows each spin packet $j$ to slowly approach the mean ensemble inversion
$p=\sum_{k}\rho_{k}\langle\sigma_{z}^{k}\rangle$ with a characteristic
relaxation rate $\mathcal{J}$, expressed as

$\frac{d}{dt}\langle\sigma_{z}^{j}\rangle=\Lambda_{\langle\sigma_{z}^{j}\rangle}+\mathcal{J}\left(p-\langle\sigma_{z}^{j}\rangle\right),$
(1)

where $\Lambda_{\langle\sigma_{z}^{j}\rangle}$ stands for the coherent
(relaxation-free) part of the dynamics of the system (see Supplementary
Methods). We also include a small constant drive $\eta$, steadily contributing
to the right-hand side of $\frac{d}{dt}\langle a\rangle$, modeling the effect
of cavity noise and intrinsic fluctuations that continually trigger the
recurring superradiant emission pulses. These intrinsic fluctuations are
caused by quantum, thermal and technical noise effects and necessitate a
higher-order cumulant expansion for a rigorous description [23]. To simplify
the analysis we restrict ourselves to the first-order semi-classical
description using $\eta$ to effectively model these stochastic processes. In
Fig. 4b we see the simulated effective filling of the spectral hole over time,
where inversion is transferred inwards from the sides of the spin
distribution. The corresponding simulated cavity amplitude captures
qualitatively the measured dynamics, as shown in Fig. 4c. The value of the
hole-filling rate $\mathcal{J}/2\pi=$16\text{\,}\mathrm{kHz}$$ used in this
simulation is in good agreement with the measured timescale
$T\sim\mathcal{J}^{-1}$ in Fig. 3b. Notably, $\mathcal{J}$ is still on the
same order of magnitude than the average nearest-neighbor spin-spin
interaction strength $J_{n}/2\pi\approx$52\text{\,}\mathrm{kHz}$$.

The interplay of the gradual spectral hole-filling and the threshold behavior
$\overline{pC}>1$ gives rise to the appearance of pulsed revivals, as plotted
in Fig. 4c. The revival pulses appear as soon as the threshold is exceeded and
the inverted spin state becomes unstable again, being triggered by the noise
photons in the cavity. Each revival pulse extracts inversion, reducing the
overall value of $p$, while individual spectral holes become shallower,
requiring less time to fill. However, the consecutive pulses take longer to
reach their maximum amplitude, as the self-accelerating superradiant avalanche
process is slower when starting with a lower effective number of participating
spins. Thus, the increasing delay time is inversely proportional to $p$ [24].
As the threshold value $\overline{pC}$ oscillates distinctly above and below
$1$ for initially separated superradiant pulses, oscillations decrease as the
pulses merge. A quasi-steady state is eventually reached, where energy lost
via the cavity is replenished by a steady in-streaming from the sides of the
spin distribution, while the threshold remains below $1$. The process halts
when the masing emission cannot be upheld due to significant inversion loss.

Figure 5:  Probing superradiant dynamics after a hole-burning pulse. a, Cavity
dynamics when applying a strong microwave pulse (red shading) to the spins
while detuned (green shading) and subsequently triggering the superradiant
decay and revival pulses. This will create a spectral hole in the ensemble,
resulting in a slice of decreased spin inversion at varying frequencies for
different runs. A resonant hole-burning pulse reproduces a spectral hole
similar to the one created by the initial superradiant decay when no hole-
burning pulse is applied. When the hole-burning pulse is off-resonant at the
sides of the spin distribution, it still leads to an earlier triggering of the
initial superradiant decay, but will otherwise recreate similar dynamics as
when no pulse is applied. b, Maximum amplitude of the superradiant decay pulse
for different hole-burning frequencies, revealing the frequency distribution
of the detuned spins, although appearing with a larger width due to power
broadening. Figure 6:  Influence of effective spectral hole area on the
refilling time. a, Evaluation of multiple cooldowns with different cavity
linewidths $\kappa$, measuring revival dynamics. Notably, the change of
linewidth is likely attributed to evaporation of the grease used to secure the
diamond sample in place, resulting in a cleaner surface over several cooldown
cycles. Our focus here lies on the effective area of the spectral hole
generated by the initial superradiant decay, characterized by its depth and
width. The depth of a spectral hole correlates with the initial inversion
$p_{0}$, as the spins involved in the superradiant decay effectively undergo a
$\pi$-rotation. Meanwhile, the width is determined by $\kappa$, given that
only spins within the cavity linewidth interact with the mode. The initial
inversion $p_{0}$ is extracted from the initial superradiant decay using our
numerical model, adjusting $\kappa$ while keeping other model parameters
constant. The time difference $\Delta t$ of the maxima from the initial
superradiant decay to the first revival pulse is compared with the effective
area of the spectral hole $p_{0}\kappa$, showing a consistent linear trend.
This time difference is a measure of the duration required to fill up the
spectral hole with effective area $p_{0}\kappa$, bringing the ensemble above
the threshold for $\overline{pC}$ to allow for the superradiant revival pulse.
b, Three experimental runs with different initial inversions $p_{0}$ for the
cooldown with $\kappa/2\pi=$418\text{\,}\mathrm{kHz}$$, illustrating the
methodology to attain $\Delta t$.

We provide additional evidence supporting the spectral hole-filling model in
Fig. 5 we show that deliberately creating a spectral hole with a microwave
pulse during the preparation of the inverted state can closely replicate the
dynamics when the hole is generated by the initial superradiant decay.
Moreover in Fig. 6 we demonstrate that the spectral hole filling time $\Delta
t$ is consistently related to the effective area of the spectral hole given by
$p_{0}\kappa$. The depth of a spectral hole is correlated with the initial
inversion $p_{0}$, as the spins involved in the superradiant decay effectively
undergo a $\pi$-rotation. The width of a spectral hole is determined by the
cavity linewidth $\kappa$, given that only spins within the cavity linewidth
interact with the mode. The relation $\Delta t\propto p_{0}\kappa$ holds
across various cooldowns with different cavity linewidths.

Spin-spin interactions
Our results suggest that direct spin-spin interactions play a crucial role in
the spectral hole-filling process. Spin-phonon processes cannot provide the
necessary mechanism, as they are too weak with a long spin-lattice relaxation
time of $T_{1}\approx$134\text{\,}\mathrm{s}$$ in our sample [19] and the
phonon modes not being populated at $25\text{\,}\mathrm{mK}$ [25]. A simple
dynamics involving spin-flips between neighboring spins is insufficient to
explain the observed behavior. A network of resonant single spins is
characterized by comparable energy mismatches determined from the difference
of individual spin detunings $|\Delta_{s}^{i}-\Delta_{s}^{j}|\sim W$ and
coupling rates $J_{ij}$. It remains disconnected because the number of
resonant single spins $nVJ_{ij}/W$ in a sample with density $n$ stays constant
as the volume $V$ grows, with the dipole-dipole interaction scaling as
$J_{ij}\propto r^{-3}$. The interconnection within the spin network only
appears when higher-order many-body interactions are considered, leading to a
divergence in the number of resonant spin pairs with growing volume. The
lowest-order relevant process involves the dynamics between different spin
pairs, satisfying energy conservation in a four-spin resonance condition.
These dynamics enable percolation in spin-excitation transport by connecting
resonant spin pairs into an extended network, as demonstrated in Ref. [26].
This network connects spins not only in space but also on different energies,
thus providing a many-body mechanism for refilling of the spectral hole.

Discussion
Our experimental platform provides a new path toward fundamental insights into
higher-order many-body interactions within disordered spin systems, unraveling
the impact of long-range interactions on the relaxation processes of quantum
many-body systems. The observed spectral hole refilling requires that a
typical eigenstate [27] in the interacting spin network – when projected onto
the basis-states of non-interacting spins – has a spectral bandwidth that is
broader than the width of the spectral hole. This allows for the transfer of
spin excitations from the spins outside into the spectral hole and hence for
equilibration over time. This may shed some new light on eigenstate
thermalization and the associated dynamics [28, 29] in closed and open many-
body systems. Beyond these fundamental aspects, our work holds implications
for advancing quantum technologies, particularly in spin-based microwave
amplification [30] and solid-state masers [31], with expected ultranarrow
linewidths [32].

Acknowledgments
We thank E. Demler, H. Ritsch, S. Yelin, and T. Zhang for fruitful
discussions. We acknowledge support by the Austrian Science Fund (FWF)
projects I3765 (MICROSENS), P34314 (Spins in Quantum Solids). W.J.M and K.N
would like to acknowledge support from the MEXT Quantum Leap Flagship Program
(MEXT QLEAP) Grant No. JPMXS0118069605.

## References

  * Walther  _et al._ [2006] H. Walther, B. T. Varcoe, B.-G. Englert, and T. Becker, Cavity quantum electrodynamics, Reports on Progress in Physics 69, 1325 (2006).
  * Dicke [1954] R. H. Dicke, Coherence in spontaneous radiation processes, Physical Review 93, 99 (1954).
  * Scully and Svidzinsky [2009] M. O. Scully and A. A. Svidzinsky, The super of superradiance, Science 325, 1510 (2009).
  * Gross and Haroche [1982] M. Gross and S. Haroche, Superradiance: An essay on the theory of collective spontaneous emission, Physics Reports 93, 301 (1982).
  * Mok  _et al._ [2023] W.-K. Mok, A. Asenjo-Garcia, T. C. Sum, and L.-C. Kwek, Dicke superradiance requires interactions beyond nearest neighbors, Physical Review Letters 130, 213605 (2023).
  * Hepp and Lieb [1973] K. Hepp and E. H. Lieb, On the superradiant phase transition for molecules in a quantized radiation field: the dicke maser model, Annals of Physics 76, 360 (1973).
  * Wang and Hioe [1973] Y. K. Wang and F. Hioe, Phase transition in the dicke model of superradiance, Physical Review A 7, 831 (1973).
  * Baumann  _et al._ [2010] K. Baumann, C. Guerlin, F. Brennecke, and T. Esslinger, Dicke quantum phase transition with a superfluid gas in an optical cavity, Nature 464, 1301 (2010).
  * Ferioli  _et al._ [2023] G. Ferioli, A. Glicenstein, I. Ferrier-Barbut, and A. Browaeys, A non-equilibrium superradiant phase transition in free space, Nature Physics 19, 1345 (2023).
  * van Loo  _et al._ [2013] A. F. van Loo, A. Fedorov, K. Lalumière, B. C. Sanders, A. Blais, and A. Wallraff, Photon-mediated interactions between distant artificial atoms, Science 342, 1494 (2013).
  * Araújo  _et al._ [2016] M. O. Araújo, I. Krešić, R. Kaiser, and W. Guerin, Superradiance in a large and dilute cloud of cold atoms in the linear-optics regime, Physical Review Letters 117, 073002 (2016).
  * Zhong  _et al._ [2017] T. Zhong, J. M. Kindem, J. Rochman, and A. Faraon, Interfacing broadband photonic qubits to on-chip cavity-protected rare-earth ensembles, Nature Communications 8, 14107 (2017).
  * Kim  _et al._ [2018] J. Kim, D. Yang, S.-h. Oh, and K. An, Coherent single-atom superradiance, Science 359, 662 (2018).
  * Rainò  _et al._ [2018] G. Rainò, M. A. Becker, M. I. Bodnarchuk, R. F. Mahrt, M. V. Kovalenko, and T. Stöferle, Superfluorescence from lead halide perovskite quantum dot superlattices, Nature 563, 671 (2018).
  * Scheibner  _et al._ [2007] M. Scheibner, T. Schmidt, L. Worschech, A. Forchel, G. Bacher, T. Passow, and D. Hommel, Superradiance of quantum dots, Nature Physics 3, 106 (2007).
  * Zhu  _et al._ [2024] C. Zhu, S. C. Boehme, L. G. Feld, A. Moskalenko, D. N. Dirin, R. F. Mahrt, T. Stöferle, M. I. Bodnarchuk, A. L. Efros, P. C. Sercel, _et al._ , Single-photon superradiance in individual caesium lead halide quantum dots, Nature , 1476 (2024).
  * Bradac  _et al._ [2017] C. Bradac, M. T. Johnsson, M. v. Breugel, B. Q. Baragiola, R. Martin, M. L. Juan, G. K. Brennen, and T. Volz, Room-temperature spontaneous superradiance from single diamond nanocrystals, Nature Communications 8, 1205 (2017).
  * Angerer  _et al._ [2018] A. Angerer, K. Streltsov, T. Astner, S. Putz, H. Sumiya, S. Onoda, J. Isoya, W. J. Munro, K. Nemoto, J. Schmiedmayer, _et al._ , Superradiant emission from colour centres in diamond, Nature Physics 14, 1168 (2018).
  * Kersten  _et al._ [2023] W. Kersten, N. de Zordo, O. Diekmann, T. Reiter, M. Zens, A. N. Kanagin, S. Rotter, J. Schmiedmayer, and A. Angerer, Triggered superradiance and spin inversion storage in a hybrid quantum system, Physical Review Letters 131, 043601 (2023).
  * Julsgaard and Mølmer [2012] B. Julsgaard and K. Mølmer, Dynamical evolution of an inverted spin ensemble in a cavity: Inhomogeneous broadening as a stabilizing mechanism, Physical Review A 86, 063810 (2012).
  * Skribanowitz  _et al._ [1973] N. Skribanowitz, I. P. Herman, J. C. MacGillivray, and M. S. Feld, Observation of Dicke superradiance in optically pumped HF gas, Physical Review Letters 30, 309 (1973).
  * Carmichael [1999] H. J. Carmichael, Statistical methods in quantum optics 1: master equations and Fokker-Planck equations, Vol. 1 Springer Science & Business Media, (1999).
  * Kubo [1962] R. Kubo, Generalized cumulant expansion method, Journal of the Physical Society of Japan 17, 1100 (1962).
  * Moi  _et al._ [1983] L. Moi, P. Goy, M. Gross, J. M. Raimond, C. Fabre, and S. Haroche, Rydberg-atom masers. I. A theoretical and experimental study of super-radiant systems in the millimeter-wave domain, Physical Review A 27, 2043 (1983).
  * Astner  _et al._ [2018] T. Astner, J. Gugler, A. Angerer, S. Wald, S. Putz, N. J. Mauser, M. Trupke, H. Sumiya, S. Onoda, J. Isoya, _et al._ , Solid-state electron spin lifetime limited by phononic vacuum modes, Nature Materials 17, 313 (2018).
  * Yao  _et al._ [2014] N. Y. Yao, C. R. Laumann, S. Gopalakrishnan, M. Knap, M. Mueller, E. A. Demler, and M. D. Lukin, Many-body localization in dipolar systems, Physical Review Letters 113, 243002 (2014).
  * Goldstein  _et al._ [2010] S. Goldstein, J. L. Lebowitz, C. Mastrodonato, R. Tumulka, and N. Zanghi, Normal typicality and von Neumann’s quantum ergodic theorem, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 466, 3203 (2010).
  * Srednicki [1994] M. Srednicki, Chaos and quantum thermalization, Phys. Rev. E 50, 888 (1994).
  * Srednicki [1999] M. Srednicki, The approach to thermal equilibrium in quantized chaotic systems, Journal of Physics A: Mathematical and General 32, 1163 (1999).
  * Sherman  _et al._ [2022] A. Sherman, O. Zgadzai, B. Koren, I. Peretz, E. Laster, and A. Blank, Diamond-based microwave quantum amplifier, Science Advances 8, eade6527 (2022).
  * Breeze  _et al._ [2018] J. D. Breeze, E. Salvadori, J. Sathian, N. M. Alford, and C. W. Kay, Continuous-wave room-temperature diamond maser, Nature 555, 493 (2018).
  * Wu  _et al._ [2022] Q. Wu, Y. Zhang, X. Yang, S.-L. Su, C. Shan, and K. Mølmer, A superradiant maser with nitrogen-vacancy center spins, Science China Physics, Mechanics & Astronomy 65, 1 (2022).
  * Sandner  _et al._ [2012] K. Sandner, H. Ritsch, R. Amsüss, C. Koller, T. Nöbauer, S. Putz, J. Schmiedmayer, and J. Majer, Strong magnetic coupling of an inhomogeneous nitrogen-vacancy ensemble to a cavity, Physical Review A 85, 053806 (2012).
  * Tavis and Cummings [1968] M. Tavis and F. W. Cummings, Exact solution for an N-molecule—radiation-field Hamiltonian, Physical Review 170, 379 (1968).

Supplementary Material for: Self-Induced Superradiant Masing

## Spin ensemble

The spin system used in this work is the negatively charged nitrogen-vacancy
center in diamond. The ground state Hamiltonian of the defect with spin $S=1$
is given by $\mathcal{H}=\hbar DS_{z}^{2}+\mu\vec{B}\cdot\vec{S}$, with the
zero field splitting $D/2\pi=$2.88\text{\,}\mathrm{GHz}$$ and
$\mu/2\pi=$28\text{\,}\mathrm{MHz}\text{\,}{\mathrm{mT}}^{-1}$$. The diamond
is cut from a larger sample with an initial nitrogen concentration of
approximately 200 ppm and naturally abundant $\\!{}^{13}\mathrm{C}$ isotopes,
treated with neutron irradiation with a total fluence of
$5\text{\times}{10}^{17}\text{\,}{\mathrm{cm}}^{-1}$ for
$50\text{\,}\mathrm{h}$ for the creation of lattice vacancies, and
subsequently annealed at $900\text{\,}\mathrm{\SIUnitSymbolCelsius}$ for
$3\text{\,}\mathrm{h}$ for the formation of $\mathrm{NV^{\text{-}}}$ centers.
The high amount of lattice damage due to the neutron irradiation is the reason
for the large inhomogeneous broadening $W/2\pi=$9.2\text{\,}\mathrm{MHz}$$ and
rather short
$T_{2}=1/\gamma_{\perp}\approx$0.9\text{\,}\mathrm{\SIUnitSymbolMicro s}$$.

Supplementary Fig. 1:  Spin distribution and steady-state spectroscopy
measurements. a, Measurement of the inhomogeneously broadened spin
distribution in the dispersive regime, with spins detuned by around
$\delta/2\pi\approx$50\text{\,}\mathrm{MHz}$$ from the cavity. Following a
$100\text{\,}\mathrm{ms}$ weak microwave drive at different frequencies, with
the spin system initially in the ground state, only pins with the
corresponding frequency will end up in an equal mixture of up and down spins,
while other spins are unaffected. This change in the overall spin ensemble
inversion leads to a dispersive shift $\chi\propto\langle S_{z}\rangle$ of the
cavity resonance frequency, measured using a vector network analyzer. b,
Transmission spectroscopy of the cavity using high (red) and low (blue)
microwave power. Using high input power, the whole spin ensemble is brought
into a completely mixed state, effectively decoupling it from the cavity. A
Lorentzian fit yields a linewidth of $\kappa/2\pi=$418\text{\,}\mathrm{kHz}$$.
Using low input power, the normal-mode splitting of the coupled system in the
ground state is measured and fitted with a collective coupling strength of
$g_{\mathrm{coll}}/2\pi=$4.6\text{\,}\mathrm{MHz}$$.

## Hybrid system characterization

We measure the inhomogeneously broadened spin distribution in the dispersive
regime, as shown in Supplementary Fig. 1a. This distribution is well described
by a $q$-Gaussian function [33]

$\rho(\omega)=\left(1-\left(1-q\right)(\omega-\omega_{0})^{2}/\delta_{q}^{2}\right)^{\frac{1}{1-q}}\,,$
(2)

with $\delta_{q}=W\sqrt{(q-1)/(2^{q-1}-1)}$, where $W$ is the full-width at
half-maximum, using a value for the shape parameter of $q=1.39$. We determine
the cavity linewidth $\kappa/2\pi=$418\text{\,}\mathrm{kHz}$$, by measuring
the steady-state transmission of the coupled system with a high input power of
the vector network analyzer, see Supplementary Fig. 1b. This brings the whole
spin ensemble into a completely mixed state, effectively decoupling it from
the cavity. Using low input power, we measure the normal-mode splitting of the
system in the ground state. A steady-state solution of Eqs. (4) provides the
fitted value of $g_{\mathrm{coll}}/2\pi=$4.6\text{\,}\mathrm{MHz}$$. Comparing
this value with the single spin coupling
$g_{0}/2\pi\approx$2\text{\,}\mathrm{Hz}$$ (obtained from a finite-element
simulation of the cavity) we get an estimation of the number of spins
$N=g_{\mathrm{coll}}^{2}/g_{0}^{2}\approx$5.3\text{\times}{10}^{12}\text{\,}$$.
Taking the volume of the diamond sample of around
$5.1\text{\times}{10}^{6}\text{\,}{\mathrm{\SIUnitSymbolMicro m}}^{3}$ and the
carbon density in diamond of
$n_{c}=$1.755\text{\times}{10}^{23}\text{\,}{\mathrm{cm}}^{-3}$$ into account
results in an $\mathrm{NV^{\text{-}}}$ concentration of approximately 6 ppm.

## Microwave setup, cavity, and inversion pulse

We employ I/Q mixing with an arbitrary waveform generator and a microwave
source at the cavity frequency to synthesize the inversion pulse. After
amplification with a high-power amplifier, the pulse is injected into cavity
port 1. The outgoing signal, exiting at cavity port 2, undergoes further
amplification both inside and outside the cryostat. It is demodulated using
another frequency source and recorded by a digitizer with a maximum sampling
rate of $2\text{\,}\mathrm{GHz}$. Additional details on the superconducting
cavity design and the protocol for initializing the inverted state, including
details of the inversion pulse and detuning loop switching, can be found in
Ref. [19].

## Effective theory

Our system is described by the driven Tavis-Cummings Hamiltonian in the
rotating frame [34],

$\displaystyle\begin{split}\mathcal{H}=&\hbar\Delta_{\mathrm{c}}a^{\dagger}a+\frac{\hbar}{2}\sum_{i}\Delta_{\mathrm{s}}^{i}\sigma_{z}^{i}+\mathrm{i}\hbar
g_{0}\sum_{i}\left(a^{\dagger}\sigma_{-}^{i}-\sigma_{+}^{i}a\right)+\mathrm{i}\hbar\eta\left(a^{\dagger}-a\right),\end{split}$
(3)

with $a^{\dagger}(a)$ being the creation (annihilation) operator of the cavity
mode and $\sigma_{z}^{i},\sigma_{\pm}^{i}$ being the Pauli-$z$ and
raising/lowering operators for the $i^{\mathrm{th}}$ spin, respectively. A
possible external driving field is represented by $\eta$. In our simulations,
we discretize the spin frequency distribution $\rho(\omega)$ into
$N_{\rho}=501$ spin packets with equal spacing. Each of the $N_{\rho}$
numerical spins, indexed by $j$, couples to the cavity with coupling strength
$g_{\rho}=g_{\mathrm{coll}}/\sqrt{N_{\rho}}$ and is weighted according to
$\sum_{j}\rho_{j}=1$. The inhomogeneous broadening of the spin ensemble is
modeled with the detunings
$\Delta_{\mathrm{s}}^{j}=\omega_{\mathrm{s}}^{j}-\omega_{\mathrm{p}}$, while
$\Delta_{\mathrm{c}}=\omega_{\mathrm{c}}-\omega_{\mathrm{p}}$ describes the
detuning of the cavity, both with respect to the driving frequency
$\omega_{\mathrm{p}}$.

The system’s losses caused by coupling to the environment are accounted for by
introducing a cavity loss rate $\kappa$ and a spin decoherence rate
$\gamma_{\perp}$ using a Lindblad master equation approach. We disregard
$T_{1}$ processes as the spin-lattice relaxation rate is negligible compared
to all other dynamical timescales of our system. To simplify the analysis,
higher-order correlations between spins and cavity are neglected, ending up
with the well-known Maxwell-Bloch equations,

$\displaystyle\frac{d}{dt}\langle a\rangle=$
$\displaystyle-(\kappa+\mathrm{i}\Delta_{\mathrm{c}})\langle
a\rangle+g_{\rho}N_{\rho}\sum_{j}\rho_{j}\langle\sigma_{-}^{j}\rangle+\eta,$
(4a) $\displaystyle\frac{d}{dt}\langle\sigma_{-}^{j}\rangle=$
$\displaystyle-(\gamma_{\perp}+\mathrm{i}\Delta_{\mathrm{s}}^{j})\langle\sigma_{-}^{j}\rangle+g_{\rho}\langle
a\rangle\langle\sigma_{z}^{j}\rangle,$ (4b)
$\displaystyle\frac{d}{dt}\langle\sigma_{z}^{j}\rangle=$
$\displaystyle-2g_{\rho}\left(\langle
a^{\dagger}\rangle\langle\sigma_{-}^{j}\rangle+\langle
a\rangle\langle\sigma_{+}^{j}\rangle\right)+\mathcal{J}\left(p-\langle\sigma_{z}^{j}\rangle\right).$
(4c)

The last term in Eq. (4c) phenomenologically describes the gradual filling of
a spectral hole at the relaxation rate $\mathcal{J}$ up to the mean ensemble
inversion $p=\sum_{k}\rho_{k}\langle\sigma_{z}^{k}\rangle$, cf. Eq. (1). We
solve these equations numerically to qualitatively model the observed
superradiant masing phenomena.

The instability criterion $\overline{pC}>1$ and the effective ensemble
linewidth $\Gamma$ are evaluated using the following definition

$C=\sum_{j}\rho_{j}N_{\rho}C_{j}=\frac{g_{\rho}^{2}N_{\rho}}{\kappa}\sum_{j}\rho_{j}\left(\gamma_{\perp}+\frac{{\Delta_{\mathrm{s}}^{j}}^{2}}{\gamma_{\perp}}\right)^{-1}=\frac{g_{\mathrm{coll}}^{2}}{\kappa\Gamma}.$
(5)

## Spin-spin interaction

We evaluate the nearest-neighbor dipole coupling rate $J_{n}$ from the
prefactor in the Hamiltonian

$\mathcal{H}_{\mathrm{dipole}}=-\hbar\sum_{i,j>i}\,\underbrace{\frac{\mu_{0}\mu^{2}\hbar}{4\pi
r_{ij}^{3}}}_{J_{ij}}\,\left[3(\mathbf{S}_{i}\cdot\mathbf{\hat{r}}_{ij})(\mathbf{S}_{j}\cdot\mathbf{\hat{r}}_{ij})-\mathbf{S}_{i}\cdot\mathbf{S}_{j}\right]$
(6)

using the typical distance between $\mathrm{NV^{\text{-}}}$ spins of
$r=n^{-1/3}\approx$10\text{\,}\mathrm{nm}$$, where $n$ is the density of
$\mathrm{NV^{\text{-}}}$ spins with 6 ppm in the diamond lattice. This results
in a value for the typical strength of the nearest-neighbor coupling rate of
$J_{n}/2\pi\approx$52\text{\,}\mathrm{kHz}$$.


# Preliminary Bias Results in Search Engines

Gizem Gezici  Huawei Turkey R&D CenterIstanbulTurkey<EMAIL_ADDRESS>
###### Abstract.

This report aims to report my thesis progress so far. My work attempts to show
the differences in the perspectives of two search engines, Bing and Google on
several selected controversial topics. In this work, we try to make a
distinction on the viewpoints of Bing & Google by using sentiment as well as
the ranking of the document returned from these two search engines on the same
queries, these queries are related mainly to controversial topics. You can
find the methods we used with experimental results below.

††copyright: rightsretained††conference: ACM SIGIR conference; July 8-12; En
Harbor, Michigan USA

##  1\. Introduction

The core component of my thesis work and our main goal is to show the
existence of bias in search engines (Bing & Google) which is a very strong
claim, therefore we started to differentiate the attitudes of these two search
engines on some controversial topics. You can find the details of our method
with the experimental results we obtained in the following sections after the
overview of our unpublished work, effective crowd labelling procedure.

##  2\. Sentiment-wise Comparison of Two Search Engines

Two popular search engines were mainly compared from sentiment perspective by
exploiting three different levels of sentiment metrics such as document-level,
sentence-level, and aspect-level.. Initially, we computed sentiment scores by
using TextBlob as well as the polarity values of the words in a widely used
domain-independent lexicon, SentiWordNet (Baccianella et al., 2010). However,
obtained values from SentiWordNet were not reasonable and did not reflect the
real sentiment of the given document. Therefore, we decided to use only
TextBlob for sentiment polarity detection in our experiments.

###  2.1. Document-level Sentiment Analysis

For document-level analysis, we obtain a sentiment value for each document
from TextBlob without any pre-processing step on documents, TextBlob handles
the punctuation in itself. This part of the analysis is more coarse-grained
relatively to the subsequent parts.

###  2.2. Sentence-level Sentiment Analysis

Differently from document-level, in sentence-level sentiment analysis we view
each document as a pile of sentences. This leads to a more fine-grained
analysis; thus in this part, we split the given document into sentences and
obtain polarity value for each of its sentences from TextBlob. In order to
obtain a sentiment score for the given document, we compute an average
sentiment by summing the polarity values of all the sentences and dividing it
by the number of sentences in the document. That is to say, from TextBlob’s
point of view, each sentence is a document and processed accordingly. For
clarification, again we have one polarity value for each document to compare
the retrieved document sets of two search engines, although we compute these
scores in distinct ways.

###  2.3. Aspect-level Sentiment Analysis

In addition to document and sentence-level, we also analyzed the documents on
aspect-level with the hope that it may convey different type of information
that is useful for the comparison of the search engines. In analyzing the
documents from the aspect-level perspective, we implemented a Java script on
Eclipse environment in order to find dependencies between words. In this
script, we read documents, split the documents into sentences and with the
help of Stanford NLP Parser (Manning et al., 2014) we obtain relations between
words. In this way, grammatically we can have deeper understanding of
documents and this additional information is exploited while analyzing the
documents in terms of sentiment. As a side note, detailed information about
the grammatical relations can be found in the Stanford NLP Parser
manual111https://nlp.stanford.edu/software/dependencies_manual.pdf.

In extracting dependencies between words in sentences, we only pay attention
to the relations of the controversial topic keywords. We select these keywords
for each topic as follows, if the given topic is composed of one word, we use
that word as the keyword for the corresponding topic. If the topic contains
more than one word, then we tokenize the topic title and all of the tokens are
used as the keywords for that topic. Note that in our current dataset, topic
titles are composed of at most two words, thus we have to deal with two-
keywords case in our dependency analysis differently from the simple case of
one-keyword topics. You can find sample relations extracted from the one-
keyword controversial queries abortion and brexit in Table 1. For two-keywords
queries, we obtain dependencies for both of the keywords in the given topic
and use all of these dependencies for the analysis. You can find sample
relations extracted from the two-keywords controversial query gay marriage in
Table 2.

While extracting relations between words, initially we find keyword/s for the
given topic, then we merely extract the relations of those keyword/s as
mentioned above. Apart from this, we obtain the relations between keywords and
only sentiment-baring words i.e., nouns, adjectives, adverbs, verbs. This is
stemmed from the fact that we use dependency information for sentiment
analysis and non sentiment-baring words will not be useful in our analyses.
After getting relations from the parser, we will compute the sentiment score
of the words that have grammatical relations with the keywords of the given
topic. Sentiment polarities are computed by using TextBlob for all the related
words in a document, then we compute an average sentiment score of each
document for comparison.

Word-Relation-Keyword
---
demand - nmod:on - abortion
illegal - nsubj - abortion
laws - compound - abortion
ban - dobj - abortion
restrict - dobj - abortion
negotiations - compound - brexit
bill - compound - brexit
risk - nmod:of - brexit
chaotic - amod - brexit
doubts - compound - brexit
Table 1. Sample Dependencies of Abortion & Brexit Word-Relation-Keyword
---
rights - amod - gay
community - amod - gay
activist - amod - gay
fans - amod - gay
nice - amod - gay
legalizing - dobj - marriage
support - nmod:for - marriage
recognize - dobj - marriage
outlaw - dobj - marriage
opposed - dobj - marriage
Table 2. Sample Dependencies of Gay Marriage

##  3\. Experimental Results

###  3.1. Dataset

In order to observe remarkable differences between the retrieval results of
two popular search engines, we selected 15 controversial topics from the
website of https://www.procon.org/. We collected data using news API of the
corresponding search engines: Bing and Google. With the aim of retrieving more
documents related to each topic, we extended the query set by using Google
Trends. Then, we implemented scripts on python to automate the crawling
process with the API keys of the search engines.

For each query sent to the search engine, we crawled the first 10 documents
returned from that search engine. The dataset is composed of 2560 documents in
total for one search engine. Topic distribution of the crawled dataset is
displayed in Table 3. In the table, one can see that there are 200 documents
crawled for the controversial query of abortion, this means that we have 20
extended queries for this specific topic since we use only the first 10
documents retrieved from the corresponding search engine. We note that the
data collection process was fulfilled in a controlled environment such that
the same queries were sent to the search engines almost at the same moment
since the documents to be retrieved may vary by time.

Topic | # of documents
---|---
Abortion | 200
Animal Testing | 80
Assisted Suicide | 80
Brexit | 180
Climate Change | 350
Gay Marriage | 220
Gun Control | 260
Medical Marijuana | 130
Minimum Wage | 220
Obamacare | 120
Prostitution | 70
Syrian Refugees | 130
Transgender Military | 80
Travel Ban | 140
Trump | 300
Table 3. Controversial Topic Distributions

###  3.2. Evaluation Framework

In evaluating two search engines, our aim is to compare these two search
engines on conservative/liberal perspective towards a given controversial
topic. For comparison, we utilize TextBlob sentiment scores at three different
levels , document, sentence, and aspect-level.

In order to make a consistent comparison among all controversial topics, one
needs to think about the connection between the semantic orientation and the
perspective of the given documents on the scope of their controversial topics.
For instance, if a document has a positive sentiment score and its
controversial topic is abortion, then the document’s perspective tends to be
more liberal, whereas if a document shows a negative attitude towards brexit,
similarly its perspective would be more close to liberal. This is because of
the fact that the underlying perspective of a document on conservative/liberal
axis in our evaluation scheme does not only depend on its semantic orientation
but also on the corresponding controversial topic. For this reason, after
obtaining polarity values at three different levels via TextBlob, we
transformed these values for some of the controversial topics in order to make
a consistent comparison for all controversial topics. In order to fulfill the
transformation task, the polarity values were simply multiplied by -1 for
animal testing, brexit, minimum wage, travel ban, and trump.

In our evaluation scheme, we used two metrics and computed these at three
different levels, document, sentence, and aspect-level. As the first metric,
sentiment scores obtained from TextBlob were used after the transformation
process as mentioned above, if necessary. Secondly, we proposed a new metric
that combines rank and sentiment information and takes into account both of
these for comparison. Also for the second metric, we used transformed
sentiment scores to provide a consistent perspective analysis, instead of the
actual polarity values obtained from TextBlob.

####  3.2.1. Sentiment Polarity Metric

Our first metric compares the two search engines by purely using sentiment
scores returned from TextBlob and the range for polarity values is [-1, 1]. We
obtained sentiment scores for documents, sentences, and aspects, then computed
an average polarity value as mentioned in the previous section for each
document by exploiting document, sentence, and aspect-level sentiment values
separately. That means that we only care about document sentiment scores for
comparison purposes, yet by computing these document scores in three different
levels of sentiment analysis then transforming some of these polarity values
if needed. You can find the table of mean of mean average sentiment scores in
Table 4 and comparison plot of two search engines in terms of mean avg
polarity scores in Figure 1.

Sentiment-level | Bing | Google
---|---|---
Document-level | 0.0260 | 0.0241
Sentence-level | 0.0195 | 0.0178
Aspect-level | -0.0018 | 0.0001
Table 4. Mean of Mean Average Sentiment Scores over All Controversial Topics Sentiment-level | Bing | Google
---|---|---
Document-level | 0.7845 | 0.7916
Sentence-level | 0.7821 | 0.7967
Aspect-level | 0.6099 | 0.5234
Table 5. Mean Average NDCG-Senti Scores over All Controversial Topics Dataset | p-value | Is Statistically Significant?
---|---|---
DOCLevel_MeanAvgScore | 0.9515 | NO
SENTLevel_MeanAvgScore | 0.9441 | NO
ASPLevel_MeanAvgScore | 0.3775 | NO
DOCLevel_AvgNDCGScore | 0.6225 | NO
SENTLevel_AvgNDCGScore | 0.2562 | NO
ASPLevel_AvgNDCGScore | 0.1841 | NO
Table 6. Two-tail t-test p-values Over All Controversial
Topics Figure 1. Mean Avg-Pol Scores of 15 Controversial Topics

####  3.2.2. NDCG-Senti Metric

In addition to the sentiment polarity metric, we also needed a more
informative metric that will be convenient to compare particularly the
retrieved document sets of two search engines. For this reason, we introduced
a new metric, NDCG-Senti that blends ranking and sentiment information, which
is a variant of commonly used NDCG metric. We utilized the traditional formula
of NDCG denoted in NDCG-FORMULA. For our ranking-sentiment metric NDCG-Senti,
we replaced relevance score $rel_{i}$ with sentiment score $pol_{i}$ in the
formula.

With a slight modification in the traditional formula of NDCG, we came up with
a modified NDCG scoring function which also takes document rank into
consideration while comparing document sets of search engines. However, there
is still one more issue in the computation of transformed NDCG function that
relevance grades in the traditional NDCG metric are always positive, whereas
the real sentiment values as well as the transformed ones become negative
since the polarity values obtained from TextBlob lie between -1 and 1. Note
that the polarity values of documents were firstly transformed and then they
were used for the computation of this second metric. In short, we computed
NDCG-Senti scores with transformed and subsequently normalized sentiment
scores of TextBlob. The normalization was applied as follows: Transformed
document polarity values of each query were normalized to the range of [0,1],
then our new metric was obtained by using the non-negative sentiment
polarities in each query.

In order to get rid of the negative values, we utilize min-max normalization
technique to map the polarities onto the range of [0, 1]. For each query,
there are 10 transformed sentiment polarities of the documents returned from
each search engine. We compute a minimum and a maximum value from these
polarity values of the document set of each query. Then, we normalize the
polarity values and obtain non-negative scores for the given set of documents.
Subsequently, we compute a NDCG-Senti score for the given set of documents
with transformed sentiment polarities. In other words, we compute different
min-max value pairs for each document set and normalize the polarity values
for each query separately. Then we computed an average NDCG-Senti score for
each controversial topic by using the NDCG-Senti score of each query in the
corresponding topic. You can find the table of mean average NDCG-Senti scores
in Table 5 and the comparison plot of two search engines in terms of average
NDCG-Senti scores is displayed in Figure 2.

Figure 2. Avg-Normalized NDCG Scores of 15 Controversial Topics

In addition to the comparative analysis in terms of average NDCG-Senti scores
for 15 controversial topics, we also compared two search engines by using
NDCG-Senti scores of each query in a controversial topic. In this analysis, we
compared two search engines per controversial topic and put the number of
instances above/below and on the y=x axis to show the tendencies properly.
While the previous analyses give the overall orientation of Bing & Google on
the whole dataset, per-controversial topic plots can elaborately depict the
differences in their attitudes towards the given topics. You can find sample
results per each controversial topic below.

####  3.2.3. Statistical Significance Test

After having computed and plotted sentiment and NDCG-Senti scores in Figure 1
and 2 to understand the overall tendencies of two search engines, we also need
to have a significance test to check if these differences are statistically
significant. For this, we applied the following procedure:

  * •

For each given data sample, check that it is normally distributed by creating
a histogram from the data points.

    * –

If so, then go to next step.

    * –

If not, transform the data points by using ”log”, ”exp” etc.

      * *

Check that if the given data sample is normally distributed by creating a
histogram from the transformed data points.

      * *

If not, try with a different transformation function for normality check and
use other normality check methods such as QQplot until you observe that the
data is normally distributed.

  * •

Making sure that both of the data samples (Bing & Google) are normally
distributed, apply F-test to see that the two data distributions have equal or
unequal variances.

  * •

Apply two-tail t-test with equal or unequal variances to understand if the
means of these two distributions are significantly different.

You can find the two-tail p values with .95 confidence level in Table 6.

Figure 3. Normalized NDCG Scores of Abortion Figure 4. Normalized NDCG Scores
of Animal Testing Figure 5. Normalized NDCG Scores of Assisted Suicide Figure
6. Normalized NDCG Scores of Brexit Figure 7. Normalized NDCG Scores of
Climate Change Figure 8. Normalized NDCG Scores of Gay Marriage Figure 9.
Normalized NDCG Scores of Gun Control Figure 10. Normalized NDCG Scores of
Medical Marijuana Figure 11. Normalized NDCG Scores of Minimum Wage Figure 12.
Normalized NDCG Scores of Obamacare Figure 13. Normalized NDCG Scores of
Prostitution Figure 14. Normalized NDCG Scores of Syrian Refugees Figure 15.
Normalized NDCG Scores of Transgender Military

Figure 16. Normalized NDCG Scores of Travel Ban

Figure 17. Normalized NDCG Scores of Trump

##  4\. Future Work

For the effective crowd labelling framework, we are waiting for the
acceptance. If the paper is accepted, then we will extend the existing
framework. For the second part, we are preparing the results for conference
publication. Afterwards, that part can be hopefully extended for journal
publication.

## References

  * (1)
  * Baccianella et al. (2010) Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2010\.  Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining.. In _LREC_ , Vol. 10. 2200–2204.
  * Manning et al. (2014) Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014.  The Stanford CoreNLP Natural Language Processing Toolkit. In _Association for Computational Linguistics (ACL) System Demonstrations_. 55–60.  http://www.aclweb.org/anthology/P/P14/P14-5010