#  Observation of strong attenuation within the photonic band gap of
multiconnected networks

Pengbo Zhu  Runkai Chen  Xiangbo Yang<EMAIL_ADDRESS>Yanglong Fan  Huada
Lian  Guangdong Provincial Key Laboratory of Nanophotonic Functional Materials
and Devices, School of Information and Optoelectronic Science and Engineering,
South China Normal University, Guangzhou 510006, China  Zhen-Yu Wang
<EMAIL_ADDRESS>Key Laboratory of Atomic and Subatomic Structure and
Quantum Control (Ministry of Education), and School of Physics, South China
Normal University, Guangzhou 510006, China  Guangdong Provincial Key
Laboratory of Quantum Engineering and Quantum Materials, and Guangdong-Hong
Kong Joint Laboratory of Quantum Matter, South China Normal University,
Guangzhou 510006, China

###### Abstract

We theoretically and experimentally study a photonic band gap (PBG) material
made of coaxial cables. The coaxial cables are waveguides for the
electromagnetic waves and provide paths for direct wave interference within
the material. Using multiconnected coaxial cables to form a unit cell, we
realize PBGs via (i) direct interference between the waveguides within each
cell and (ii) scattering among different cells. We systematically investigate
the transmission of EM waves in our PBG materials and discuss the mechanism of
band gap formation. We observe experimentally for the first time the wide band
gap with strong attenuation caused by direct destructive interference.

##  I Introduction

Dielectric structures with dielectric constants arranged periodically on the
wavelength scale can exhibit photonic band gaps (PBGs), in which the
propagation of electromagnetic (EM) waves are inhibited [1, 2]. When an
incident wave enters a PBGs, it decays rapidly, forming an attenuation mode, a
property that is of great importance for the modulation of EM waves [3, 4, 5].
Photonic crystals are conventional PBG materials with PBGs and have attracted
large interests both theoretically and experimentally [6, 7, 8, 9, 10, 11, 12,
13, 14, 15, 16]. However, direct measurement of EM waves in photonic crystals
is challenging [4, 17, 18, 13].

A periodic network structure consisting of waveguides has been proposed and
localization of states have been observed inside this material [19, 20]. Such
a waveguide network essentially introduces resonant loops, which can be
resonated and antiresonated in a certain frequency range [19, 20, 21, 22]. In
particular, the waveguide network made of coaxial cables is easy to implement
in experiments and allows for a more intuitive study of the EM wave
transmission inside the structure, as the amplitude and phase changes of the
nodes in each part of the network can be better measured, which will allow for
further in-depth study and understanding of the EM wave behavior in the PBGs
[19, 22, 20, 21].

It is also convenient to modify the network structure to have interesting
features. For example, it was theoretically found that the use of multiple
waveguides to connect the same pair of nodes in the network can lead to strong
attenuation of the wave propagation within the PBGs [23]. This motivates
further theoretical studies in various structures with interesting properties
[24, 25, 26, 27]. Optical waveguide networks can produce rich photon
attenuation modes [28], interesting comb optical transmission spectra [29,
30], extremely wide PBGs [23, 12, 21], super-photon localization [31, 32, 28,
24]. Some of the properties of optical waveguides are applied to
microcantilever sensor [33], passive optical device [34], and the physics of
${\mathcal{P}T}$-symmetry can also be investigated in it [35, 36, 37, 38].

In this paper, we experimentally design a one-dimensional (1D) networks system
constructed from coaxial cables, where multiple cables are used to realize
different paths for the EM waves such that the EM waves can have direct
interference within each unit cell [see Fig. 1]. Changing the relative length
ratios of the coaxial cables, we find that direct destructive interference
between different paths leads to a very wide PBG. Within this wide band gap,
the propagation of EM wave is strongly prohibited. Remarkably, under certain
conditions, the incident EM wave in a PBG can not travel through even one unit
cell and are almost totally reflected at the boundary of the material, because
of the strong attenuation caused by direct interference.

The paper is organized as follows: Section II describes the modeling and
methodology used to study our PBG material as well as our experimental setup.
Section III investigates the energy band structure and attenuation
characteristics of our material. We draw our conclusions in Sec. IV.

##  II Model, Theory, and Methods

###  II.1 Network Structure

Fig. 1(a) is a schematic structure of our PBG material made of connected
waveguides containing $N$ unit cells. In each cell there are two connected
waveguides with lengths ${{x}_{1}}$ and ${{x}_{2}}$ for direct interference,
and these interference structures are linked by waveguides of a length $d$. We
use coaxial cables to realize the waveguides for EM waves. Fig. 1(b)
illustrates a unit cell in which the cables are connected to each other by
connectors. The ports of the connectors can be used to measure the amplitudes
of the EM waves. Fig. 1(c) shows the structure of a coaxial cable, which
consists of a center conductor, a layer of insulating material, a mesh fabric
shield, and an outer sheath.

Compared with traditional photonic crystals, the structure of our networks is
easier to realize; the use of coaxial cables is more flexible from the
measurement point of view. In photonic crystals, EM waves are measured mainly
on the surface of the material [4, 16, 20], whereas in the waveguide networks,
one can measure the phase and amplitude of EM waves inside any unit cells,
thus bringing convenience to experimental studies.

Figure 1: Schematic diagram of the waveguide network model. (a) Schematic
structure of a one-dimensional coaxial cable network containing $N$ unit
cells. Each unit cell has two waveguides of different lengths ${{x}_{1}}$ and
${{x}_{2}}$ for interference. The structures for interference are linked by
coaxial cables of length $d$. An incident EM wave ${{E}_{i}}$ will have a
reflected (${{E}_{r}}$) and a transmitted (${{E}_{o}}$) parts. (b) We use
connectors to realize the structure of a cell of the model. (c) The structure
of the coaxial cable for our EM waveguides.

###  II.2 Transmission line networks

Pictures of our experimental samples are shown in Fig. 2, with Fig. 2(a)
showing a network system consisting of 5 unit cells. The connectors used for
our experiments are shown in Fig. 2(b). The red ports of the connector are
used to connect the upper and lower arms of the interference structure in each
unit cell, while the blue ports are used to connect the interference
structures of successive unit cells or for measurement of the EM waves. In our
experiments, we used a coaxial cable (type RG58C/U) with the waveguide length
shown in Fig. 2(c). The details of the cable-node connection can be seen in
the single-cycle unit cell formed by the connectors and the cables, as shown
in Fig. 2(d).

Figure 2: Transmission line networks. (a) Photograph of an experimental
sample, which has a size of $N=5$. The nodes are marked with red dots and the
red curves indicate the connectivity of the network. (b) Nodes formed by
connectors.Red circles mark the ports connecting the upper and lower arms of
the interference structure. Blue circles indicate the ports for connecting the
unit cells or for connecting measurement cables. (c) A coaxial cable waveguide
with connectors at the ends. (d) A unit cell consisting of cables and
connectors.

###  II.3 Equations for transmission line networks with single materials

For our networks system shown in Fig. 1,the propagation of EM waves in a
coaxial cable line satisfies the following homogeneous wave equation [20, 21]:

$\displaystyle{\frac{{{\partial}^{2}}{{\psi}_{nm}}(x)}{\partial{{x}^{2}}}=\frac{\varepsilon{{\omega}^{\text{2}}}}{{{c}_{0}}^{2}}{{\psi}_{nm}}(x),}$
(1)

where ${{\psi}_{nm}}(x)$ denotes the voltage wave function of any segment
between nodes $n$ and $m$, $x$ is the distance measured from node $m$,
$\omega=2\pi f$ is the angular frequency of EM wave, ${c}_{0}$ is the wave
speed of EM wave in vacuum,
$\varepsilon={\varepsilon}^{\prime}+i{\varepsilon}^{\prime\prime}$ is the
relative permittivity of the dielectric for the coaxial cables, and
$\varepsilon^{\prime}$ and $\varepsilon^{\prime\prime}$ are the real and
imaginary parts of the relative permittivity, respectively. The EM wave
function can be written as a linear combination of two plane waves traveling
in opposite directions [21, 23, 39, 25, 31]:

$\displaystyle{{\psi}_{nm}}={{\alpha}_{nm}}{{e}^{i\kappa
x}}+{{\beta}_{nm}}{{e}^{-i\kappa x}},$ (2)

where $i=\sqrt{-1}$ is the imaginary unit,
$\kappa=\frac{\omega}{c}\sqrt{\varepsilon}=\frac{\omega}{c}\sqrt{\varepsilon^{\prime}+i\varepsilon^{\prime\prime}}$
[20, 21]. In the case of $\varepsilon^{\prime\prime}\ll\varepsilon^{\prime}$,
it is deduced that $\kappa$ has the simple form that $\kappa\cong
k+(\text{1}/2L)i$, where
$k={\omega\sqrt{\varepsilon^{\prime\prime}}}/{{{c}_{0}}}\;$ and
$L={\varepsilon^{\prime}}/{k\varepsilon^{\prime\prime}}\;$ is the absorption
length. For any node $n$ of the network, its wave function is continuous at
the node [21, 23, 32, 25, 31]:

$\displaystyle\begin{array}[]{*{35}{l}}{{\left.{{\psi}_{nm}}\right|}_{x=0}}={{\psi}_{n}},\\\
{{\left.{{\psi}_{nm}}\right|}_{x={{x}_{nm}}}}={{\psi}_{m}},\end{array}$ (5)

where ${{\psi}_{n}}$ and ${{\psi}_{m}}$ represent the wavefunction at the
nodes $n$ and $m$, respectively. $x_{nm}$ is the length of a waveguide
connecting the nodes $n$ and $m$. With the use of Eqs. (2) and (5), one
obtains [25, 31, 32]:

$\displaystyle{{\psi}_{nm}}={{\psi}_{n}}\frac{\sinh\left[i\kappa({{x}_{nm}}-x)\right]}{\sinh(i\kappa{{x}_{nm}})}+{{\psi}_{m}}\frac{\sinh(i\kappa
x)}{\sinh(i\kappa{{x}_{nm}})},$ (6)

At each node $m$, the wave function is continuous and the derivative of the
wave function at node $m$ gives the flux conservation condition [23, 25, 32]:

$\displaystyle{{\left.\sum\limits_{n}{\frac{\partial{{\psi}_{nm}}(x)}{\partial
x}}\right|}_{x=0}}=0,$ (7)

where $n$ is the sum of all nodes directly connected to $m$. Substituting Eq.
(6) into Eq. (7), one gets [19, 20, 21, 39, 25, 23, 31]:

$\displaystyle-{{\psi}_{n}}\sum\limits_{m}{\coth(i\kappa{{x}_{nm}})}+\sum\limits_{m}{\frac{{{\psi}_{m}}}{\sinh(i\kappa{{x}_{nm}})}}=0.$
(8)

for the network consisting of coaxial cables without considering dissipation.
The coth and sinh denote hyperbolic tangent and hyperbolic sine functions,
respectively. We have added a constant $\gamma$ to the $i\kappa{{x}_{nm}}$
term of Eq. (8) to account for the additional losses caused by the connectors
if we consider the dissipation of the material.

We obtain the absorption length $L$ of the cables and the attenuation $d$ of
the connectors by using the calculated transmission coefficients which fit the
measured transmission coefficients in a single cell. Using the cable
specifications given by the vendor, as well as our experimental measurements
and data fitting, we obtained $L/{\rm{m}}=665{{(f/{\rm{MHz}})}^{-0.52}}$ and
$\gamma=-0.005$. We used $\varepsilon^{\prime}=2.3$.

Therefore, the transmission coefficient $t$ theoretically is calculated using
the generalized eigenfunction method to solve the equation, where $t$ is the
transmission amplitude of the outgoing wave. The transmittance is
${{\left|t\right|}^{2}}$ [25, 31, 32, 39, 21].

Figure 3: Band structures for different lengths of $x_{1}$, $x_{2}$, and $d$ ,
with the gray area indicating the band gap. Here we assume no absorption and
losses,i.e., $L=\infty$ and $\gamma=\text{0}$.

###  II.4 Generalized Floquet-Bloch ’s Theorem

The network equation describes the connection relationship between adjacent
nodes in a network. In order to quantitatively analyze the energy bands of the
system, we derive the dispersion relation of the network structure. To tackle
the problem that in our network system the length of a unit cell is not well
defined when the same pair of nodes in the unit cell are connected with cables
of different lengths, we use the generalized version of the Floquet-Bloch
theorem with the following Bloch function [23, 39, 25, 31, 32]:

$\displaystyle{\psi}_{n+T}({\Phi})=\psi_{n}({\Phi})e^{i{\Phi}T}.$ (9)

where the integer $n$ denotes an arbitrarily node index, the integer $T$ is
the period of the network configuration, and $\Phi$ is a dimensionless phase
factor. For the 0, 2, and 4 nodes in Fig. 1, it follows from Eq. (8):

$\left\\{\begin{aligned}
\begin{matrix}-{{\psi}_{1}}\left[\coth(i\kappa{{x}_{1}})+\coth(i\kappa{{x}_{2}})+\coth(i\kappa
d)\right]+[\frac{{{\psi}_{0}}}{\sinh(i\kappa
d)}+\frac{{{\psi}_{2}}}{\sinh(i\kappa{{x}_{1}})}+\frac{{{\psi}_{2}}}{\sinh(i\kappa{{x}_{2}})}]=0,\\\
-{{\psi}_{2}}\left[\coth(i\kappa{{x}_{1}})+\coth(i\kappa{{x}_{2}})+\coth(i\kappa
d)\right]+[\frac{{{\psi}_{3}}}{\sinh(i\kappa
d)}+\frac{{{\psi}_{1}}}{\sinh(i\kappa{{x}_{1}})}+\frac{{{\psi}_{1}}}{\sinh(i\kappa{{x}_{2}})}]=0,\\\
-{{\psi}_{3}}\left[\coth(i\kappa{{x}_{1}})+\coth(i\kappa{{x}_{2}})+\coth(i\kappa
d)\right]+[\frac{{{\psi}_{2}}}{\sinh(i\kappa
d)}+\frac{{{\psi}_{4}}}{\sinh(i\kappa{{x}_{1}})}+\frac{{{\psi}_{4}}}{\sinh(i\kappa{{x}_{2}})}]=0.\\\
\end{matrix}\end{aligned}\right.$ (10)

From Eq. (9), the wave functions at node 0 and node 4 are related to the wave
function at node 2 as [23, 24, 32]:

$\displaystyle\begin{array}[]{*{35}{l}}{{\psi}_{0}}={{\psi}_{2}}{{e}^{-i{\Phi}}},\\\
{{\psi}_{4}}={{\psi}_{2}}{{e}^{i{\Phi}}},\\\ \end{array}$ (13)

Substituting Eq. (13) into Eq. (10) gives

$\displaystyle\cos\Phi=f({{{\omega\sqrt{\varepsilon^{\prime}}}/{c}}_{0}})\equiv\frac{{{\text{A}}^{2}}{{\text{B}}^{2}}-C-2B\sinh(i\kappa
d)}{2B\left[\sinh(i\kappa{{x}_{1})}+\sinh(i\kappa{{x}_{2}})\right]},$ (14)

where

$\displaystyle\begin{array}[]{*{35}{l}}A=\coth(izd)+\sum\limits_{i=1}^{2}{\coth(iz{{x}_{i}})},\\\
B=\sinh(izd)\prod\limits_{i=1}^{2}{\sinh(iz{{x}_{i}})},\\\
C={{\sinh}^{2}}(iz{{x}_{1}}){{\sinh}^{2}}(iz{{x}_{2}})\\\
+{{\sinh}^{2}}(iz{{x}_{1}}){{\sinh}^{2}}(izd)+{{\sinh}^{2}}(iz{{x}_{2}}){{\sinh}^{2}}(izd).\end{array}$
(18)

From Eq. (14) we obtain the band structure of our networks. Because of Eq.
(9), when the solution $\Phi$ is a complex number with a non-zero imaginary
part $\rm{Im}(\Phi)$, the propagation of EM wave among the network nodes has a
decay rate given by the amplitude of $\rm{Im}(\Phi)$. That is, we have a band
gap when $\Phi$ has no real solution in Eq. (14). When the amplitude of
$\rm{Im}(\Phi)$ is larger, the propagation of EM wave among the network nodes
has a stronger attenuation. On the other hand, a real $\Phi$ solution
corresponds to a propagation mode among the network nodes. Therefore, we
obtain the dispersion relation for the energy bands from the existence of
solutions with a real $\Phi$ [23].

Figure 4: The experimentally measured transmittance $|t|^{2}$, as well as the
theoretically calculated values for the networks S1, S2 and S3 are shown in
Fig. 4. The results in Fig. 4 show that the transmittance at the PBGs is
getting smaller for a larger size $N$ of the network. It is more interesting
to see that the attenuation of EM waves within the PBGs is significantly
larger for the network S3, which has the ratio $x_{1}/x_{2}=2$, when comparing
with other networks S1 and S2 which have $x_{1}/x_{2}=1$. This shows that the
destructive interference between the two arms $x_{1}$ and $x_{2}$
significantly enhances the formation of PBGs. The results are consistent with
the band structures in Fig. 3

##  III Results and discussion

###  III.1 Observation of large PBGs

The structure of our network allows us to tune the interference effect between
the two arms by changing the lengths $x_{1}$ and $x_{2}$ (see Fig. 1). Without
lost of generality, we assume $x_{1}\geq x_{2}$. Furthermore, changing the
length $d$ of the cables that connect the interference structures together, we
can also modify the scattering among different cells. From Eq. (14) and Fig.
3, we can see that the band structure changes with the ratio between $x_{1}$
and $x_{2}$ as well as the length $d$. The forbidden bands are indicated by
gray areas in Fig. 3. As demonstrated in Fig. 3, the ratio between the lengths
$x_{1}$ and $x_{2}$ of the two arms has a large influence on the PBGs [23, 24,
25].

For the case of $x_{1}=x_{2}$ there is no destructive interference, and
therefore when $d=0$ there is no PBG. When $d>0$ there are small PBGs even
$x_{1}=x_{2}$, which is due to scattering among different cells. When
$x_{1}/x_{2}=2$, there is a strong destructive interference, and as a
consequence, a large band gap is formed.

###  III.2 Experimental demonstration of transmittance

To experimentally probe the system, we connected an Agilent E4438C vector
signal generator at one side of the network to input voltage waves. An
oscilloscope was connected at the other side of the network to measure the
transmission coefficients and intensities. Based on the theoretical results
shown in Fig. 1, we prepared different samples of the networks with three
different sets of parameters, namely, Structure 1 (S1) with
$({{x}_{1}},{{x}_{2}},d)=(1,1,1)$ m, S2 with $({{x}_{1}},{{x}_{2}},d)=(1,1,2)$
m, and S3 with $({{x}_{1}},{{x}_{2}},d)=(2,1,1)$ m.

The experimentally measured transmittance $|t|^{2}$, as well as the
theoretically calculated values for the networks S1, S2 and S3 are shown in
Fig. 4. The results in Fig. 4 show that the transmittance at the PBGs is
getting smaller for a larger size $N$ of the network. It is more interesting
to see that the attenuation of EM waves within the PBGs is significantly
larger for the network S3, which has the ratio $x_{1}/x_{2}=2$, when comparing
with other networks S1 and S2 which have $x_{1}/x_{2}=1$. This shows that the
destructive interference between the two arms $x_{1}$ and $x_{2}$
significantly enhances the formation of PBGs. The results are consistent with
the band structures in Fig. 3.

Figure 5: (a) IPR spectra for the S1 networks with different values of $N$.
(b) as (a) but for the S3 networks.

###  III.3 Wave Intensity Distribution at Characteristic Frequencies

In order to investigate the propagation modes of EM waves, we have
systematically probed the S1 and S3 networks with various periods $N=5$, 12,
and 25. Firstly, we use the inverse participation ratio [40, 20, 39]:

${\text{IPR}={{(\sum\nolimits_{n}{\left|{{\psi}_{n}}^{2}\right|})}^{2}}}/{\sum\nolimits_{n}{{{\left|{{\psi}_{n}}\right|}^{4}}}}\,$
(19)

to investigate the wave intensity distribution. For the propagation modes, the
value of IPR increases with the size (i.e., $N$) of the network [20]. While
for the case that the frequency of a EM wave lying within a PBG, the value of
IPR hardly changes with the network size.

We plot the IPR spectral distributions in Fig. 5, where we can see that for
frequencies near the band edge, IPR increases with the size of the networks.
As the frequency enters the gap, the IPR becomes smaller and the size
dependence becomes smaller.

Figure 6: Intensity of the wave distribution for $N=5$ for different
frequencies of incident EM waves in an S1 network. (a) is the simulated
results while (b) is the results of experiment. Figure 7: Intensity of the
wave distribution for $N=5$ for different frequencies of incident EM waves in
an S3 network. (a) is the simulated results while (b) is the results of
experiment.

To have a clear picture of the EM wave distribution inside the network, we
further conducted experiments to measure the intensities of the EM waves of
all the nodes in the networks, for the EM wave frequencies indicated in Fig.
5. We show in Figs. 6 and 7 the voltage intensities of all the nodes for the
networks S1 (with $x_{1}=x_{2}$) and S3 (with $x_{1}=2x_{2}$), respectively.
One can see that the theoretical results agree with the experimental values.
The results show that at the frequencies when the IPR increases with the
network size $N$, the energy of EM wave have a broad distribution, which
indicates an extended state. At the frequencies when the IPR is not sensitive
to the network size, the intensities are negligible over a broad range of
nodes, which suggests an evanescent mode [28].

Note that as we can see in Fig. 7 for the network S3, which has
$x_{1}=2x_{2}$, an ultra strong attenuation of EM wave within a PBG is
observed. In particular, for the frequency $f_{23}=99$ MHz close to the center
of the PBG, all the nodes within the network have negligible intensities,
because the amplitude of the attenuation mode attenuates to zero even within
the first unit cell and all the EM wave energy is reflected at the input side.
This ultra strong attenuation is due to the direct destructive interference
between the two arms of lengths $x_{1}$ and $x_{2}$. For the network S1, which
has $x_{1}=x_{2}$, the attenuation of EM wave within PBGs is weaker, when one
compares the results in Figs. 6 and 7 as well as the plots in Fig. 8.

Figure 8: Simulated wave intensity distribution with N=12 for different
frequencies of incident EM waves. (a) Intensity distribution map for the S1
network; (b) Intensity distribution map for the S3 network.

The results show that the PBG due to direct interference between the
waveguides within each cell is quite different from the conventional PBGs due
to scattering among different cells and provides a much stronger attenuation
of EM waves.

Figure 9: Calculated wave intensity distribution as a function of the location
$x$ relative to node $1$ inside a S3 network with $N=5$ for a wave frequency
$f_{23}=99$ MHz. The length between the input node 0 and node 1 is 1 m.

To further investigate the strong attenuation mechanism of the network of S3
at the frequency $f_{23}=99$ MHz, in Fig. 9 we calculated the wave intensity
distribution between the signal generator output and the first contact of the
network using the theory of Eq. (6). We find that the wave intensities in the
coaxial cable from the signal output to the first node of the network are all
less than 0.0021. Meanwhile, we performed experimental verification and found
that the voltage waves are indeed attenuated rapidly in the first section of
the coaxial cable. It is clear that this is not due to the geometric
scattering structure in the network, but is the result of the interference
between the incident and reflected waves in a single coaxial cable.

##  IV Conclusions

We have theoretically and experimentally study a PBG material made of
multiconnected coaxial cables. We have observed for the first time large PBGs
which is due to direct wave interference within each unit cell. From the
measured transmission spectra and intensities of EM waves inside the networks,
we found that direct interference between two waveguides within the same unit
cell provides a new mechanism of PBG formation. In particular, when the length
ratio between the two arms $x_{1}/x_{2}=2$, a PBG with ultra strong
attenuation of EM waves is formed due to direct destructive interference. The
strong attenuation within the PBG induces an ultra strong reflection of EM
waves at the boundary of the network. Our results demonstrate a new way to
realize PBGs and may have applications to the control of various waves (e.g.,
EM waves or acoustic waves). It would also be interesting to study how the
multiconnected waveguides would influence the topological transport in
transmission line network [41].

###### Acknowledgements.

This work was supported by the National Natural Science Foundation of China
(Grant Numbers 11674107, 61475049, 11775083, 61774062, 61771205, 12074131) and
the Natural Science Foundation of Guangdong Province, China (Grant No.
2021A1515012030).

## References

  * Yablonovitch [1987] E. Yablonovitch, Inhibited spontaneous emission in solid-state physics and electronics, Phys. Rev. Lett. 58, 2059 (1987).
  * John [1987] S. John, Strong localization of photons in certain disordered dielectric superlattices, Phys. Rev. Lett. 58, 2486 (1987).
  * Askitopoulos  _et al._ [2011] A. Askitopoulos, L. Mouchliadis, I. Iorsh, G. Christmann, J. J. Baumberg, M. A. Kaliteevski, Z. Hatzopoulos, and P. G. Savvidis, Bragg polaritons: Strong coupling and amplification in an unfolded microcavity, Phys. Rev. Lett. 106, 076401 (2011).
  * Felbacq and Smaâli [2004] D. Felbacq and R. Smaâli, Bloch modes dressed by evanescent waves and the generalized goos-hänchen effect in photonic crystals, Phys. Rev. Lett. 92, 193902 (2004).
  * Armenise  _et al._ [2010] M. N. Armenise, C. E. Campanella, C. Ciminelli, F. Dell’Olio, and V. M. Passaro, Phononic and photonic band gap structures: modelling and applications, Physics Procedia 3, 357 (2010).
  * Povinelli  _et al._ [2001] M. L. Povinelli, S. G. Johnson, S. Fan, and J. D. Joannopoulos, Emulation of two-dimensional photonic crystal defect modes in a photonic crystal with a three-dimensional photonic band gap, Phys. Rev. B 64, 075313 (2001).
  * Foteinopoulou [2012] S. Foteinopoulou, Photonic crystals as metamaterials, Physica B Condens. Matter 407, 4056 (2012).
  * Lodahl  _et al._ [2004] P. Lodahl, A. Floris van Driel, I. S. Nikolaev, A. Irman, K. Overgaag, D. Vanmaekelbergh, and W. L. Vos, Controlling the dynamics of spontaneous emission from quantum dots by photonic crystals, Nature 430, 654 (2004).
  * Pradhan and Watson [1999] R. D. Pradhan and G. H. Watson, Impurity effects in coaxial-connector photonic crystals: A quasi-one-dimensional periodic system, Phys. Rev. B 60, 2410 (1999).
  * Haché and Poirier [2002] A. Haché and L. Poirier, Anomalous dispersion and superluminal group velocity in a coaxial photonic crystal: Theory and experiment, Phys. Rev. E 65, 036608 (2002).
  * Haché and Poirier [2002] A. Haché and L. Poirier, Long-range superluminal pulse propagation in a coaxial photonic crystal, Appl Phys Lett. 80, 518 (2002).
  * Cheung  _et al._ [2004] S.-K. Cheung, T.-L. Chan, Z.-Q. Zhang, and C. T. Chan, Large photonic band gaps in certain periodic and quasiperiodic networks in two and three dimensions, Phys. Rev. B 70, 125104 (2004).
  * D’orazio  _et al._ [2003] A. D’orazio, M. D. Sario, V. Petruzzelli, and F. Prudenzano, Photonic band gap filter for wavelength division multiplexer, Opt. Express 11 3, 230 (2003).
  * Tufarelli  _et al._ [2012] T. Tufarelli, F. Ciccarello, and M. Kim, Dynamics of spontaneous emission in a single-end photonic waveguide, Phys. Rev. A 87 (2012).
  * Ao  _et al._ [2009] X. Ao, Z. Lin, and C. T. Chan, One-way edge mode in a magneto-optical honeycomb photonic crystal, Phys. Rev. B 80, 033105 (2009).
  * Fan  _et al._ [2013] Y. Fan, Z. Wei, H. Li, H. Chen, and C. M. Soukoulis, Photonic band gap of a graphene-embedded quarter-wave stack, Phys. Rev. B 88, 241403 (2013).
  * Povinelli  _et al._ [2003] M. L. Povinelli, S. G. Johnson, J. D. Joannopoulos, and J. B. Pendry, Toward photonic-crystal metamaterials: Creating magnetic emitters in photonic crystals, Appl Phys Lett 82, 1069 (2003).
  * Rao and Hughes [2007] V. S. C. M. Rao and S. Hughes, Single quantum dot spontaneous emission in a finite-size photonic crystal waveguide: Proposal for an efficient “on chip” single photon gun, Phys. Rev. Lett. 99, 193901 (2007).
  * Zhang and Sheng [1994] Z.-Q. Zhang and P. Sheng, Wave localization in random networks, Phys. Rev. B 49, 83 (1994).
  * Zhang  _et al._ [1998] Z. Q. Zhang, C. C. Wong, K. K. Fung, Y. L. Ho, W. L. Chan, S. C. Kan, T. L. Chan, and N. Cheung, Observation of localized electromagnetic waves in three-dimensional networks of waveguides, Phys. Rev. Lett. 81, 5540 (1998).
  * Li  _et al._ [2000] M. Li, Y. Liu, and Z.-Q. Zhang, Photonic band structure of sierpinski waveguide networks, Phys. Rev. B 61, 16193 (2000).
  * Ciminelli  _et al._ [2005] C. Ciminelli, F. Peluso, and M. N. Armenise, Parametric analysis of 2d guided-wave photonic band gap structures, Opt. Express 13, 9729 (2005).
  * Wang and Yang [2007] Z.-Y. Wang and X. Yang, Strong attenuation within the photonic band gaps of multiconnected networks, Phys. Rev. B 76, 235104 (2007).
  * Xu  _et al._ [2021] X. Xu, X. Yang, D. Deng, H. Liu, and Z. Wei, Ultrawide photonic bandgap and ultrastrong photonic localization produced by series of periodic networks, Ann Phys 533, 2000584 (2021).
  * Cai  _et al._ [2011] L. Cai, X. Yang, and J. Lu, Large photonic band gap and strong attenuation of multiconnected sierpinski network, J. Electromagn. Waves Appl. 25, 147 (2011).
  * Tang  _et al._ [2014] Z. Tang, X. Yang, J. Lu, and C. T. Liu, Super-strong photonic localization in symmetric two-segment-connected triangular defect waveguide networks, Opt. Commun 331, 53 (2014).
  * Zheng  _et al._ [2019] J. Zheng, X. Yang, X. Zhang, D. Deng, and H. Liu, Transmission characteristics of one-dimensional periodic optical waveguide networks, Phys. Rev. A 99, 023809 (2019).
  * Xu  _et al._ [2015] X. Xu, X. Yang, S. Wang, T. C. yi Liu, and D. Deng, Sufficient condition for producing photonic band gaps in one-dimensional optical waveguide networks, Opt. Express 23 21, 27576 (2015).
  * Yang  _et al._ [2013] X. Yang, H. Song, and T. C. Liu, Comb-like optical transmission spectrum resulting from a four-cornered two-waveguide-connected network, Phys. Lett. A 377, 3048 (2013).
  * Wang  _et al._ [2014] Y. Wang, X. Yang, J. Lu, G. Zhang, and C. T. Liu, Comb-like optical transmission spectra generated from one-dimensional two-segment-connected two-material waveguide networks optimized by genetic algorithm, Phys. Lett. A 378, 1200 (2014).
  * Lu  _et al._ [2012] J. Lu, X. Yang, and L. Cai, Large photonic band gap and strong attenuation of multiconnected peano network, Opt. Commun 285, 459 (2012).
  * Xiao  _et al._ [2012] Q. Xiao, X. Yang, J. Lu, and C. Liu, Huge photonic band gaps with strong attenuations resulted from quasi-one-dimensional waveguide networks composed of triangular fundamental loops, Opt. Commun 285, 3775 (2012).
  * Jing  _et al._ [2020] Y. Jing, G. Fan, R. Wang, Z. Zhang, M. Wang, X. Cai, J. Wei, X. Chen, H. Li, and Y. Li, Analysis for an improved nanomechanical microcantilever sensor on optical waveguides, IEEE Access 8, 63856 (2020).
  * Asnawi  _et al._ [2021] Asnawi, N. A. Siddiq, M. S. Muntini, and Y. H. Pramono, Flexible and low-cost fabrication of optical waveguides based on sno2 for passive optical devices, J. Phys. Conf Ser 1805, 012043 (2021).
  * Wu  _et al._ [2019] H. Wu, X. Yang, D. Deng, and H. Liu, Reflectionless phenomenon in $\mathcal{PT}$-symmetric periodic structures of one-dimensional two-material optical waveguide networks, Phys. Rev. A 100, 033832 (2019).
  * Zhi  _et al._ [2018] Y. Zhi, X. Yang, J. Wu, S. Du, P. Cao, D. Deng, and C. T. Liu, Extraordinary characteristics for one-dimensional parity-time-symmetric periodic ring optical waveguide networks, Photon. Res. 6, 579 (2018).
  * Wang  _et al._ [2022] Q. Wang, X. Yang, D. Deng, H. Liu, and Z. Wei, Quasi-blackbody, bidirectional super reflection, and new total reflection produced by periodic optical waveguide networks based on three-order $\mathcal{PT}$-symmetric substructures, Ann Phys 534, 2200207 (2022).
  * Wu and Yang [2017] J. Wu and X. Yang, Ultrastrong extraordinary transmission and reflection in pt-symmetric thue-morse optical waveguide networks, Opt. Express 25, 27724 (2017).
  * Hu  _et al._ [2017] X. Hu, X. Yang, and D. Deng, Super-strong photonic localizations in symmetric defect waveguide-ring networks, IEEE Photon. J 9, 1 (2017).
  * Edwards and Thouless [1972] J. T. Edwards and D. J. Thouless, Numerical studies of localization in disordered systems, J. Phys. C 5, 807 (1972).
  * Jiang  _et al._ [2019] T. Jiang, M. Xiao, W.-J. Chen, L. Yang, Y. Fang, W. Y. Tam, and C. T. Chan, Experimental demonstration of angular momentum-dependent topological transport using a transmission line network, Nat. Commun. 10, 434 (2019).


# Flares from Space Debris in LSST Images

Abraham Loeb  Astronomy Department, Harvard University, 60 Garden St.,
Cambridge, MA 02138, USA

###### Abstract

Owing to the exceptional sensitivity of the Vera C. Rubin Observatory, we
predict that its upcoming LSST images will be contaminated by numerous flares
from centimeter-scale space debris in Low Earth Orbits (LEO). Millisecond-
duration flares from these LEO objects are expected to produce detectable
image streaks of a few arcseconds with AB magnitudes brighter than 14.

##  1 Introduction

The European Space Agency (ESA)
reports111https://www.esa.int/Space_Safety/Space_Debris/Space_debris_by_the_numbers
that as of December 6, 2023, the space debris in orbit around Earth includes
$1.3\times 10^{8}$ objects in the size range of 0.1-1 cm, $\sim 10^{6}$
objects between 1-10 cm and $3.65\times 10^{4}$ objects larger than 10 cm. A
subset of these objects is in Low Earth Orbits (LEO) with a semi-major axis
below an altitude of $2\times 10^{3}~{}{\rm km}$. In this Note, we examine the
implications of this LEO debris for the upcoming Legacy Survey of Space & Time
(LSST) of the Vera C. Rubin Observatory in Chile (Schildknecht, 2007; Ivezić
et al., 2019; Bosch et al., 2019; Esteves et al., 2023; Hernandez et al.,
2023; Schwamb et al., 2023).

##  2 Method

We define the average albedo of an object of radius $R$ and distance $d$ which
is illuminated on one hemisphere by sunlight by the following fraction of
light reflected from it,

$A\left({\pi R^{2}\over 2\pi d^{2}}\right)~{}.$ (1)

Hereafter, we encapsulate the unknown surface properties of the object as well
as the orientation and geometry of the source relative to the Sun and the
observer in the value of $A$, for which we adopt a fiducial value of 0.1.
Given the average solar illumination of $f=1.4\times 10^{6}~{}{\rm
erg~{}s^{-1}}$ at a characteristic photon frequency of $\nu\sim 6\times
10^{14}~{}{\rm Hz}$, we calculate the AB magnitude of the object to be,

${\rm AB}=-2.5\log_{10}\left[{(df/d\nu)\over{\rm
erg~{}s^{-1}~{}cm^{-2}~{}Hz^{-1}}}\right]-48.6=16.2-5\log_{10}\left[\left({R_{0}\over
d_{8}}\right)\sqrt{A_{-1}}\right],$ (2)

where $A_{-1}=(A/0.1)$, $R_{0}=(R/1~{}{\rm cm})$ and $d_{8}=(d/10^{8}~{}{\rm
cm})$.

##  3 Results

Data from the Zwicky Transient Facility (ZTF) shows that the sunlight glints
from known LEO satellites generate flashes of duration $10^{-3\pm 0.5}~{}{\rm
s}$ (see Figure 4 in Karpov & Peloton (2023), and also Corbett et al. (2020);
Nir et al. (2021a, b)). Given a typical orbital speed of $\sim 8~{}{\rm
km~{}s^{-1}}$, these flash durations translate to streaks of angular length
$\sim(10^{-3}~{}{\rm s}\times 8~{}{\rm km~{}s^{-1}})/10^{3}d_{8}~{}{\rm
km})=1.7d_{8}^{-1}$ arcseconds. The light from the flares is therefore
expected to spread across no more than a few arcseconds, independently of the
LSST exposure time which is 4 orders of magnitude longer.

For a few millisecond flares, LSST at the Rubin Observatory (Ivezić et al.,
2019) is expected to have a brightness sensitivity of ${\rm AB}\sim 14$ (given
the survey’s $\sim 25$-magnitude limit for 30 second exposures; see Figure 5
in Karpov & Peloton (2023)). According to equation (2), this implies that LSST
will detect all space debris objects in LEO that satisfy,

$\left({R_{0}\over d_{8}}\right)\sqrt{A_{-1}}\gtrsim 3.$ (3)

This includes all LEO objects with a radius $R\gtrsim(3/A_{-1})~{}{\rm cm}$ at
a distance of $d\sim 10^{3}~{}{\rm km}$.

##  4 Discussion

Assuming that merely a tenth of the $\sim 10^{6}$ space debris objects in the
size range of 1-10 cm reported by ESA satisfy the condition in equation (3),
we find that LSST images will be contaminated by flares from $\sim 10^{5}$
objects that repeat every LEO orbital time of 1.5-2 hours (and are most
visible during the evening and morning twilight). This number exceeds by an
order of magnitude the number of large satellites currently in orbit around
Earth. Out of the entire debris population, only $3.515\times 10^{4}$ objects
are regularly tracked and catalogued by Space Surveillance
Networks.222https://www.esa.int/Space_Safety/Space_Debris/Space_debris_by_the_numbers

So far, the LSST team contemplated a novel strategy to mitigate the impact of
large commercial satellite constellations in LEO (Hu et al., 2022). However,
the above numbers suggest that image contamination by untracked space debris
might pose a bigger challenge.

## Acknowledgements

This work was supported in part by Harvard’s Black Hole Initiative, which is
funded by grants from JFT and GBMF.

## References

  * Bosch et al. (2019) Bosch, J., AlSayyad, Y., Armstrong, R., et al. 2019, in Astronomical Society of the Pacific Conference Series, Vol. 523, Astronomical Data Analysis Software and Systems XXVII, ed. P. J. Teuben, M. W. Pound, B. A. Thomas, & E. M. Warner, 521, doi: 10.48550/arXiv.1812.03248
  * Corbett et al. (2020) Corbett, H., Law, N. M., Soto, A. V., et al. 2020, ApJ, 903, L27, doi: 10.3847/2041-8213/abbee5
  * Esteves et al. (2023) Esteves, J. H., Utsumi, Y., Snyder, A., et al. 2023, PASP, 135, 115003, doi: 10.1088/1538-3873/ad0a73
  * Hernandez et al. (2023) Hernandez, F., Beckett, G., Clark, P., et al. 2023, arXiv e-prints, arXiv:2311.13981, doi: 10.48550/arXiv.2311.13981
  * Hu et al. (2022) Hu, J. A., Rawls, M. L., Yoachim, P., & Ivezić, Ž. 2022, ApJ, 941, L15, doi: 10.3847/2041-8213/aca592
  * Ivezić et al. (2019) Ivezić, Ž., Kahn, S. M., Tyson, J. A., et al. 2019, ApJ, 873, 111, doi: 10.3847/1538-4357/ab042c
  * Karpov & Peloton (2023) Karpov, S., & Peloton, J. 2023, Contributions of the Astronomical Observatory Skalnate Pleso, 53, 69, doi: 10.31577/caosp.2023.53.4.69
  * Nir et al. (2021a) Nir, G., Ofek, E. O., Ben-Ami, S., et al. 2021a, MNRAS, 505, 2477, doi: 10.1093/mnras/stab1437
  * Nir et al. (2021b) Nir, G., Ofek, E. O., & Gal-Yam, A. 2021b, Research Notes of the American Astronomical Society, 5, 27, doi: 10.3847/2515-5172/abe540
  * Schildknecht (2007) Schildknecht, T. 2007, A&A Rev., 14, 41, doi: 10.1007/s00159-006-0003-9
  * Schwamb et al. (2023) Schwamb, M. E., Jones, R. L., Yoachim, P., et al. 2023, ApJS, 266, 22, doi: 10.3847/1538-4365/acc173


# Mixing Modes: Active and Passive Integration of Speech, Text, and
Visualization for Communicating Data Uncertainty

Chase Stokes1,20000-0001-7644-9021, Chelsea Sanker30000-0001-6106-9587,
Bridget Cogley40009-0002-0125-5171, and Vidya Setlur10000-0003-3722-406X
1Tableau Research, Palo Alto, USA 2University of California, Berkeley, USA
3Stanford University, Palo Alto, USA 4Versalytix, USA

###### Abstract

Interpreting uncertain data can be difficult, particularly if the data
presentation is complex. We investigate the efficacy of different modalities
for representing data and how to combine the strengths of each modality to
facilitate the communication of data uncertainty. We implemented two
multimodal prototypes to explore the design space of integrating speech, text,
and visualization elements. A preliminary evaluation with 20 participants from
academic and industry communities demonstrates that there exists no one-size-
fits-all approach for uncertainty communication strategies; rather, the
effectiveness of conveying uncertain data is intertwined with user preferences
and situational context, necessitating a more refined, multimodal strategy for
future interface design. Materials for this paper can be found on OSF.

<ccs2012> <concept> <concept_id>10003120.10003145</concept_id>
<concept_desc>Human-centered computing Visualization</concept_desc>
<concept_significance>500</concept_significance> </concept> <concept>
<concept_id>10003120.10003121.10003128</concept_id> <concept_desc>Human-
centered computing Interaction techniques</concept_desc>
<concept_significance>500</concept_significance> </concept> </ccs2012>

[500]Human-centered computing Visualization [500]Human-centered computing
Interaction techniques

††volume: 43††issue: 3

##  1 Introduction

Communicating data uncertainty can be challenging, especially for audiences
with limited statistical expertise [MRH∗05, Hul19]. However, accurately
communicating uncertainty is critical for decision-making, as it could impact
risk assessments.

Prior work has explored visual representations for uncertain data [KKHM16,
FWM∗18, PPKH21]. Other modes of communication, such as text or speech
explanations, can mitigate some of these issues but have limitations. For
instance, audiences with limited graphical literacy more easily interpret
text, but text cannot convey statistical information. [Sch97]. Speech conveys
tone not present in written text, but is ephemeral and poses challenges for
conveying complex information [SW95]. Integrating visual, textual, and speech
elements may use one method’s strengths to offset another’s limitations,
facilitating more effective data communication.

Depending on the user’s context, different modes may be used for decision-
making. For example, text-to-speech features can be useful when the user’s
hands are occupied, such as while driving. However, these passive contexts
limit users’ ability to further interact with the data. We implemented two
multimodal interface types: passive, which provided an integrated presentation
of information while minimizing user effort, and active, which emphasized
user-driven interaction and in-depth exploration [EFN12]. We then completed a
preliminary evaluation of the prototypes, focusing on design implications and
future research directions.

##  2 Related Work

Figure 1: Diagram showing multimodal components and possible user interactions
with passive and active interfaces.

###  2.1 Representations of Uncertainty

A range of studies have examined the use of uncertainty visualizations, from
surveys [BHJ∗14, PKH21, SLSR08] to specialized tools [JED∗20] and libraries
[Kay23]. This research includes applications such as weather forecasting
[PPKH21] and public transportation [FWM∗18, KKHM16]. Different kinds of
uncertainty visualizations have been evaluated for decision-making [PPKH21,
FWM∗18, KKHM16].

Text representations of uncertainty often use hedges to signal uncertainty,
such as ‘might’ or ‘possibly’ [Lak73, SVF∗12]. These hedges can reflect either
inherent uncertainty in the data being described or the author’s uncertainty
about it. Text descriptions can also convey statistics about uncertain
outcomes.

Hedges are also used in speech along with phonetic patterns like speech rate,
pauses, and pitch [SLW73, JP15]. Uncertain speech often has rising intonation
[SC93], slower speech rate [SLW73, SC93], and more frequent pauses [SLW73].
Some studies find a higher pitch associated with lower confidence [JP15],
while others find a lower pitch associated with lower confidence [SLW73].
These acoustic characteristics can also impact how confident or uncertain a
listener deems the speaker [KLSG22].

Building upon this rich body of research, our work further explores the
integration of speech, text, and visualization through the implementation of
multimodal prototypes for both passive and active consumption of uncertainty
communication.

###  2.2 Multimodal Techniques for Communicating Information

Existing research has explored how multimodal interaction can blend various
modes of communication, such as visual, auditory, and haptic cues, to
effectively convey information. Bromley and Setlur [BS23] explored the use of
language with visualizations by employing hedges to describe trends (e.g.,
“gradual fall”).

Multimodal techniques can be useful for conveying uncertainty to users who may
be blind or have low vision. Sharif et al. [SZW23] provide empirical findings
from semi-structured interviews with 16 screenreader users to identify user
preferences in uncertainty visualizations. Several studies explore how non-
speech audio can be used to represent data, either on its own or in
combination with visualizations [LWS96, LBH∗97, Bea11]. Previous work has also
explored differences between passive and active communication of data
insights. Both variants result in similar levels of comprehension [RSK∗21];
interactivity can increase user enjoyment [DAV08], while static visualizations
offer a more direct message [MG18].

##  3 Prototype Implementation

Our work builds upon this research to explore the research question, “What are
user preferences for multimodal presentations of uncertain data?” We
implemented two prototypes with passive and active interface behaviors. Design
choices were made in consultation with a UX designer at a visual analytics
company with 10+ years of experience and one of the co-authors who is an
interactive visualization designer with 15+ years of experience. Prototype
design was iterative, with feedback provided on four occasions.

###  3.1 Decision-making Context

The interfaces studied here were situated in a decision-making context. Users
assumed the role of a road maintenance company contracted to treat roads with
salt to prevent icing. Users were tasked with applying salt to the roads when
the temperature was at or below 32ºF (0ºC) to prevent ice from forming. They
were informed that salt supplies were limited, so maintenance companies must
balance cost (and supply) with damage prevention. To make this decision, they
viewed and listened to a forecast depicting predicted temperature lows for a
given evening.

The appropriateness of a user’s decision to salt the roads depends on the
costs of salting and potential damages; information not provided in this
scenario. The decision context was used to provide a situation of use for the
interface rather than to assess rationality. Prior work examined each mode of
information in decision-making, finding that speech led to higher trust in
information, while text was associated with lower decision confidence
[SSCS24]. This related paper studied uncertainty data in the same decision
context as our work, but differed in research questions and methodology.

Figure 2:  Insights from comparing active and passive interfaces. Different
features may be better suited for specific interfaces, while some user
interactions are consistent.

###  3.2 System Components

The interface flow for both prototypes is shown in Figure 1. Each prototype
employs visualizations, text descriptions with hedges, and speech elements
with pauses and rate changes to convey data uncertainty. The speech forecast
is displayed at the top of the page. A D3 visualization [BOH11] and a text
description of uncertainty are shown below the MP3 player. An event listener
in the interface monitors time progression in the speech module, triggering
animated updates in the visualization and text modules. In addition, the
active prototype leverages interaction with the visual and textual modes,
including additional linking between the two.

####  3.2.1 Data Module

Raw distribution data is integrated into a Node.js [Ope23] application using
an Express framework, serving the dataset as a static file. Upon loading the
prototype, a random trial is selected from the dataset. This selection process
links the columns of the .csv file to the unique trial number. Dataset values
in the .csv are derived by selecting 100 points from a normal distribution,
such that the resulting distributions may not have been normally distributed.
This variation provided more ecologically valid distributions, as natural data
tends to not always follow a normal distribution.

Text templates and timings also serve as a static JavaScript file, including
each sentence and the timestamp it occurs in the speech element. The text-to-
speech synthesis engine computes the timing information while generating
speech from the SSML syntax. These timings are available as static files.

####  3.2.2 Multimodal Modules

Visualization module. To communicate data uncertainty in visualizations, we
employ density and 100-quantile dot plots [KKHM16]. Density plots are created
using a kernel density estimator to determine the continuous probability curve
from discrete data points. The quantile dot plot is created using a histogram,
with 20 evenly spaced bins based on the range of the x-axis scale.

Text module. The text descriptions are generated with natural language
templates containing hedges and summary statistics from the visualized dataset
[SSCS24]. These statistics include the mean of the distribution, the range of
the middle 50% of data, the full range of the data, and a verbal
representation of distribution skew. The skew value is computed using the
skewness function in R [KN22], then mapped to magnitudes (“slightly” to
“significantly”) and a positive or negative direction. Hedges were included in
each sentence to communicate uncertainty (e.g., ‘might,’ ‘could’). A standard
black color is used to render the text, with a gray color (#757575) applied to
hedges to indicate a higher level of uncertainty. Colors employed in the
prototypes are compliant with the WCAG AA guidelines for color contrast
[(W319].

Speech module. The text templates are translated into Google Speech Synthesis
Markup Language (SSML) [Goo24] to provide adjustments in pitch, rate of
speech, and pauses for communicating uncertainty. For hedges in the text, we
modify the speech to $65\%$ of the original rate and lower the pitch by $5\%$.
We apply the same pitch treatment to numerical values, slowing the speech to
$70\%$ and adding a $0.2$ second break before the values.

####  3.2.3 Interaction Module

In both prototypes, the forecast can be replayed to view and listen to the
information. During the replay, the animated sequence for the text and
visualization also repeats. The active prototype includes detail-on-demand
tooltips and linking between modes, shown in Figure 1. There are three
distinct hover interactions (hedges, numerical values, visualization) that
provide dynamic feedback and detailed insights about the data. A video
demonstrating these behaviors is included in the supplementary material.

When a user hovers over a hedge in the text description, the word visually
responds by “wobbling” at $3$º angles and applying a $0.5$px blur effect
Hovering over a numerical value in the text shows a tooltip featuring an icon
array visualizing the likelihood of the hovered number occurring within the
dataset. Simultaneously, the corresponding section of the visualization that
represents this number is highlighted and wobbles to draw attention and convey
uncertainty. Upon hovering over a mark in the visualization, a tooltip appears
with a text description of the cumulative likelihood of achieving the
corresponding value or lower.

##  4 Preliminary User Study

Using the passive and active prototypes as a design probe, we conducted a
preliminary evaluation to qualitatively assess the overarching idea of
combining speech, text, and visualization modes for communicating data
uncertainty information.

###  4.1 Method

We employed a within-subjects design to elicit subjective user feedback for
both interfaces. Participants engaged with both prototypes. Half of the
participants viewed the active prototype first; half viewed the passive.
Sessions lasted approximately 45 minutes and were recorded with the
participant’s consent. One author coded the transcripts in close consultation
with a second author.

For each interface, participants viewed the speech-linked animation,
accompanied by the narration and linked text transcript. They could replay the
animation and narration or interact with available features. They were
encouraged to think aloud to share their observations and reactions to the
interface. The facilitator then asked for specifics about the potential use of
the information and possible changes to the interface. After participants had
viewed both interfaces, the facilitator asked about their usage of the
different modes and their perception of each interface’s strengths and
weaknesses.

###  4.2 Participants

The $20$ participants ($8$ female, $12$ male) were fluent in English. Half
were graduate students from a nearby university, and half were professionals
employed at a technology enterprise with a variety of backgrounds (e.g.,
technical engineering, product management). Participants were recruited via
internal mailing lists and Slack channels. Graduate students were compensated
with a $20 Amazon gift card. Professionals were not compensated due to company
policy prohibiting payment. We refer to participants using [$P$#].

###  4.3 Study Findings

Code counts from the interview transcripts can be found in Figure 2, along
with supplementary and recruitment materials. Overall, participants used the
speech and text modes as supplementary to the visualization for both
interfaces. Interacting with the active interface was helpful for decision-
making and providing further detail on event likelihoods. We observed a slight
difference in interaction behaviors: passive-first participants replayed the
speech element more ($1.5$ times on average) than active-first ($0.6$ times).

Replay and navigation. Replay frequency and interaction with the speech
element were much higher for the passive interface ($17$ participants) than
the active interface ($8$ participants). When viewing the passive interface,
$12$ participants used the speech element to navigate between animation
stages, compared to $6$ for the active interface. Some of the participants
felt that there were too many elements in the interface ($13$ active, $11$
passive).

Trends in interaction. In the active interface, participants could choose the
visualization displayed (dot plot or density plot), switching an average of
$11.5$ times. $14$ participants preferred the density plot, citing familiarity
with the chart type. Each participant interacted with the density plot an
average of $6.9$ times, taking an average of $14.5$ seconds (s) per
interaction. Similarly, they interacted with the dot plot an average of $6.6$
times with an average of $10.2$s per interaction. The text component had fewer
interactions, an average of $5.2$ interactions per participant, but each
interaction was longer, averaging $30.0$s.

Decision-making. Participants frequently reported that they used
visualizations in the decision-making process ($17$ active, $15$ passive). The
interactions with the visualizations were particularly useful, with $13$
participants reporting that they relied primarily on information from the
visualization tooltips. According to $P5$, “Some of the interactions, if not
all of them, I feel, were more valuable… I got to the decision much faster.”

The text tooltips were less useful overall, with $9$ participants finding them
redundant and not decision-relevant. Rather than offering the likelihood of
going below a given temperature, as in the visualization tooltips, the text
tooltips offered the likelihood of being at a given temperature. $9$
participants found this relatively unhelpful, as this information did not
greatly inform or change their decisions. However, $9$ participants still
found the interaction beneficial for overall understanding - “the information
from decision-making perspective doesn’t seem super helpful… but I absolutely
love seeing the vis.” [$P10$].

##  5 Discussion and Future Work

The exploration of these two interfaces provided useful insights for future
multimodal system design. The active interface provided users with a rich set
of interactions, and participants often switched between visualizations,
suggesting that providing more than one way to represent the data visually was
useful. However, for both interfaces, the narrative and animation straddled
the line between distracting and engaging. These varied responses underscore
the challenges in designing clear multimodal data representations.

Benefits of each mode. Participants most often used visualizations in the
decision-making process. Many of the participants also liked the narration as
a source of information - “it’s very useful if you’re too lazy to read the
text… Audio complemented with text is very powerful” [$P14$]. Speech also
improved comprehension of the data display; “the audio was helpful to
understand the overall instructions of how to read the view” [$P20$]. These
findings are consistent with prior results in decision-making with unimodal
data representations for visualization and speech: decision quality was
highest with visualization; trust was highest with speech [SSCS24].

The staged chart composition illustrated how to read the chart, which was
useful for some users: “The animation is quite cool, and I understand how to
read the graph if I don’t know how” [$P14$]. However, several participants
admitted that the multimodality was overwhelming. $P1$ commented, “I couldn’t
keep track of all of that information… there was just too much going on.”
Future work should investigate how to balance multiple modes but not overwhelm
users with too many interface components. Different speakers, text
descriptions, and visualizations could also affect user experiences and
preferences.

Context of use. Participants noted crucial differences between the potential
usage context of the interfaces. The passive interface was viewed as better
suited for a more casual setting - “ I think if it was not a serious decision…
I would actually prefer the [passive]… I could just see it and then just draw
insight from it quickly” [$P16$], or may also be better suited for users with
less experience in data; $P6$ suggested, “That would be something that would
probably appeal to a lot of folks who don’t have a desire to go in and be
analysts themselves..” The active interface seemed a better fit for an
audience with some expertise in similar types of data interpretation,
particularly those who are “proficient in the tool already” [$P12$].

Future work should explore multimodality in different settings and with users
of different levels of expertise. While casual users might appreciate a
guided, narrative experience with high-level insights (the passive interface),
expert users may seek detailed, interactive tools for data exploration (the
active interface).

##  6 Conclusion

This work discusses two interfaces that explore the paradigms of expressing
data uncertainty in both active and passive consumption contexts. Preliminary
user feedback illuminated use cases and challenges when presenting and
interpreting multimodal data uncertainty across these contexts. This research
identifies the need for refined, context-specific multimodal strategies in
uncertainty communication, different situational contexts (e.g., for analysis
vs. during travel), and user groups (e.g., data experts vs. casual users).

## References

  * [Bea11] Bearman N.:  Using Sound to Represent Uncertainty in Future Climate Predictions for the UK.  In _17th International Conference on Auditory Display_ (2011), International Community for Auditory Display.
  * [BHJ∗14] Bonneau G.-P., Hege H.-C., Johnson C. R., de Oliveira Neto M. M., Potter K. C., Rheingans P., Schultz T.:  Overview and State-of-the-Art of Uncertainty Visualization.  In _Scientific Visualization: Uncertainty, Multifield, Biomedical, and Scalable Visualization_ (London, 2014), Hansen C. D., Chen M., Johnson C. R., Kaufman A. E., Hagen H., (Eds.), Springer, pp. 3–27.
  * [BOH11] Bostock M., Ogievetsky V., Heer J.:  D3: Data-Driven Documents.  _IEEE Transactions on Visualization and Computer Graphics 17_ , 12 (2011), 2301–2309.
  * [BS23] Bromley D., Setlur V.:  What Is the Difference Between a Mountain and a Molehill? Quantifying Semantic Labeling of Visual Features in Line Charts.  In _IEEE Transactions on Visualization and Computer Graphics_ (2023).
  * [DAV08] Dyer S., Adamo-Villani N.:  Animated Versus Static User Interfaces: A Study of Mathsigner™.  _International Journal of Human and Social Sciences 3_ , 6 (2008).
  * [EFN12] Endert A., Fiaux P., North C.:  Semantic Interaction for Visual Text Analytics.  In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_ (New York, NY, USA, 2012), CHI ’12, Association for Computing Machinery, p. 473–482.
  * [FWM∗18] Fernandes M., Walls L., Munson S., Hullman J., Kay M.:  Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making.  In _Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems_ (New York, NY, USA, 2018), Association for Computing Machinery, pp. 1–12.
  * [Goo24] Google:  Speech Synthesis Markup Language (SSML) Reference, 2024.  Google Cloud Documentation.  URL: https://cloud.google.com/text-to-speech/docs/ssml.
  * [Hul19] Hullman J.:  Why authors don’t visualize uncertainty.  _IEEE transactions on visualization and computer graphics 26_ , 1 (2019), 130–139.
  * [JED∗20] Jena A., Engelke U., Dwyer T., Raiamanickam V., Paris C.:  Uncertainty Visualisation: An Interactive Visual Survey.  In _2020 IEEE Pacific Visualization Symposium (PacificVis)_ (2020), IEEE, pp. 201–205.
  * [JP15] Jiang X., Pell M. D.:  On How the Brain Decodes Vocal Cues About Speaker Confidence.  _Cortex 66_ (2015), 9–34.
  * [Kay23] Kay M.:  _ggdist: Visualizations of Distributions and Uncertainty_.  Northwestern University, 2023.  R package version 3.3.0.  URL: https://mjskay.github.io/ggdist/.
  * [KKHM16] Kay M., Kola T., Hullman J. R., Munson S. A.:  When (Ish) is My Bus? User-Centered Visualizations of Uncertainty in Everyday, Mobile Predictive Systems.  In _Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems_ (New York, NY, USA, 2016), Association for Computing Machinery, pp. 5092–5103.
  * [KLSG22] Kirkland A., Lameris H., Székely E., Gustafson J.:  Where’s the Uh, Hesitation? The Interplay Between Filled Pause Location, Speech Rate and Fundamental Frequency in Perception of Confidence.  In _Proceedings of Interspeech_ (Incheon, Korea, 2022), Interspeech, pp. 4990–4994.
  * [KN22] Komsta L., Novomestky F.:  _Moments, Cumulants, Skewness, Kurtosis and Related Tests_.  CRAN, 2022.  R package version 0.14.1.  URL: https://cran.r-project.org/web/packages/moments/moments.pdf.
  * [Lak73] Lakoff G.:  Hedges: A Study in Meaning Criteria and the Logic of Fuzzy Concepts.  _Journal of Philosophical Logic 2_ , 4 (1973), 458–508.
  * [LBH∗97] Lodha S. K., Beahan J., Heppe T., Joseph A., Zane-Ulman B.:  MUSE: A Musical Data Sonification Toolkit.  In _Proceedings of the 4th International Conference on Auditory Display, Palo Alto, California, November 2–5, 1997_ (1997), Georgia Institute of Technology.
  * [LWS96] Lodha S., Wilson C., Sheehan R.:  LISTEN: Sounding Uncertainty Visualization.  In _Proceedings of Seventh Annual IEEE Visualization ’96_ (1996), IEEE, pp. 189–195.
  * [MG18] Mahajan K. N., Gokhale L. A.:  Comparative Study of Static and Interactive Visualization Approaches.  _International Journal on Computer Science and Engineering (IJCSE), e-ISSN_ (2018), 0975–3397.
  * [MRH∗05] MacEachren A., Robinson A., Hopper S., Gardner S., Murray R., Gahegan M., Hetzler E.:  Visualizing Geospatial Information Uncertainty: What We Know and What We Need to Know.  _Cartography and Geographic Information Science 32_ (07 2005), 139–160.
  * [Ope23] OpenJS Foundation:  Node.js, 2023.  https://nodejs.org.
  * [PKH21] Padilla L., Kay M., Hullman J.:  Uncertainty Visualization.  In _Wiley StatsRef: Statistics Reference Online_. Wiley, 02 2021, pp. 1–18.
  * [PPKH21] Padilla L. M., Powell M., Kay M., Hullman J.:  Uncertain about Uncertainty: How Qualitative Expressions of Forecaster Confidence Impact Decision-making with Uncertainty Visualizations.  _Frontiers in Psychology 11_ (2021), 579267.
  * [RSK∗21] Röbbelen A., Schmieding M. L., Kopka M., Balzer F., Feufel M. A.:  Interactive Versus Static Decision Support Tools for COVID-19: Randomized Controlled Trial.  _JMIR Public Health and Surveillance 8_ (2021).  URL: https://api.semanticscholar.org/CorpusID:248240389.
  * [SC93] Smith V. L., Clark H. H.:  On the Course of Answering Questions.  _Journal of Memory and Language 32_ (1993), 25–38.
  * [Sch97] Schriver K. A.:  _Dynamics in Document Design: Creating Text for Readers_.  John Wiley & Sons, Inc., USA, 1997.
  * [SLSR08] Skeels M., Lee B., Smith G., Robertson G.:  Revealing Uncertainty for Information Visualization.  In _AVI ’08: Proceedings of the working conference on advanced visual interfaces_ (05 2008), vol. 9, pp. 376–379.
  * [SLW73] Scherer K. R., London H., Wolf J. J.:  The Voice of Confidence: Paralinguistic Cues and Audience Evaluation.  _Journal of Research in Personality 7_ (1973), 31–44.
  * [SSCS24] Stokes C., Sanker C., Cogley B., Setlur V.:  From Delays to Densities: Exploring Data Uncertainty through Speech, Text, and Visualization.  In _Eurographics Conference on Visualization (EuroVis)_ (2024), The Eurographics Association.  to appear.
  * [SVF∗12] Szarvas G., Vincze V., Farkas R., Móra G., Gurevych I.:  Cross-genre and Cross-domain Detection of Semantic Uncertainty.  _Computational Linguistics 38_ , 2 (2012), 335–367.
  * [SW95] Sperber D., Wilson D.:  _Relevance: Communication and Cognition_.  Blackwell, Oxford, 1986/1995.
  * [SZW23] Sharif A., Zhong R., Wang Y.:  Conveying Uncertainty in Data Visualizations to Screen-Reader Users Through Non-Visual Means.  In _The 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’23)_ (New York, NY, USA, 2023), Association for Computing Machinery.
  * [(W319] (W3C) W. W. W. C.:  WAI Web Accessibility Tutorials: Complex Images.  World Wide Web Consortium website, 2019.  https://www.w3.org/WAI/tutorials/images/complex/.