# Blind Room Parameter Estimation using Multiple
Multichannel Speech Recordings

###### Abstract

Knowing the geometrical and acoustical parameters of a room may benefit
applications such as audio augmented reality, speech dereverberation or audio
forensics. In this paper, we study the problem of jointly estimating the total
surface area, the volume, as well as the frequency-dependent reverberation
time and mean surface absorption of a room in a blind fashion, based on two-
channel noisy speech recordings from multiple, unknown source-receiver
positions. A novel convolutional neural network architecture leveraging both
single- and inter-channel cues is proposed and trained on a large, realistic
simulated dataset. Results on both simulated and real data show that using
multiple observations in one room significantly reduces estimation errors and
variances on all target quantities, and that using two channels helps the
estimation of surface and volume. The proposed model outperforms a recently
proposed blind volume estimation method on the considered datasets.

##  1 Introduction

Audio augmented reality (AAR) has received increased interest in the recent
years [1]. A key task in AAR is to simulate virtual sound sources that are
consistent with the user’s environment via, e.g., a hear-through headset. To
achieve this in an immersive way, a major hurdle is to estimate the relevant
acoustical parameters of the room the user evolves in. The notion of
reverberation fingerprint (RF) was introduced in [2] as a compact way to
characterize rooms for realistic binaural rendering on AAR headphones. The RF
consists of the room’s volume $V$ in $\textrm{m}^{3}$ and its reverberation
time per octave band $\textrm{RT}_{60}(b)$ in s, where
$b\in\mathcal{B}=\\{125,250,$
$500,1\textrm{k},2\textrm{k},4\textrm{k}\\}~{}\textrm{Hz}$. Under ideal
diffuse sound field conditions, these parameters are related via Sabine’s
well-known formula [3]

$\textrm{RT}_{60}(b)\approx 0.16\frac{V}{\bar{\alpha}(b)\cdot S}$ (1)

where $S=\sum_{i=1}^{K}S_{i}$ is the total area of the room’s $K$ surfaces in
$\textrm{m}^{2}$ and $\bar{\alpha}(b)=\sum_{i=1}^{K}\alpha_{i}(b)S_{i}/S$ is
the area-weighted mean absorption coefficient in octave band
$b\in\mathcal{B}$.

An attractive research direction to retrieve such parameters is to estimate
them solely from audio recordings of unknown sound sources in the room, using
microphones placed on the headset. In [4], an algorithm to blindly estimate
the RF based on the decay envelope of a single-channel clean speech signal was
proposed and showed encouraging results on simulated data. In the same year,
the authors of [5] trained a statistical model on a large dataset generated
from simulated room impulse responses (RIRs) to blindly estimate the position
of a broadband source and the mean absorption coefficient of walls above 1 kHz
using a binaural receiver and inter-channel cues. Experiments were limited to
a fixed room geometry, a fixed receiver position, and simulated data. Later,
[6] paved the way for blind room volume estimation from single-channel noisy
speech using a convolutional neural network trained using both simulated and
real RIRs. The reported results showed that this method can estimate a broad
range of volumes within a factor of 2 on real data from the Acoustic
Characterization of Environments (ACE) challenge [7]. Following the ACE
challenge, a number of neural-network-based methods were proposed to blindly
estimate reverberation times and direct-to-reverberant ratios from speech
signals [8, 9, 10, 11]. In particular, [11] proposes a method to generate
augmented training datasets from real RIRs. These works estimate broadband
values rather than frequency-dependent ones, which may limit the rendering
realism in the context of AAR. They also all use single-channel signals. As an
interesting alternative approach, [12] proposes to learn a latent
representation referred to as room embedding, which is then used to condition
a waveform-to-waveform network that converts a signal recorded in one
environment to another. However, this approach cannot be straightforwardly
applied to spatial AAR, whereas explicitly knowing room parameters offers more
flexibility in binaural rendering. Moreover, explicit room parameters may be
useful in themselves for other applications such as speech dereverberation
[13] or audio forensics [14].

In this paper, we push these recent research efforts further by proposing a
blind method to jointly estimate the total surface area $S$, the volume $V$,
the reverberation times $\textrm{RT}_{60}(b)$ and the mean absorption
coefficients $\bar{\alpha}(b)$ in all octave bands $b\in\mathcal{B}$. On top
of that, we investigate whether using multichannel input and/or fusing
estimates across multiple source-receiver positions inside the same room help.
To this end, we propose a new convolutional neural network architecture that
combines single- and inter-channel features in a joint embedding layer, and
uses a likelihood-based loss function that yields adaptive variance estimates,
which allow us to fuse multiple independent observations of the same room in a
statistically principled way. The model is trained on a carefully-crafted
realistic simulated dataset and tested on both simulated and real wet speech
recordings111Our code to reproduce this paper is available at
github.com/prerak23/RoomParamEstim..

Figure 1: Diagram of the proposed neural network architecture.

##  2 Simulated Dataset

To train and validate our approach, a large dataset of noisy speech signals
annotated with target room acoustic parameters is created. First, synthetic
RIRs are generated using Roomsim [15]. Roomsim is a hybrid shoe-box room
acoustic simulator combining the image-source method [16] to simulate specular
reflections that dominate the early part of RIRs, and the diffuse-rain method
[15] to model scattered reflections using stochastic ray-tracing. Simulations
are run using a sampling frequency of 48 kHz, a reflection order of 10 for the
image-source method and 2000 rays for the diffuse-rain method. 20,000 rooms
are simulated with length, width and height drawn uniformly at random in
$[3,10]$ m, $[3,10]$ m and $[2.5,4]$ m, respectively, resulting in
$S\in[48,360]$ $\textrm{m}^{2}$ and $V\in[18,400]$ $\textrm{m}^{3}$. For each
room, 5 RIRs corresponding to different source-receiver positions are
generated. Omnidirectional sources and receivers are used and placed uniformly
at random at least 30 cm from each surface and from each other. The receiver
is a two-microphone array placed parallel to the floor with an aperture of
22.5 cm, that is a typical head or headset width. For each room, a single
scattering coefficient drawn uniformly at random in $[0.2,1]$ is used on all
surfaces and octave bands for the diffuse-rain method. The absorption
coefficients of the 6 surfaces in 6 octave bands are sampled using the
reflectivity-biased sampling strategy proposed in [17]. Each $\alpha_{i}(b)$
is drawn uniformly at random inside ranges based on measured databases of
commonly encountered surface materials, while giving each surface a 50$\%$
probability of having a frequency-independent reflective profile
($\alpha_{i}(b)<0.12$). As showed in [17], this yields more realistic and more
diverse distributions of absorption and reverberation times
($\textrm{RT}_{60}(b)\in[0.2,3.2]$ s and $\bar{\alpha}(b)\in[0.02,0.6]$ in our
dataset) than, e.g., by sampling each $\alpha_{i}(b)$ uniformly in $[0,1]$.

The obtained RIRs are downsampled to 16 kHz and convolved with random speech
excerpts from the LibriSpeech [18] corpus. The resulting 3 s two-channel
reverberated signals are then corrupted with both static microphone noise,
i.e., independent additive white Gaussian noise on each channel, and
spatially-diffuse babble noise, i.e., speech-shaped noise convolved with the
late part ($>$50 ms) of a random RIR in the room. For the noise levels to be
realistic, signals from sources that are placed further away from the receiver
should exhibit lower signal-to-noise ratios (SNRs). To achieve this, for each
room, we first generate a reference signal using a random speech source placed
1 meter in front of a receiver (not used in the final dataset). Static and
diffuse noise levels for this signal are set to obtain SNRs drawn uniformly at
random in $[70,90]~{}\textrm{dB}$ and $[30,60]~{}\textrm{dB}$, respectively
(different levels for each source-receiver position in the room are set).
These noise levels are then kept fixed for the final mixtures, irrespective of
the distance from the speech source to the receiver. This resulted in an
overall SNR range of $[-10,65]~{}\textrm{dB}$ across the dataset.

The 5$\times$20k = 100k two-channel noisy speech signals are divided into
training, validation and test sets of respective size 80k, 10k and 10k with no
room or speech-signal overlap between them. Each room is annotated with the 14
target parameters, namely, 6 mean absorption coefficients and 6 reverberation
times in all octave bands, the total surface area $S$ and the volume $V$. For
each room and each octave band, a unique reverberation time is estimated by
taking the median value over the 5 source-receiver positions available. The
values are obtained by linear regression over the -5 dB to -25 dB decay of
Schroeder curves [19].

##  3 Neural Network Model

###  3.1 Proposed Architecture, Cost Function, and Fusion Method

The proposed neural network architecture is depicted in Fig. 1. Single-channel
(SC) and inter-channel (IC) features are extracted from the time-domain two-
channel input signal $x$ in the form of spectrograms. We use the short-time
Fourier transform (STFT) with 96 ms Hann windows and 50% overlap to obtain a
complex spectrogram $\\{X_{i}(f,t)\\}_{f,t=1}^{F,T}$ for each channel $i$,
with $F=769$ positive frequency bins and $T=63$ time frames for a 3 s input
signal (our architecture works on arbitrary input length). This choice
performed best among 32, 64, 96 and 128 ms windows in our preliminary
experiments. Then, SC features are computed as $|X_{1}(f,t)|$, which performed
better than $|X_{1}(f,t)|^{2}$, $|X_{1}(f,t)|^{1/2}$, or $\log|X_{1}(f,t)|$.
IC features are obtained by concatenating inter-channel level differences
(ILD) and phase differences (IPD):

$\displaystyle\textrm{ILD}(f,t)$
$\displaystyle=\log|X_{1}(f,t)|-\log|X_{2}(f,t)|,$ (2)
$\displaystyle\textrm{IPD}(f,t)$
$\displaystyle=\left[\operatorname{Re},\operatorname{Im}\left(\frac{X_{1}(f,t)X_{2}^{*}(f,t)}{|X_{1}(f,t)X_{2}^{*}(f,t)|}\right)\right].$
(3)

These features are then processed through 1D convolutional blocks (1D-Conv),
which were recently proposed in the Conv-TasNet architecture in the context of
speech separation [20]. These blocks consist of separable convolutions (depth-
wise and point-wise) intertwined with rectified linear unit (ReLU) activations
and followed by layer normalization [21]. The latter proved to be crucial in
our experiments, as it creates scale-invariant representations. For SC
features, three 1D-Conv blocks with increasing dilation factors along the
frequency-axis and a kernel size of 11 are used, while only one block is used
on IC features, as this showed to give best results. The obtained
representations are concatenated along the frequency axis and average-pooled
along the time-axis to yield a time-independent, 1248-dimensional embedding
vector. The embedding vector is finally passed through 3 fully-connected
layers of respective dimensions 96, 48 and 28 to obtain $2\times D=14$ outputs
consisting of the estimated room parameters
$\mbox{\boldmath$\mu$}_{\boldsymbol{\theta}}(\mbox{\boldmath$x$})\in\mathbb{R}^{D}$
and the estimated variances
$\mbox{\boldmath$\sigma$}^{2}_{\boldsymbol{\theta}}(\mbox{\boldmath$x$})\in\mathbb{R}^{D}$
(or uncertainties) on these parameters.

The network parameters $\boldsymbol{\theta}$ are optimized by minimizing the
following Gaussian negative log-likelihood loss function:

$\displaystyle\mathcal{L}_{\boldsymbol{\theta}}(\mbox{\boldmath$x$},\mbox{\boldmath$y$})$
$\displaystyle=-\log
p_{\boldsymbol{\theta}}(\mbox{\boldmath$y$}|\mbox{\boldmath$x$})=-\log\mathcal{N}(\mbox{\boldmath$y$};\mbox{\boldmath$\mu$}_{\boldsymbol{\theta}}(\mbox{\boldmath$x$}),\mbox{\boldmath$\sigma$}^{2}_{\boldsymbol{\theta}}(\mbox{\boldmath$x$}))$
$\displaystyle\stackrel{{\scriptstyle
c}}{{=}}\frac{1}{2}\sum_{d=1}^{D}\log\sigma^{2}_{d,\boldsymbol{\theta}}(\mbox{\boldmath$x$})+\frac{(y_{d}-\mu_{d,\boldsymbol{\theta}}(\mbox{\boldmath$x$}))^{2}}{\sigma_{d,\boldsymbol{\theta}}^{2}(\mbox{\boldmath$x$})}$
(4)

where $\mbox{\boldmath$y$}\in\mathbb{R}^{D}$ denotes the ground truth room
parameters. A benefit of this approach is that it adaptively weights errors on
individual parameters. The estimated variances can also be used to fuse
estimated obtained from $J$ independent observations
$\\{\mbox{\boldmath$x$}_{j}\\}_{j=1}^{J}$ of the same room using the following
formula derived from Bayes’ rule:

$\displaystyle
p_{\boldsymbol{\theta}}(y_{d}|\bar{\mbox{\boldmath$x$}}=[\mbox{\boldmath$x$}_{1},\dots,\mbox{\boldmath$x$}_{J}])=\mathcal{N}(y_{d};\bar{\mu}_{d,\boldsymbol{\theta}}(\bar{\mbox{\boldmath$x$}}),1/\bar{\gamma}_{d,\boldsymbol{\theta}}^{2}(\bar{\mbox{\boldmath$x$}}))$
(5) $\displaystyle\text{with
}\bar{\mu}_{d,\boldsymbol{\theta}}(\bar{\mbox{\boldmath$x$}})=\sum_{j=1}^{J}\frac{\gamma_{d,\boldsymbol{\theta}}^{2}(\mbox{\boldmath$x$}_{j})}{\bar{\gamma}_{d,\boldsymbol{\theta}}^{2}(\bar{\mbox{\boldmath$x$}})}\mu_{d,\boldsymbol{\theta}}(\mbox{\boldmath$x$}_{j}),\;\bar{\gamma}_{d,\boldsymbol{\theta}}^{2}(\bar{\mbox{\boldmath$x$}})=\sum_{j=1}^{J}\gamma_{d,\boldsymbol{\theta}}^{2}(\mbox{\boldmath$x$}_{j})$

where
$\gamma_{d,\boldsymbol{\theta}}^{2}(\mbox{\boldmath$x$}_{j})=1/\sigma_{d,\boldsymbol{\theta}}^{2}(\mbox{\boldmath$x$}_{j})$
is the estimated precision for observation $\mbox{\boldmath$x$}_{j}$ and
$\bar{\mbox{\boldmath$\mu$}}_{d,\boldsymbol{\theta}}(\bar{\mbox{\boldmath$x$}})$
is the fused estimate.

To avoid issues due to scale differences, the ground truth parameters are
normalized at training time by dividing them by their standard deviations over
the training set, which are saved and multiplied with the network output at
test time. The network is trained using the ADAM optimizer [22] with a
learning rate of $10^{-4}$ and a batch size of 120. A dropout rate of 0.2 and
0.4 was used in conv-blocks and in fully connected layers to avoid over-
fitting. We used a patience of 15 epochs on the validation set for early
stopping. Our models generally converged in 100–150 epochs.

###  3.2 Other Tried Architectures

In addition to 1D-Conv blocks, Conv-TasNet introduced the idea of learnable
input filters [20]. We tried replacing the STFT by such learnable filters to
estimate our SC features, but this yielded worse performance. We also tried an
architecture inspired by x-vectors, a state-of-the-art model for speaker
recognition [23], but this did not perform well on our data. We experimented
with replacing the proposed maximum-likelihood loss (4) by a simple mean
squared error loss. This yielded significantly higher errors while losing the
benefit of easily fusing multiple observations in one room. Estimating
$\log(V)$ and $\log(S)$ instead of $V$ and $S$ as suggested in [6] did not
improve results on our data, most likely because the ranges we consider for
these quantities are smaller. Finally, instead of estimating
$\bar{\mbox{\boldmath$\alpha$}},\textrm{RT}_{60},V$ and $S$ jointly in a
multi-task fashion, we trained the same architecture to estimate each of these
quantities individually. The obtained results were identical, while increasing
the number of model parameters and training time by a factor of 4, which
suggests that our proposed network achieves efficient weight sharing.

##  4 Experiments and results

###  4.1 Simulated Data

Figure 2: Mean absolute error achieved on simulated data by [6] vs. the proposed model with one- or two-channel inputs, as a function of the number of source-receiver positions fused in each room. Shaded areas indicate 95% confidence intervals. Input | Feature | $\bar{\alpha}$ | $\textrm{RT}_{60}~{}(\textrm{s})$ | $S~{}(\textrm{m}^{2})$ | $V~{}(\textrm{m}^{3})$
---|---|---|---|---|---
1mic, 1sig | SC | 0.055 | 0.225 | 55.0 | 68.8
1mic, 2sig | SC | 0.058 | 0.222 | 55.4 | 69.7
2mic, 1sig | SC | 0.057 | 0.221 | 54.0 | 67.7
2mic, 1sig | SC+IC | 0.055 | 0.236 | 49.9 | 63.7
Table 1: Mean absolute error achieved on simulated data for one source-receiver position using different inputs and features. Bold numbers indicate the best statistically significant result per column based on 95% confidence intervals, when there is one. Octave bands | 1 position | 5 positions
---|---|---
$\bar{\alpha}$ | $\textrm{RT}_{60}~{}(\textrm{s})$ | $\bar{\alpha}$ | $\textrm{RT}_{60}~{}(\textrm{s})$
125 Hz | 0.056 | 0.392 | 0.051 | 0.320
250 Hz | 0.060 | 0.305 | 0.055 | 0.249
500 Hz | 0.057 | 0.228 | 0.051 | 0.170
1 kHz | 0.053 | 0.188 | 0.050 | 0.146
2 kHz | 0.054 | 0.165 | 0.051 | 0.122
4 kHz | 0.052 | 0.139 | 0.049 | 0.106
Table 2: Mean absolute errors on $\bar{\alpha}(b)$ and $\textrm{RT}_{60}(b)$
in the 6 octave bands achieved by the proposed model with two channels and 1
or 5 source-receiver positions per room on simulated data.

We first evaluate the proposed approach on the 2,000 unseen rooms of the
simulated test set (see Section 2), using between 1 and 5 two-channel 3-s
noisy speech recordings at different source-receiver positions per room, based
on the fusion method in (5). Two variants of our model are compared: the full
two-channel architecture depicted in Fig. 1, and the same architecture without
the upper inter-channel processing part of the network, i.e., using only one
channel. These variants are compared to our implementation of the single-
channel, single-position blind room volume estimation method of [6] trained on
the same data. We could not find competing methods for blind total surface
estimation or blind frequency-dependent reverberation time and mean absorption
coefficient estimation in the literature. The metric used is the mean absolute
error on each room parameter.

As can be seen in Fig. 2, increasing the number of fused observations per room
significantly reduces errors on all parameters. Using five positions and two
channels, mean absolute errors of $0.052$ for $\bar{\alpha}$ (training range
$[0.02,0.6]$), $0.18~{}\textrm{s}$ for $\textrm{RT}_{60}$ (training range
$[0.2,3.2]$), $42~{}\textrm{m}^{2}$ for $S$ (training range $[48,360]$) and
$54~{}\textrm{m}^{3}$ for $V$ (training range $[18,400]$) are obtained. The
proposed model significantly outperforms the one in [6] for volume estimation,
reducing the error by 13% using one channel and one observation, and by 31%
using two channels and five observations. Interestingly, we observe that using
two channels instead of one significantly reduces surface and volume
estimation error without significantly impacting mean absorption and
reverberation time estimation, according to 95% confidence intervals. This may
be interpreted by the fact that the latter mostly govern the late, spatially-
diffuse regime of RIRs, and hence should have limited correlation with inter-
channel cues that mostly capture spatial characteristics. Conversely, $S$ and
$V$ are inherently spatial quantities as they relate to the room’s geometry
and hence early echoes, which do correlate with IC cues [24, 25].

We hence carry an additional experiment to study whether the improvement
observed when using two channels is truly explained by concatenating SC and IC
representations at the embedding layer (see Fig. 1), or is only due to
doubling the number of signal samples in a given source-receiver position,
which mitigates noise. Four different embeddings are compared, namely (i) the
SC representation of one speech signal (1mic, 1sig), (ii) the average of the
SC representations of two speech signals (1mic, 2sig), (iii) the concatenation
of the SC representations of the two individual channels of one speech signal
(2mic, 1sig) and (iv) the proposed concatenation of SC and IC representations.
Speech signals are always 3-s long. As can be seen in Table 1, the only model
which significantly improves surface and volume estimation is the one using IC
features, despite the fact that the 2nd and 4th models also benefit from the
same number of input signal samples.

So far, the reported errors on $\bar{\alpha}$ and $\textrm{RT}_{60}$ were
averaged over all 6 octave bands. Table 2 reports detailed errors per octave
band. As can be seen, the mean absorption is well estimated in all octave
bands, while reverberation-time errors steadily decrease by a factor of 3 from
125 Hz to 4 kHz. This could be explained by the fact that less information is
available in the narrower, lower octave bands.

###  4.2 Real Data

To check how well our method generalizes to real data, we use the recently
released dEchorate dataset [25], which contains wet speech recordings made
inside the acoustic lab at Bar-Ilan University. The lab is a shoe-box room of
size $5.7\times 6\times 2.4~{}\textrm{m}$ ($S=125~{}\textrm{m}^{2}$,
$V=82~{}\textrm{m}^{2}$) where each of the walls, floor and ceiling can be set
either to either a reflective mode or an absorbent mode. 5 arrays of 6
omnidirectional microphones and 6 directional loudspeakers are placed inside
the room, yielding $5\times 6=30$ multi-channel speech recordings per room
configuration. For each room, ground truth $\bar{\alpha}$ and
$\textrm{RT}_{60}$ values are provided for the four octave bands from 500 Hz
to 4 kHz. Hence, results in lower octave bands are omitted here. In our
experiments, we use $3\times 30=90$ 3-s speech recordings corresponding to the
2-channel sub-arrays with aperture $22.5~{}\textrm{cm}$ and to the 3 room
configurations involving 3 or more reflective surfaces, as these most closely
match the considered scenario. For these rooms, $\bar{\alpha}(b)$ ranges from
0.16 to 0.35 and $\textrm{RT}_{60}(b)$ from 0.25 to 0.66 s.

Table 3 reports mean absolute errors using the proposed approach with or
without IC features. We report results using either 1 or 5 source positions
and a fixed receiver. For the latter, we exclude one out of the 6 available
source positions for each test, so that there are 90 tests in each case. The
mean absolute volume estimation error using the single-channel, single-
position approach of [6] is reported as well. Encouragingly, errors obtained
with our approach are of comparable orders to those obtained on simulated
data. In the single-channel, single-position case, volume estimation errors
obtained with our model are comparable to [6]. For $\textrm{RT}_{60}$, $S$ and
$V$, we are able to reproduce the observation that increasing the number of
source-receiver positions significantly decreases errors, at least using IC
features. We also observe again that using IC features significantly improves
the estimation of $S$ and $V$. The errors obtained on $\bar{\alpha}$ are less
consistent than those on simulated data, which may be due to the difficulty of
reliably annotating mean absorption coefficients in real rooms.

Method | Features | # pos | $\bar{\alpha}$ | $\textrm{RT}_{60}$ | $S$ | $V$
---|---|---|---|---|---|---
[6] | SC | 1 | - | - | - | 137.8
Ours | SC | 1 | 0.061 | 0.134 | 129.6 | 154.5
Ours | SC | 5 | 0.060 | 0.097 | 125.8 | 149.1
Ours | SC+IC | 1 | 0.084 | 0.101 | 89.4 | 107.6
Ours | SC+IC | 5 | 0.094 | 0.062 | 50.2 | 68.8
Table 3: Mean absolute error achieved over 3 rooms from the real dEchorate dataset. Bold numbers indicate the best statistically significant result per column, based on 95% confidence intervals. Method | Features | # pos | $\bar{\alpha}$ | $\textrm{RT}_{60}$ | $S$ | $V$
---|---|---|---|---|---|---
[6] | SC | 1 | - | - | - | 10.0
Ours | SC | 1 | 0.030 | 0.161 | 27.2 | 31.8
Ours | SC | 5 | 0.024 | 0.090 | 19.6 | 23.0
Ours | SC+IC | 1 | 0.031 | 0.100 | 34.7 | 39.7
Ours | SC+IC | 5 | 0.015 | 0.054 | 16.5 | 18.9
Table 4: Standard deviation of parameter estimates for room ”011100” of the
real dEchorate dataset.

Finally, Table 4 shows the standard deviations of estimated values by the same
methods over the 30 recordings from the room with 3 reflective surfaces.
Encouragingly, the relatively low standard deviations reveal the ability of
the models to provide parameter estimates that are stable within a room, and
do not depend much on the source-receiver position. Moreover, it can be seen
that, as expected, using five observations in a room instead of one
systematically decreases the standard deviation of estimates.

##  5 Conclusion

This study revealed that using inter-channel cues can significantly improve
the blind estimation of a room’s volume and surface from noisy speech, while
for estimating reverberation and absorption parameters, a single channel is
sufficient. It also highlights that fusing multiple measurements reduces
estimation errors and variances on all parameters. Finally, we showed that a
system trained on a carefully simulated training set offers reasonable
generalization capabilities to real data. Future work will include the use of
data-augmentation and domain adaptation techniques to improve real data
results, extensions to binaural or ambisonic receivers, and the joint
estimation of local parameters such as the positions and properties of the
source and individual surfaces in the room, with the help of early acoustic
echoes.

##  6 Acknowledgments

This work was made with the support of the French National Research Agency
through project HAIKUS “Artifical Intelligence applied to augmented acoustic
scenes” (ANR-19-CE23-0023). Experiments presented in this paper were carried
out using the Grid’5000 testbed, supported by a scientific interest group
hosted by Inria and including CNRS, RENATER and several Universities as well
as other organizations (see https://www.grid5000.fr). We thank the authors of
[6] for their helpful advice.

## References

  * [1] V. Valimaki, A. Franck, J. Ramo, H. Gamper, and L. Savioja, “Assisted listening using a headset: Enhancing audio perception in real, augmented, and virtual environments,” _IEEE Signal Processing Magazine_ , vol. 32, no. 2, pp. 92–99, 2015.
  * [2] J.-M. Jot and K. S. Lee, “Augmented reality headphone environment rendering,” in _2016 AES International Conference on Audio for Virtual and Augmented Reality_ , 2016, pp. 8–2.
  * [3] C. F. Eyring, “Reverberation time in “dead” rooms,” _The Journal of the Acoustical Society of America_ , vol. 1, no. 2A, pp. 217–241, 1930.
  * [4] P. Murgai, M. Rau, and J.-M. Jot, “Blind estimation of the reverberation fingerprint of unknown acoustic environments,” in _AES 143rd Convention_ , 2017, p. 9905.
  * [5] S. Kataria, C. Gaultier, and A. Deleforge, “Hearing in a shoe-box: binaural source position and wall absorption estimation using virtually supervised learning,” in _2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2017, pp. 226–230.
  * [6] A. F. Genovese, H. Gamper, V. Pulkki, N. Raghuvanshi, and I. J. Tashev, “Blind room volume estimation from single-channel noisy speech,” in _2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2019, pp. 231–235.
  * [7] J. Eaton, N. D. Gaubitch, A. H. Moore, and P. A. Naylor, “Estimation of room acoustic parameters: The ACE challenge,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 24, no. 10, pp. 1681–1693, 2016\.
  * [8] F. Xiong, S. Goetze, and B. T. Meyer, “Joint estimation of reverberation time and direct-to-reverberation ratio from speech using auditory-inspired features,” _arXiv preprint arXiv:1510.04620_ , 2015.
  * [9] H. Gamper and I. J. Tashev, “Blind reverberation time estimation using a convolutional neural network,” in _16th International Workshop on Acoustic Signal Enhancement (IWAENC)_ , 2018, pp. 136–140.
  * [10] M. Chai, T. Li, M. Zhu, T. Wang, and W. Zhang, “Blind estimation of reverberation time using binaural complex ideal ratio mask,” in _2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)_, 2019, pp. 378–383.
  * [11] N. J. Bryan, “Impulse response data augmentation and deep neural networks for blind room acoustic parameter estimation,” in _2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020, pp. 1–5.
  * [12] J. Su, Z. Jin, and A. Finkelstein, “Acoustic matching by embedding impulse responses,” in _2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020, pp. 426–430.
  * [13] B. Wu, M. Yang, K. Li, Z. Huang, S. M. Siniscalchi, T. Wang, and C.-H. Lee, “A reverberation-time-aware DNN approach leveraging spatial information for microphone array dereverberation,” _EURASIP Journal on Advances in Signal Processing_ , vol. 2017, no. 1, pp. 1–13, 2017.
  * [14] A. H. Moore, M. Brookes, and P. A. Naylor, “Roomprints for forensic audio applications,” in _2013 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)_ , 2013, pp. 1–4.
  * [15] S. M. Schimmel, M. F. Muller, and N. Dillier, “A fast and accurate “shoebox” room acoustics simulator,” in _2009 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2009, pp. 241–244.
  * [16] J. Allen and D. Berkley, “Image method for efficiently simulating small-room acoustics,” _The Journal of the Acoustical Society of America_ , vol. 65, pp. 943–950, 1979.
  * [17] C. Bastien, A. Deleforge, and C. Foy, “Mean absorption coefficient estimation from impulse responses: Deep learning vs. Sabine,” in _E-FA 2020 - Forum Acusticum 2020_ , Dec. 2020.
  * [18] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in _2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2015, pp. 5206–5210.
  * [19] M. R. Schroeder, “New method of measuring reverberation time,” _The Journal of the Acoustical Society of America_ , vol. 37, no. 6, pp. 1187–1188, 1965.
  * [20] Y. Luo and N. Mesgarani, “Conv-TasNet: Surpassing ideal time–frequency magnitude masking for speech separation,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 27, no. 8, p. 1256–1266, Aug 2019\.
  * [21] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” _stat_ , vol. 1050, p. 21, 2016.
  * [22] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in _ICLR (Poster)_ , 2015. [Online]. Available: http://arxiv.org/abs/1412.6980
  * [23] D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X-vectors: Robust DNN embeddings for speaker recognition,” in _2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2018, pp. 5329–5333.
  * [24] D. Di Carlo, A. Deleforge, and N. Bertin, “MIRAGE: 2D source localization using microphone pair augmentation with echoes,” in _2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2019, pp. 775–779.
  * [25] D. Di Carlo, P. Tandeitnik, C. Foy, N. Bertin, A. Deleforge, and S. Gannot, “dEchorate: a calibrated room impulse response database for echo-aware signal processing,” 2021, (under review). [Online]. Available: https://hal.archives-ouvertes.fr/hal-03207860


# Microwave photo-association of fine-structure-induced Rydberg
$(n+2)D_{5/2}nF_{J}$ macro-dimer molecules of cesium

Jingxu Bai  State Key Laboratory of Quantum Optics and Quantum Optics Devices,
Institute of Laser Spectroscopy, Shanxi University, Taiyuan 030006, China
Yuechun Jiao<EMAIL_ADDRESS>State Key Laboratory of Quantum Optics and
Quantum Optics Devices, Institute of Laser Spectroscopy, Shanxi University,
Taiyuan 030006, China  Collaborative Innovation Center of Extreme Optics,
Shanxi University, Taiyuan 030006, China  Rong Song  State Key Laboratory of
Quantum Optics and Quantum Optics Devices, Institute of Laser Spectroscopy,
Shanxi University, Taiyuan 030006, China  Georg Raithel  Department of
Physics, University of Michigan, Ann Arbor, Michigan 48109-1120, USA  Suotang
Jia  State Key Laboratory of Quantum Optics and Quantum Optics Devices,
Institute of Laser Spectroscopy, Shanxi University, Taiyuan 030006, China
Collaborative Innovation Center of Extreme Optics, Shanxi University, Taiyuan
030006, China  Jianming Zhao<EMAIL_ADDRESS>State Key Laboratory of
Quantum Optics and Quantum Optics Devices, Institute of Laser Spectroscopy,
Shanxi University, Taiyuan 030006, China  Collaborative Innovation Center of
Extreme Optics, Shanxi University, Taiyuan 030006, China

###### Abstract

Long-range $(n+2)D_{5/2}\,nF_{J}$ Rydberg macro-dimers are observed in an
ultracold cesium Rydberg gas for $39\leq n\leq 48$. Strong dipolar “flip”
($\langle D_{5/2}F_{5/2}|\hat{V}_{dd}|F_{5/2}D_{5/2}\rangle$, $\langle
D_{5/2}F_{7/2}|\hat{V}_{dd}|F_{7/2}D_{5/2}\rangle$) and “cross” ($\langle
D_{5/2}F_{7/2}|\hat{V}_{dd}|F_{5/2}D_{5/2}\rangle$) couplings lead to bound,
fine-structure-mixed $(n+2)D_{5/2}nF_{J}$ macro-dimers at energies between the
$F_{J}$ fine-structure levels. The $DF$ macro-dimers are measured by microwave
photo-association from optically prepared $[(n+2)D_{5/2}]_{2}$ Rydberg pair
states. Calculated adiabatic potential curves are used to elucidate the
underlying physics and to model the $DF$ macro-dimer spectra, with good
overall agreement. Microwave photo-association allows Franck-Condon tuning,
which we have studied by varying the detuning of a Rydberg-atom excitation
laser. Further, in Stark spectroscopy we have measured molecular DC electric
polarizabilities that are considerably larger than those of the atomic states.
The large molecular polarizabilities may be caused by high-$\ell$ mixing. The
observed linewidths of the Stark-shifted molecular lines provide initial
evidence for intra-molecular induced-dipole-dipole interaction.

Ultracold gases in the $\mu$K-regime have opened a new avenue to the
investigation of interacting many-body systems. While interactions between
ultracold ground-state atoms generally remain fairly weak, interactions
involving Rydberg atoms (atoms with principal quantum numbers $n\gtrsim$10)
are readily observable in typical cold atomic gases. Molecules in which a
ground-state atom is located inside the Rydberg atom Greene  _et al._ (2000);
Hamilton  _et al._ (2002); Khuskivadze  _et al._ (2002); Bendkowsky  _et al._
(2009); Anderson  _et al._ (2014); Krupp  _et al._ (2014); Booth  _et al._
(2015); Niederprüm  _et al._ (2016); Saßmannshausen  _et al._ (2015); Bai  _et
al._ (2020) have attracted considerable attention due to their permanent
electric dipole moments Booth  _et al._ (2015); Niederprüm  _et al._ (2016).
Rydberg-atom-ion molecules, which were predicted and observed Duspayev  _et
al._ (2021); Deiß  _et al._ (2021); Zuber  _et al._ (2022); Zou  _et al._
(2023), are bound by electric multipolar interactions, as are molecules
consisting of two Rydberg atoms. The latter, known as Rydberg macro-dimers,
were predicted in Boisseau  _et al._ (2002) and observed, for instance, in
Overstreet  _et al._ (2009); Deiglmayr  _et al._ (2014); Saßmannshausen and
Deiglmayr (2016); Han  _et al._ (2018); Hollerith  _et al._ (2019, 2022);
Hollerith and Zeiher (2023). Rydberg-pair interactions, which include the
dipole-dipole ($\thicksim n^{4}/R^{3}$) and the van-der-Waals ($\thicksim
n^{11}/R^{6}$) Gallagher (1994) interactions ($R$ is the internuclear
distance), give rise to long-range adiabatic potential curves. Some of these
have wells that support meta-stable Rydberg-pair macro-dimer molecules.

Deiglmayr et al. have prepared Cs macro-dimer molecules near $nS\,n^{\prime}F$
and $[nP]_{2}$ asymptotes for 22$\leq n\leq 32$ Deiglmayr  _et al._ (2014),
and near $nP\,(n+1)S$ for $n$=43 and 44 Saßmannshausen and Deiglmayr (2016),
where molecules are bound through long-range dipolar interaction. Han et al.
Han  _et al._ (2018) have reported on $[62D]_{2}$ macrodimer molecules
prepared via a two-color, two-photon photo-association scheme, and have
modeled their results using molecular-potential computations Han  _et al._
(2019). Hollerith et al. Hollerith  _et al._ (2019, 2022); Hollerith and
Zeiher (2023) directly observed Rydberg macrodimers with bond lengths up to
0.7 $\mu$m in optical lattices using an ion microscope, manipulated the
molecular arrangement, and proposed applications in quantum computing.

Figure 1: (a) Sketch of the experimental setup. Rydberg macro-dimer molecules
are produced by laser excitation of $D$-state Rydberg atoms and microwave
photo-association. (b) Level diagram. A Rydberg pair,
$|DD\rangle=[(n+2)D_{5/2}]_{2}$, is laser-excited from
$|GG\rangle=[6S_{1/2}]_{2}$ by 852-nm and 509-nm lasers (852-nm-laser detuning
$\Delta_{852}/2\pi=360$ MHz). A scanned microwave field excites $nF_{J}$ atoms
or photo-associates $|DF\rangle=(n+2)D_{5/2}\,nF_{J}$ FS-mixed macro-dimers,
which are detected as described in the text. (c) Visualization of microwave
photo-association with incident and bound vibrational wave-functions on
respective potential curves.

In this letter, we report on the observation of fine-structure-mixed (FS-
mixed) $(n+2)D_{5/2}\,nF_{J}$ Rydberg macro-dimer molecules bound by dipolar
interactions, the longest-range multipole coupling between neutral atoms.
Instead of using the common method of laser photo-association, here we prepare
the $(n+2)D_{5/2}\,nF_{J}$ macro-dimers by microwave photo-association. The
microwave field drives a transition from an optically-prepared
$[(n+2)D_{5/2}]_{2}$ Rydberg-pair state, which is relatively weakly-
interacting, into a more strongly-interacting $(n+2)D_{5/2}\,nF_{J}$ macro-
dimer state. We provide spectroscopic evidence for FS-mixed $DF$ macro-dimer
molecules and explain their formation mechanism. We further address Franck-
Condon tuning afforded by microwave photo-association spectroscopy, and
investigate the Stark effect and the linewidths of $DF$ macro-dimer signals in
weak electric fields.

The experiment is performed in a Cs magneto-optical trap with a temperature
$\sim$ 100 $\mu$K and peak density $\sim$ $10^{10}$ cm-3. The atom temperature
is decreased to $\sim$40 $\mu$K after 3 ms of molasses cooling. After
switching off the molasses, a $(n+2)D_{5/2}$ Rydberg state is populated by
coincident and counter-propagating 852-nm and 509-nm laser pulses of 1-$\mu$s
duration, as shown in Fig. 1 (a). Rydberg-atom pairs are then photo-associated
into $(n+2)D_{5/2}\,nF_{J}$ macro-dimers by a 20-$\mu$s microwave pulse. An
electric-field ramp is then applied for state-selective field ionization
Gallagher (1994). Time-resolved detection of the liberated ions with a
microchannel plate detector and a boxcar integrator then allows selective
detection of $nF$-type Rydberg atoms and molecular constituents. Prior to the
experiment, we employ Stark and Zeeman spectroscopy to reduce stray electric
and magnetic fields to below 2 mV/cm and 5 mG, respectively. More experimental
details are given in Bai  _et al._ (2023).

Figure 2:  (a) Calculated potential energy curves vs internuclear separation
$R$ for $39D_{5/2}\,37F_{J}$ pairs with the indicated values for $M$ and
parity ($-/+$ means odd/even). Symbol areas are proportional to calculated
microwave transition rates from the launch state $[39D_{5/2}]_{2}$. (b)
Experimental microwave spectrum, averaged over 50 measurements. In (a),
between $37F_{7/2}$ and $37F_{5/2}$, we find molecular potential wells that
correspond with signals measured in (b). These signatures reflect FS-mixed
$39D_{5/2}\,37F_{J}$ macro-dimers. Dashed horizontal lines across (a) and (b)
mark the detunings of prominent potential maxima and minima. (c) Simulated
microwave spectra for two Wigner-Seitz radii, showing good qualitative
agreement with (b).

In Fig. 2 (a) we show calculated adiabatic potentials for Cs Rydberg pair
states in $39D_{5/2}\,37F_{J}$ for the indicated values of $M$, the conserved
sum of the projections of the total electronic angular momenta onto the
internuclear axis, and parity. For details of the calculations see
Supplemental Material sup . The detuning is defined to be zero at the
$39D_{5/2}\,37F_{5/2}$ asymptote. In Fig. 2 (b), we show the measured photo-
association spectrum versus microwave detuning. The narrow and wide features
in (b) near zero detuning and $\approx-20$ MHz show the $39D_{5/2}\to
37F_{5/2}$ and $39D_{5/2}\to 37F_{7/2}$ atomic resonances, which are broadened
due to dipolar Rydberg-pair interactions. About halfway in-between the atomic
resonances the spectrum in Fig. 2 (b) exhibits a narrow signal that coincides
with Rydberg-dimer potential wells seen in Fig. 2 (a), as indicated with the
dashed lines.

At large distances ($\gtrsim 8~{}\mu$m in Fig. 2), the dipole-dipole
interactions are dominant and lead to asymptotic potentials $\propto
C_{3}/R^{3}$. The $C_{3}$-coefficients are much larger for
$(n+2)D_{5/2}\,nF_{7/2}$ than for $(n+2)D_{5/2}\,nF_{5/2}$ due to differences
in angular matrix elements. Averaged over all $M$ and parities, the magnitudes
of the $C_{3}$-coefficients average to $\sim 1.3$ GHz$\mu m^{3}$ and $\sim
0.07$ GHz$\mu m^{3}$ for the $37F_{7/2}$ and $37F_{5/2}$ resonances,
respectively. Also, the respective 96 and 72 long-range potentials are fairly
evenly distributed and symmetric about the asymptotic energies. As a result,
the interaction-broadened atomic $37F_{J}$-lines in Fig. 2 (b) are quite
symmetric, with the $F_{7/2}$-line being wider than the $nF_{5/2}$-line by a
factor of about 5.

The potential wells in Fig. 2 (a) correlate with signal maxima in the
experimental data in Fig. 2 (b). To explain their origin, we first note that
the strongest interactions are the resonant “flip” interactions between
(symmetrized) $|39D_{5/2},37F_{7/2}\rangle$ pair states, which have dipole-
dipole matrix elements $V_{7/2}:=\langle
DF_{7/2}|\hat{V}_{dd}|F_{7/2}D\rangle$ and cause the aforementioned massive
broadening of the atomic $F_{7/2}$-line. The matrix elements of the
$|39D_{5/2},37F_{5/2}\rangle$ flip interactions, $V_{5/2}:=\langle
DF_{5/2}|\hat{V}_{dd}|F_{5/2}D\rangle$, are only about $0.05$-times as strong,
resulting in a less-broadened atomic $F_{5/2}$-line. With decreasing $R$,
numerous $39D_{5/2}\,37F_{7/2}$-potentials push upwards into comparatively
weakly-shifting $39D_{5/2}\,37F_{5/2}$-potentials and intersect, generating a
series of anti-crossings with dipolar “cross”-coupling strengths on the order
of $\sqrt{V_{7/2}V_{5/2}}$, amounting to about a quarter of the atomic FS
splitting. With all relevant couplings being dipolar and hence relatively
strong, the potential wells that result for several $M$’s and parities are
well-distinct and robust. The vibrational states in the wells are likely
stable against non-adiabatic decay, as suggested by our experimental
observations of sharp $DF$ macro-dimer features. For further argument, we note
that, due to the proximity of the atomic FS states, the FS-mixed $DF$ wells
are “squeezed” inside a spectral range that is only a few tens of MHz wide
(for $n\sim 40$). The proximity of the atomic FS states is instrumental in
pushing the radial positions of the mixed-FS $DF$ macro-dimer wells far out
along $R$. There, intersections with other (potentially unaccounted-for)
potentials are unlikely, or, if there are any, they are likely too weakly
coupled to cause fast non-adiabatic decay.

To prove the FS-mixed character of the observed macro-dimers, we have computed
$s(R)=\langle J-\ell\rangle(R)$ of the adiabatic electronic states on several
potential curves in Fig. 2 (a) that have wells. Since $s=\pm 1/2$ for
$J=\ell+1/2$ and $J=\ell-1/2$, $s$ is a measure for FS mixing. At large $R$
outside the wells it is $s(R)\approx 1/2$, which is expected for
$39D_{5/2}\,37F_{7/2}$. Inside the wells $s(R)$ drops to about $0.1$,
indicating a FS-mixed $39D_{5/2}\,37F_{J}$ macro-dimer. We have also find that
mixing with $\ell\geq 4$-states is not a significant factor in the formation
of the wells.

To model the experimental spectra, we have computed microwave excitation rates
from the optically-prepared launch state $[(n+2)D_{5/2}]_{2}$ into the
adiabatic pair states $39D_{5/2}37F_{J}$ on the potentials in Fig. 2 (a).
Typical results are shown in Fig. 2 (c) for two indicated values of the
Wigner-Seitz radius. For details of the calculation see Supplemental Material
sup . The model spectrum reproduces the essential features measured in Fig. 2
(b), except that the measured features attributed to the $DF$-molecules appear
to be down-shifted by 1 to 2 MHz relative to the computed ones. The difference
is attributed to the positive, $\lesssim 2$ MHz, van-der-Waals level shifts of
the launch pair, which result in likewise negative shifts of the measured
molecular peaks. It is thus seen that energy shifts on the free potential in
Fig. 1 (c), and Franck-Condon overlaps between the free and bound vibrational
states, play a role in microwave photo-association spectroscopy of Rydberg
macro-dimers.

Figure 3: (a) Microwave spectra, showing interaction-broadened atomic $nF_{J}$
lines (tall peaks on the left and right) and $(n+2)D_{5/2}\,nF_{J}$ macro-
dimer signals (highlighted peaks in the middle) for $37\leq n\leq 48$. (b)
Measurements (symbols) of the energies of the macro-dimer peaks relative to
the $nF_{5/2}$ atomic lines as a function of $n$. The solid red line shows an
allometric fit with fit parameters as indicated.

From microwave spectra analogous to Fig. 2 (b) we have extracted the energy of
the most-prominent molecular $DF$ peaks relative to the $nF_{5/2}$ atomic
lines for $37\leq n\leq 48$. The magnitudes of the relative line positions,
extracted from the spectra displayed in Fig. 3 (a) and plotted against $n$ in
Fig. 3 (b), closely follow an allometric fit of the form $y=c\,n_{eff}^{d}$.
Our fit result $d=-2.90(8)$ means that the line positions track closely in
proportion with the FS splitting. Macro-dimer energy scalings near $n^{-3}$
have been observed elsewhere Samboy  _et al._ (2011); Samboy and Côté (2011);
Han  _et al._ (2018). Here, the data and scalings observed in Fig. 3 show that
the FS-mixed $DF$ macro-dimers are generic and exist over a substantial range
in $n$. The molecular potentials and the widths of the $DF$ molecular peaks in
Figs. 2 and 3 further show that the depths of the molecular-potential wells
(i.e. the molecular binding energies) are about 25% of the relative line
positions shown in Fig. 3(b).

Findings above suggest that the energy of the $DD$ launch pair state on its
repulsive interaction potential may be tuned to maximize the Franck-Condon
overlap for the microwave transitions. To study this possibility, we have
varied the detuning of the 509-nm laser to the blue side of atomic $nD_{5/2}$
resonances during the launch-state preparation. For $n=48$, we have observed a
net improvement of the molecular signal at about $200$ kHz blue detuning. The
efficacy of this type of Franck-Condon tuning is hampered by the width of the
initial kinetic-energy distribution of the atoms, which translates into a
frequency uncertainty on the order of 1 MHz on the incident channel. Also, the
blue-detuning leads to a general decrease of launch-state Rydberg-atom
density, which has a negative effect on the net macro-dimer microwave
excitation probability. Hence, our observation of a moderate molecular-signal
enhancement when blue-detuning the Rydberg excitation laser is encouraging.
Improvements of Franck-Condon tuning may be achieved by reducing the atom
temperature and by preparing the launch-state $nD_{5/2}$ atoms in a Rydberg-
atom optical lattice Malinovsky  _et al._ (2020); Cardman and Raithel (2023);
Ravon  _et al._ (2023).

Next we measure the Stark effect of the $DF$ Rydberg macro-dimers in a weak
applied static electric field, $E_{DC}$. In Fig. 4(a), we show a contour plot
of spectra versus $E_{DC}$ and microwave detuning. In Fig. 4(b-d), we present
Stark shifts of the atomic and molecular peaks, obtained from Lorentzian fits.
Both atomic and molecular lines are red-shifted due to quadratic Stark effect,
with fitted polarizabilities $\alpha$($37F_{5/2}$)=88(6) kHz/(V/cm)2,
$\alpha$($37F_{7/2}$)=75(12) kHz/(V/cm)2 and
$\alpha_{M}:=\alpha$($39D_{5/2}\,37F_{J}$)=210(18) kHz/(V/cm)2. The
polarizability $\alpha_{M}$ of the molecular state is found to be about 2.7
times larger than that of the atomic states.

Figure 4:  (a) Color-scale plot of microwave spectra near $37F_{J}$ vs DC
electric field, $E_{DC}$, and microwave detuning. Symbols in the bottom panels
show Stark shifts of the atomic line $37F_{7/2}$ (b), the $39D_{5/2}\,37F_{J}$
Rydberg macro-dimer (c), and the atomic line $37F_{5/2}$ (d), respectively.
The solid lines show best-fit functions of the type $W=-1/2\alpha
E_{DC}^{2}+\beta$, with fitted polarizabilities $\alpha$ shown in the panels.

Calculated atomic polarizabilities (see Supplemental Material sup ) yield an
isotropic average of the differential polarizability of $\approx 85$
kHz/(V/m)2 for the atomic lines in Fig. 4, which agrees with the measurements
within uncertainties. For the isotropic average of the molecular
polarizabilities, our calculation (see Supplemental Material sup ) essentially
yields the sum of the atomic polarizabilities, which is about a factor of 2.7
smaller than the measured $\alpha_{M}$. We believe that the calculation may
fail to account for minute admixtures from highly polarizable $\ell\geq 4$
states. Increasing our basis sets to include atomic states with $\ell$ up to
15, there is evidence that the macro-dimer states pick up increasing (but
still too small) high-$\ell$ character. It remains to be seen whether even
larger basis sets will yield molecular polarizabilities in agreement with the
measured $\alpha_{M}$. Also, the validity of our utilized perturbative model
for molecular polarization (see Supplemental Material sup ) could be tested.

In Fig. 5(a) we show the linewidths measured for a selection of atomic and
molecular peaks as a function of $E_{DC}$. The atomic lines are fairly broad
throughout, in agreement with Fig. 2. As a result, any extra broadening that
occurs with increasing $|E_{DC}|$ is insignificant. In contrast, the molecular
feature is narrow at $E_{DC}\sim 0$, in agreement with Fig. 2, and there is
considerable extra broadening with increasing $|E_{DC}|$, considered next.

Figure 5: (a) Linewidths of the atomic $37F_{7/2}$ (navy) and $37F_{5/2}$
(dark cyan) peaks, and of the molecular $39D_{5/2}\,37F_{J}$ (purple) peak
from Fig. 4 vs $E_{DC}$. (b) Line broadening of the molecular
$39D_{5/2}\,37F_{J}$ (red) and $48D_{5/2}\,46F_{J}$ (royal) molecular peaks vs
$E_{DC}$. In both (a) and (b), solid curves are best fits of the form
$a\,\sqrt{1+b\,E_{DC}^{4}}$.

The variation of the molecular Stark shift is $\delta
W_{S}=-\alpha_{M}E_{DC}\delta E_{DC}-E_{DC}^{2}\delta\alpha_{M}/2$. Here, the
variation $\delta E_{DC}$ is proportional to $E_{DC}$ and very small. For the
variation of the molecular polarizability we estimate $\delta\alpha_{M}\sim\pm
0.25\alpha_{M}$ (en par with the variation of the relevant atomic tensor
polarizabilities). Accounting for a background linewidth $\delta W_{0}$, at
the largest $|E_{DC}|$ probed in Fig. 4 the net linewidth $\delta
W=\sqrt{\delta W_{0}^{2}+\delta W_{S}^{2}}$ would be $\delta
W\lesssim|W_{S}|$. This is not enough to explain the data in Fig. 5, where
$\delta W\gtrsim 2|W_{S}|$ for the molecular peak at the largest $|E_{DC}|$
probed.

A permanent molecular dipole moment, $p$, could, in principle, explain the
large observed $\delta W$. However, the $\gtrsim h\times 10$-MHz strength of
Rydberg-pair interactions enforces symmetrized pair states at $E_{DC}=0$,
which have well-defined parity and $p=0$. As such, the situation differs from
Rydberg-ground molecules Greene  _et al._ (2000), which have extremely weak
“flip” interactions and allow non-symmetrized pair states with $p\neq 0$.

We last consider intra-molecular dipole-dipole interactions as a source for
the observed large values of $\delta W$. The induced intra-molecular dipoles
are parallel to $E_{DC}$ and form an angle $\theta$ relative to the
internuclear axis. For constituent polarizabilities estimated at
$\alpha_{M}/2$, the induced dipole-dipole interaction is
$W_{I}=E_{DC}^{2}(\alpha_{M}^{2}/4)(1-3\cos^{2}\theta)/(4\pi\epsilon_{0}R^{3})$.
With $\theta$ varying between 0 and $\pi$, the variation $\delta
W_{I}=E_{DC}^{2}(3\alpha_{M}^{2})/(16\pi\epsilon_{0}R^{3})$. For
$39D_{5/2}\,37F_{J}$ at $R=2~{}\mu$m, it is $\delta W_{I}\gtrsim h\times 100$
kHz. For a given vibrational state, $\delta W_{I}$ would likely be higher
because it would have to be properly averaged over $1/R^{3}$. Hence, induced
dipole-dipole interactions may add to the Stark linewidths and help explain
the results in Figs. 4 and 5. Since both the discussed $\delta W_{S}$ and
$\delta W_{I}$ scale as $E_{DC}^{2}$, the net linewidths in Fig. 5 should
follow a function $\delta W=a\sqrt{1+b\,E_{DC}^{4}}$. This function, applied
in Fig. 5, fits the data indeed quite well.

In summary, we have observed FS-induced $(n+2)D_{5/2}\,nF_{J}$ Cs Rydberg
macro-dimers, modeled the data, and explained the formation mechanisms of
these molecules. Different from the commonly used method of laser photo-
association, we have employed microwave photo-association, which is narrow-
band and sensitive to van-der-Waals interactions of the launch Rydberg-pair
states. In the future, improved atom cooling methods, lattice-trapped Rydberg
atoms, and Franck-Condon tuning principles could be used to reduce the
linewidths and to expand the utility of microwave photo-association
spectroscopy. The research may reveal detailed vibrational structures suitable
to test and improve models for Born-Oppenheimer potentials, including the
effect of non-adiabatic couplings on vibrational-level energies and lifetimes.
Future research into the mixing of Rydberg macro-dimers with high-$\ell$
levels may shed light onto unresolved issues regarding our measured molecular
polarizabilities. Intra-molecular induced-dipole-dipole interactions in
Rydberg macro-dimers, for which we have found indications, may also deserve
additional work.

We gratefully acknowledge the National Natural Science Foundation of China
(Grants No. 12120101004, No. 12241408, No. 61835007, and No. 62175136); the
Changjiang Scholars and Innovative Research Team in Universities of the
Ministry of Education of China (Grant No. IRT 17R70); and the 1331 project of
Shanxi Province. G.R. acknowledges support by the University of Michigan.

## References

  * Greene  _et al._ (2000) Chris H. Greene, A. S. Dickinson, and H. R. Sadeghpour, “Creation of polar and nonpolar ultra-long-range Rydberg molecules,” Phys. Rev. Lett. 85, 2458–2461 (2000).
  * Hamilton  _et al._ (2002) Edward L Hamilton, Chris H Greene, and H R Sadeghpour, “Shape-resonance-induced long-range molecular Rydberg states,” Journal of Physics B: Atomic, Molecular and Optical Physics 35, L199 (2002).
  * Khuskivadze  _et al._ (2002) A. A. Khuskivadze, M. I. Chibisov, and I. I. Fabrikant, “Adiabatic energy levels and electric dipole moments of Rydberg states of Rb2 and Cs2 dimers,” Phys. Rev. A 66, 042709 (2002).
  * Bendkowsky  _et al._ (2009) Vera Bendkowsky, Björn Butscher, Johannes Nipper, James Shaffer, Robert Löw, and Tilman Pfau, “Observation of ultralong-range Rydberg molecules,” Nature 458, 1005–8 (2009).
  * Anderson  _et al._ (2014) D. A. Anderson, S. A. Miller, and G. Raithel, “Photoassociation of long-range $n{D}$ Rydberg molecules,” Phys. Rev. Lett. 112, 163201 (2014).
  * Krupp  _et al._ (2014) A. T. Krupp, A. Gaj, J. B. Balewski, P. Ilzhöfer, S. Hofferberth, R. Löw, T. Pfau, M. Kurz, and P. Schmelcher, “Alignment of ${D}$-state Rydberg molecules,” Phys. Rev. Lett. 112, 143008 (2014).
  * Booth  _et al._ (2015) D. Booth, S. T. Rittenhouse, J. Yang, H. R. Sadeghpour, and J. P. Shaffer, “Production of trilobite Rydberg molecule dimers with kilo-Debye permanent electric dipole moments,” Science 348, 99–102 (2015).
  * Niederprüm  _et al._ (2016) Thomas Niederprüm, Oliver Thomas, Tanita Eichert, Carsten Lippe, Jesús Pérez-Ríos, Chris H. Greene, and Herwig Ott, “Observation of pendular butterfly Rydberg molecules,” Nature Communications 7, 12820 (2016).
  * Saßmannshausen  _et al._ (2015) Heiner Saßmannshausen, Frédéric Merkt, and Johannes Deiglmayr, “Experimental characterization of singlet scattering channels in long-range Rydberg molecules,” Phys. Rev. Lett. 114, 133201 (2015).
  * Bai  _et al._ (2020) Suying Bai, Xiaoxuan Han, Jingxu Bai, Yuechun Jiao, Jianming Zhao, Suotang Jia, and Georg Raithel, “Cesium $n{D}_{J}+6{S}_{1/2}$ Rydberg molecules and their permanent electric dipole moments,” Phys. Rev. Res. 2, 033525 (2020).
  * Duspayev  _et al._ (2021) A. Duspayev, X. Han, M. A. Viray, L. Ma, J. Zhao, and G. Raithel, “Long-range Rydberg-atom–ion molecules of Rb and Cs,” Phys. Rev. Res. 3, 023114 (2021).
  * Deiß  _et al._ (2021) Markus Deiß, Shinsuke Haze, and Johannes Hecker Denschlag, “Long-Range Atom–Ion Rydberg Molecule: A Novel Molecular Binding Mechanism,” Atoms 9, 34 (2021), 2105.00845 .
  * Zuber  _et al._ (2022) Nicolas Zuber, Viraatt Anasuri, Moritz Berngruber, Yi-Quan Zou, Florian Meinert, Robert Löw, and Tilman Pfau, “Observation of a molecular bond between ions and Rydberg atoms,” Nature 605, 453–456 (2022).
  * Zou  _et al._ (2023) Yi-Quan Zou, Moritz Berngruber, Viraatt S. V. Anasuri, Nicolas Zuber, Florian Meinert, Robert Löw, and Tilman Pfau, “Observation of vibrational dynamics of orientated Rydberg-atom-ion molecules,” Phys. Rev. Lett. 130, 023002 (2023).
  * Boisseau  _et al._ (2002) Christophe Boisseau, Ionel Simbotin, and Robin Côté, “Macrodimers: Ultralong range Rydberg molecules,” Phys. Rev. Lett. 88, 133004 (2002).
  * Overstreet  _et al._ (2009) K. R. Overstreet, A. Schwettmann, J. Tallant, D. Booth, and J. P. Shaffer, “Observation of electric-field-induced Cs Rydberg atom macrodimers,” Nature Physics 05, 581–585 (2009).
  * Deiglmayr  _et al._ (2014) Johannes Deiglmayr, Heiner Saßmannshausen, Pierre Pillet, and Frédéric Merkt, “Observation of dipole-quadrupole interaction in an ultracold gas of Rydberg atoms,” Phys. Rev. Lett. 113, 193001 (2014).
  * Saßmannshausen and Deiglmayr (2016) Heiner Saßmannshausen and Johannes Deiglmayr, “Observation of Rydberg-atom macrodimers: Micrometer-sized diatomic molecules,” Phys. Rev. Lett. 117, 083401 (2016).
  * Han  _et al._ (2018) Xiaoxuan Han, Suying Bai, Yuechun Jiao, Liping Hao, Yongmei Xue, Jianming Zhao, Suotang Jia, and Georg Raithel, “Cs $62{D}_{J}$ Rydberg-atom macrodimers formed by long-range multipole interaction,” Phys. Rev. A 97, 031403(R) (2018).
  * Hollerith  _et al._ (2019) Simon Hollerith, Johannes Zeiher, Jun Rui, Antonio Rubio-Abadal, Valentin Walther, Thomas Pohl, Dan M. Stamper-Kurn, Immanuel Bloch, and Christian Gross, “Quantum gas microscopy of Rydberg macrodimers,” Science 364, 664–667 (2019).
  * Hollerith  _et al._ (2022) Simon Hollerith, Kritsana Srakaew, David Wei, Antonio Rubio-Abadal, Daniel Adler, Pascal Weckesser, Andreas Kruckenhauser, Valentin Walther, Rick van Bijnen, Jun Rui, Christian Gross, Immanuel Bloch, and Johannes Zeiher, “Realizing distance-selective interactions in a Rydberg-dressed atom array,” Phys. Rev. Lett. 128, 113602 (2022).
  * Hollerith and Zeiher (2023) Simon Hollerith and Johannes Zeiher, “Rydberg macrodimers: Diatomic molecules on the micrometer scale,” The Journal of Physical Chemistry A 127, 3925–3939 (2023).
  * Gallagher (1994) Thomas F. Gallagher,  _Rydberg Atoms_, Cambridge Monographs on Atomic, Molecular and Chemical Physics (Cambridge University Press, 1994).
  * Han  _et al._ (2019) Xiaoxuan Han, Suying Bai, Yuechun Jiao, Georg Raithel, Jianming Zhao, and Suotang Jia, “Adiabatic potentials of cesium $(n{D}_{J})_{2}$ Rydberg–Rydberg macrodimers,” Journal of Physics B: Atomic, Molecular and Optical Physics 52, 135102 (2019).
  * Bai  _et al._ (2023) Jingxu Bai, Rong Song, Jiabei Fan, Yuechun Jiao, Jianming Zhao, Suotang Jia, and Georg Raithel, “Quantum defects of $n{F}_{J}$ levels of Cs Rydberg atoms,” Phys. Rev. A 108, 022804 (2023).
  * (26) “Supplemental computations,” URL_will_be_inserted_by_publisher.
  * Samboy  _et al._ (2011) Nolan Samboy, Jovica Stanojevic, and Robin Côté, “Formation and properties of Rydberg macrodimers,” Phys. Rev. A 83, 050501(R) (2011).
  * Samboy and Côté (2011) Nolan Samboy and Robin Côté, “Rubidium Rydberg macrodimers,” Journal of Physics B: Atomic, Molecular and Optical Physics 44, 184006 (2011).
  * Malinovsky  _et al._ (2020) V. S. Malinovsky, K. R. Moore, and G. Raithel, “Modulation spectroscopy of Rydberg atoms in an optical lattice,” Phys. Rev. A 101, 033414 (2020).
  * Cardman and Raithel (2023) R. Cardman and G. Raithel, “Driving alkali Rydberg transitions with a phase-modulated optical lattice,” Phys. Rev. Lett. 131, 023201 (2023).
  * Ravon  _et al._ (2023) B. Ravon, P. Méhaignerie, Y. Machu, A. Durán Hernández, M. Favier, J. M. Raimond, M. Brune, and C. Sayrin, “Array of individual circular Rydberg atoms trapped in optical tweezers,” Phys. Rev. Lett. 131, 093401 (2023).

##  I Supplement

###  I.1 Adiabatic potentials

To model the experimental spectra, we consider the product Hilbert space of
two Rydberg atoms, and add the multipole interaction between the atoms to the
unperturbed two-body Hamiltonian. The molecular potential energy curves are
obtained by diagonalization of the Rydberg-pair Hamiltonian within basis sub-
sets of states with conserved $M$, the sum of the projections of the total
electronic angular momenta onto the internuclear axis, and conserved parity
(which depends on whether the sum of the orbital electronic angular momenta is
even or odd). We include multipolar interactions up to order $L_{a}+L_{b}$=6,
with the dipole, quadrupole etc. terms for atoms “a” and “b” denoted
$L_{a}=1,2,...$ and $L_{b}=1,2,...$. The multipole interactions scale as
$n^{2(L_{a}+L_{b})}/R^{1+L_{a}+L_{b}}$, where $R$ is the internuclear distance
and $n$ the average principal quantum number of the Rydberg-pair states of
interest. The leading interaction of a pair of atoms in states
$a=(n+2)D_{5/2}$ and $b=nF_{J}$, which are of main interest in the present
work, is the dipole-dipole interaction, which scales as $n^{4}/R^{3}$. The
interactions drop rapidly in importance with increasing order $L_{a}+L_{b}$.
In our calculations we are setting the cutoff at $L_{a}+L_{b}\leq 6$, which is
generous. More details describing our calculations are found in References Han
_et al._ (2018, 2019) in the main text.

###  I.2 Excitation rates

To compute model spectra, we consider the microwave excitation rates from the
optically-prepared launch states $[(n+2)D_{5/2}]_{2}$ into pair states
$(n+2)D_{5/2}\,nF_{J}$. At the $R$-values of the $DF$-macro-dimers of
interest, molecular-potential calculations for the launch $[(n+2)D_{5/2}]_{2}$
pair states show positive van-der-Waals level shifts of $\lesssim 2$ MHz for
$39D$. Hence, within a reasonable approximation, we can assume that the
$[(n+2)D_{5/2}]_{2}$ pair states are only weakly perturbed. We therefore
compute the microwave excitation rates between non-interacting
$[(n+2)D_{5/2}]_{2}$ atom pairs in the microwave transition’s ground state,
and strongly-perturbed and -mixed $(n+2)D_{5/2}\,nF_{J}$ atom pairs in the
microwave transition’s excited state. This allows us to use the same methods
as described in References Han  _et al._ (2018, 2019) in the main text. The
rates, averaged over the angular alignment of the atom pairs relative to the
microwave field, are shown by the areas of the dots in Fig. 2 (a) in the main
text. To model the experimental spectra in a two-body approximation, in which
interactions between more than two atoms are ignored, we then compute
radially-averaged model spectra in which we sum over all $M$-values and
parities, and we apply a radial weighting factor
$(3/r)(R/r)^{2}\exp[-(R/r)^{3}]$ (the nearest-neighbor distribution in an
ideal gas, with $r$ denoting the Wigner-Seitz radius). Typical results are
shown in Fig. 2 (c) in the main text for two indicated values of $r$.

###  I.3 Atomic electric polarizabilities

We first calculate the polarizabilities of the atomic constituents (using
private codes). For the $39D_{5/2}\,37F_{J}$ macro-dimer, these are
$a_{s}(39D_{5/2})=-11.7$ kHz/(V/m)2, $a_{t}(39D_{5/2})=13.0$ kHz/(V/m)2,
$a_{s}(37F_{5/2})=75.3$ kHz/(V/m)2, $a_{t}(37F_{5/2})=-26.4$ kHz/(V/m)2,
$a_{s}(37F_{7/2})=74.7$ kHz/(V/m)2, and $a_{t}(37F_{7/2})=-30.5$ kHz/(V/m)2,
where the subscripts $s$ and $t$ stand for scalar and tensor polarizabilities.
We recite that for a given $m_{J}$, with the direction of $E_{DC}$ taken as
the quantization axis, the polarizability is
$\alpha(\lambda)=a_{s,\lambda}+a_{t,\lambda}(3m_{J}^{2}-J(J+1))/J/(2J-1))$,
and the level shift is $s(\lambda)=-(1/2)\alpha(\lambda)E_{DC}^{2}$, where
$|\lambda\rangle$ is a short-hand for the quantum state
$|n,\ell,J,m_{J}\rangle$ in the $E_{DC}$-frame. Taking an isotropic average,
both $37F_{J}$ lines in Fig. 4 in the main text are expected to shift
according to the average differential polarizability between the upper
$37F_{J}$ and the lower $39D_{5/2}$ states. This average computes to be
$\approx 85$ kHz/(V/m)2, which agrees within uncertainties with the values
derived from the Stark-shift fits in Fig. 4 of the main text. For completeness
we recall here that isotropic averages of the tensor polarizabilities are
zero.

###  I.4 Molecular electric polarizabilities

The symmetrized state of a Rydberg pair is of the form

$\displaystyle|\psi_{\rm{mol}}\rangle$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{2}}\sum_{\lambda_{a}\lambda_{b}}c_{ab}(|\lambda_{a}\lambda_{b}\rangle-p(-1)^{\ell_{a}+\ell_{b}}|\lambda_{b}\lambda_{a}\rangle)$

with $\lambda_{a(b)}=|n_{a(b)},\ell_{a(b)},J_{a(b)},m_{J_{a(b)}}\rangle$
denoting single-atom states in the molecular frame, and $a$ and $b$ being
subscripts for the two atoms. The coefficients $c_{ab}(R)$ are obtained in
molecular-potential calculations explained above. Rotating the state from the
molecular into the $E_{DC}$-frame with (reduced) rotation operators
$d^{(J)}_{\tilde{m_{J}},m_{J}}(\theta)$ [see, e. g., J. J. Sakurai and J.
Napolitano, “Modern Quantum Mechanics”, Cambridge University Press (2020)]
with $\theta$ denoting the angle between $E_{DC}$ and the internuclear axis,
the molecular state in the $E_{DC}$-frame takes the form

$\displaystyle|\psi_{\rm{mol}}\rangle$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{2}}\sum_{\tilde{\lambda}_{a}\tilde{\lambda}_{b}}\tilde{c}_{ab}(|\tilde{\lambda}_{a}\tilde{\lambda}_{b}\rangle-p(-1)^{\ell_{a}+\ell_{b}}|\tilde{\lambda}_{b}\tilde{\lambda}_{a}\rangle),$

with the transformed coefficients

$\tilde{c}_{ab}=c_{ab}d^{(J_{a})}_{\tilde{m}_{Ja},m_{Ja}}d^{(J_{b})}_{\tilde{m}_{Jb},m_{Jb}}\quad.$

In a perturbative picture, which should apply as long as the molecular Stark
shift is much less than the binding energy (a few MHz in our case), the
leading contribution of the molecular polarizability is the sum of the
polarizabilities of the atomic constituents. This yields a theoretical
molecular polarizability of

$\alpha_{M,T}=\sum_{\tilde{\lambda}_{a}\tilde{\lambda}_{b}}|\tilde{c}_{ab}|^{2}[\alpha(\tilde{\lambda}_{a})+\alpha(\tilde{\lambda}_{b})]\quad.$
(1)

The result is averaged over $\theta$ with the normalized weighting function
$\sin(\theta)/2$ with $\theta$ ranging from 0 to $\pi$.

We have performed the calculation for electronic adiabatic states in the
potential wells in Fig. 2 (a) of the main text and found $\alpha_{M,T}$-values
close to the sum of the $39D_{5/2}$ and $37F_{J}$ scalar atomic
polarizabilities. These are also the atomic states relevant to Fig. 4 of the
main text. Noting that the adiabatic states in the potential wells in Fig. 2
(a) of the main text carry only very small contributions from states with
$\ell\neq 2,3$, for the basis sets used, this result is not very surprising.
However, $\alpha_{M,T}$ amounts to only $\approx 1/2.7$ of the measured value,
$\alpha_{M}$.