# CCATMos: Convolutional Context-aware Transformer Network for Non-intrusive
Speech Quality Assessment

###### Abstract

Speech quality assessment has been a critical component in many voice
communication related applications such as telephony and online conferencing.
Traditional intrusive speech quality assessment requires the clean reference
of the degraded utterance to provide an accurate quality measurement. This
requirement limits the usability of these methods in real-world scenarios. On
the other hand, non-intrusive subjective measurement is the “golden standard”
in evaluating speech quality as human listeners can intrinsically evaluate the
quality of any degraded speech with ease. In this paper, we propose a novel
end-to-end model structure called Convolutional Context-Aware Transformer
(CCAT) network to predict the mean opinion score (MOS) of human raters. We
evaluate our model on three MOS-annotated datasets spanning multiple languages
and distortion types and submit our results to the ConferencingSpeech 2022
Challenge. Our experiments show that CCAT provides promising MOS predictions
compared to current state-of-art non-intrusive speech assessment models with
average Pearson correlation coefficient (PCC) increasing from 0.530 to 0.697
and average RMSE decreasing from 0.768 to 0.570 compared to the baseline model
on the challenge evaluation test set.

Index Terms: Non-intrusive Speech Quality Assessment, Speech Quality, Deep
Learning

##  1 Introduction

Evaluation of speech quality is a key component of any audio application
involving voice communications including telecommunications, consumer
electronics and hearing assistance. Objective intrusive speech quality
assessment is widely used to evaluate speech quality from various speech
processing tasks such as speech enhancement [1, 2], dereverbration [3] and
speech synthesis [4]. Perceptual Evaluation of Speech Quality (PESQ) [5] is
designed to test the speech and voice quality of speech as perceived by human
beings. Short-Time Objective Intelligibility (STOI) [6] focuses more on
intelligibility measurement. Hearing-Aid Speech Quality Index (HASQI) [7] is a
method created specifically to evaluate speech quality for hearing aid users.

All of these objective intrusive methods require a clean reference speech
signal for evaluating the quality of the given utterance. However, a clean
reference is often unavailable in real-world situations. Because of this
limitation, intrusive speech assessment methods are unable to be used in most
real-world voice communications scenarios, such as real-time phone call
quality, on-device acoustic noise suppression and virtual meeting speech
quality assessments.

On the other hand, objective non-intrusive speech assessment methods do not
require a clean reference speech signal for the evaluation process, which
increases their usability. The current objective non-intrusive specification
provided by ITU-T is P.563 [8]. P.563 is the first non-intrusive speech
quality assessment model designed for telephony applications. It assesses the
quality of speech transmitted over a narrow-band telephone channel. However,
P.563 can only applied on narrow-band signals with certain length, which
limits its applicability. Other attempts at object non-intrusive speech
assessment methods such as ANIQUE [9], and the speech-to-reverberation
modulation energy ratio (SRMR) [10] do not show enough correlation to
subjective human assessment.

Recently, there has been flurry of work aimed at making objective evaluation
methods non-intrusive by applying data-driven techniques. The approach of
these data-driven methods is often to pose intrusive speech quality assessment
as a regression problem where the goal is to map directly from the audio to
the evaluation score, eliminating the requirement of a reference signal.
Quality-Net [11] uses a bidirectional long short-term memory (BLSTM) based
network to predict both frame-level and utterance-level PESQ scores.
Similarly, STOI-Net [12] predicts the STOI by using a CNN-BLSTM network. Zhang
et al. [13] implement a multi-task framework and predict mean opinion score
(MOS), PESQ, the extended short-time objective intelligibility (eSTOI) and
signal to distortion ratio (SDR) simultaneously with a pyramid BLSTM network.

While objective metrics are valuable, subjective speech assessment by human
raters is still considered the “golden standard” of speech quality assessment
[14] since human perception of audio quality is highly subjective [15].
Subjective assessments are often conducted by aggregating perceptual ratings
from multiple raters ranging from 1 (worst) to 5 (best) into a single MOS
score per utterance. In contrast to objective metrics, raters can recognize
and evaluate degraded utterances easily with minimal reference. ITU-T P.808
[16] defines a framework to reproducibly collect MOS-annotated data by
providing tools for ensuring inter-rater reliability and spam filtering.
Unfortunately, ratings can be noisy due to inherent differences in human
perception even when adhering to current best practices [17]. Moreover,
subjective speech assessment is costly, complex and time consuming, severely
limiting its scalability.

Due to the noisy and nonlinear nature of perceptual speech quality, deep
learning-based data-driven approaches to speech quality assessment have moved
to the forefront of non-intrusive speech quality research. Recent work has
tackled the problem with several neural network designs. For instance, MOSNet
[18] uses a convolutional neural network (CNN) followed by a BLSTM to predict
both frame level and utterance level scores. DNSMOS [14] uses a self teaching
[19] approach to train a CNN-based model. AutoMOS [20] combines a
classification loss with a regression loss to predict more accurate MOS
scores. NISQA (v2.0) [21] uses a CNN to extract frame-wise features and
attention model to estimate the MOS score.

In this paper, we propose a novel end-to-end non-intrusive speech assessment
method which directly estimates MOS scores for varied real-world speech. The
proposed model takes a raw speech signal as input and generates context-aware
embeddings to capture abundant contextual information of the given speech at
multiple timescales. Several convolutional layers are then used to exploit
local features, followed by several transformer encoder blocks to aggregate
long-range utterance level features. A MOS score is predicted for each frame
and the final utterance MOS score is calculated by averaging all the predicted
frame-level predictions.

The details of our proposed model are provided in section 2 and the dataset
details are shown in section 3. Experimental results and conclusions can be
found in section 4 and 5, respectively.

Figure 1: Overall Model Structure, the components of transformer encoder are
illustrated in Figure2.

##  2 Method

In this section, we describe the detailed model structure and experimental
method for our MOS estimator. An illustration of the model architecture can be
found in Fig. 1. First, the input signal is converted into a time-frequency
(T-F) domain log-scaled magnitude spectrogram. The spectrogram is then
aggregated to generate context-aware features. Next, the context-aware
features are fed into a time-distributed neural network with three 2D
convolutional blocks, a masking layer, a transformer encoder block and several
fully connected (FC) layers. A MOS score is predicted for each context-frame
as well as for the overall utterance. The final utterance-level predicted MOS
score is calculated by average-pooling the predicted frame-level MOS.

###  2.1 Feature Extraction

We use the short-time Fourier transform (STFT) [22] to extract the T-F domain
spectrogram for each time domain input signal. The log-scaled magnitude
spectrogram is calculated next, resulting in a signal $x\in\mathbb{R}^{T\times
F}$. In order to capture longer-term contextual information in the time-
distributed network, we use context-aware frames to maintain the temporal
information during the training process. For each frame in the log-scaled
magnitude spectrogram $x_{i}$, $n$ past frames and $n$ future frames are
concatenated with the current frame $x_{i}$ to generate a context-aware frame.
Frames at the beginning and end of the signal are zero-padded where necessary.
The context size $C$ can be calculated as $2n+1$. The resultant context-aware
features can be expressed as a three-dimensional tensor
$c\in\mathbb{R}^{T\times F\times C}$.

###  2.2 Model Structure

The overall regression network mainly contains two components, a time-
distributed network used to provide the frame-level estimated MOS score and an
average pooling layer used to average the frame-level predicted MOS to the
final utterance-level predicted MOS score.

The time-distributed network contains three parts: 2D convolutional blocks,
transformer encoder blocks [23] and several fully connected (FC) layers. The
2D convolutional blocks are used to extract local contextual features. They
contain three 2D convolutional layers with kernel size $[5,5]$ and stride 1.
Each convolutional layer is followed by a ReLU activation function and an
average pooling layer with the pool size $[2,2]$. The downsampling rate for
each convolutional block is $[8,8]$, which results in an output embedding
dimension of $[T,\frac{F}{8},\frac{C}{8}]$. We choose not to use a bias vector
in order to allow the subsequent masking layer to stabilize training.

Next, the masked context-aware convolutional results are fed into several
transformer encoder blocks. The transformer encoders are used to capture
global contextual interactions at the utterance level. Each block consists of
a multi-head self-attention layer with skip connection and a fully connected
layer (illustrated in Fig.2). This is followed by two FC layers with ReLU
activation and with a maximum value of 5 in order to predict the frame-level
MOS score $\hat{M}_{x,1}...\hat{M}_{x,T}$. Finally, an average pooling layer
is used to combine the frame-level MOS predictions into an utterance-level
MOS. We apply dropout and L2 regularization throughout the network to prevent
overfitting.

| Model 1 | Model 2 | Model 3
---|---|---|---
feature | STFT | Mel | Mel
context size | 11 | 11 | 11
Conv filters | 16 | 8 | 16
Conv filter size | 5 | 5 | 5
Transformer encoders | 4 | 2 | 4
Transformer FC units | 256 | 256 | 256
Transformer Att heads | 4 | 2 | 4
FC units | 512 | 256 | 512
FC layers | 2 | 2 | 2
batch size | 4 | 16 | 4
dropout | 0.15 | 0.19 | 0.15
learning rate | 1e-5 | 4.2e-4 | 1e-5
l2 regularizer | 6e-3 | 1e-3 | 6e-3
Table 1: Network configuration for the final models. Note that Model 3 has an
identical configuration with Model 1 except the input feature type, this is
due to the experiment on feature variants is conducted after the hyper-
parameter search. Figure 2: Transformer Encoder.

###  2.3 Loss Function

Inspired by QualityNet [11], we use a MOS-weighted mean squared error (MSE)
loss for frame-level and a regular MSE loss for utterance-level MOS
prediction.

The utterance-level MOS ground truth $M_{x}$is also used as the frame-level
MOS ground truth. However, the gap between utterance-level MOS score and the
frame-level MOS score increases as the quality of the utterance decreases.
Therefore, a conditional weight is added to the frame-level MSE to de-
emphasize this bias. The conditional weights are calculated as:

$\alpha\left(M_{x}\right)=10^{\left(M_{x}-5\right)}$ (1)

where 5 is upper-bound of the MOS score. With $X$ denoting the total number of
training utterances and $T$ as the number of frames in each utterance, $Ux$,
the final loss can be expressed as:

$L=\frac{1}{X}\sum_{x=1}^{X}\left[\left(M_{x}-\hat{M}_{x}\right)^{2}+\frac{\alpha\left(M_{x}\right)}{T}\sum_{t=1}^{T(U_{x})}\left(M_{x}-\hat{M}_{x,t}\right)^{2}\right]$

(2)

###  2.4 Hyper-parameter Tuning and Ensemble Modeling

We perform hyper-parameter tuning using a Bayesian optimization strategy [24]
to search for the best combination of network parameters. The parameter search
space includes the following: context size $C$ for the input features; filter
size and number of filters for convolutional layers; number of transformer
encoders, attention heads, and feed-forward units in the transformer encoder;
hidden units and number of layers for FC layers; and finally, batch size,
learning rate, dropout rate, lambda for L2 regularizer for training
configurations.

We implement the hyper-parameter search for 24 runs and select the top two
parameter sets as our candidate networks. In addition, we train two feature
variants with log-magnitude STFT spectrograms and log-magnitude mel
spectrograms on each candidate network, resulting in four candidate models.
Finally, we perform model ensembling by selecting three best performing models
based on the evaluation results and average their output as the final MOS
prediction. The model parameters of the three selected models are shown in
Table 1.

##  3 Datasets

We use the dataset provided by the ConferencingSpeech 2022
challenge111https://github.com/ConferencingSpeech/ConferencingSpeech2022.
Details about the dataset can be found in [25]. The full training dataset is a
composite of three separate corpora. The first is the Tencent corpus, which
contains 8,366 utterances from reverberant environments and 3,197 utterances
from non-reverberant ones. All speech clips are in Chinese. In addition to
reverberation, all of the utterances contain distortions which would regularly
occur in online conferencing such as background noise, packet loss and network
jitter. The MOS label is generated using a crowdsourcing method based on ITU-T
P.808 which provide a MOS score from 1 to 5. The second dataset is the NISQA
corpus. The NISQA corpus is an open source dataset containing 14,432 English
utterances. The dataset is pre-partitioned into training, validation and test
sets and each set contains utterances with both real and simulated network
degradations, similar to Tencent. MOS labels are also crowdsourced in a
similar way to the Tencent dataset. The last dataset is the PSTN corpus which
consists of English Librivox utterances with additive noise pushed through a
PSTN send to a VoIP receive endpoint. The dataset contains 58,709 utterances
each with a fixed length of 10 seconds.

Since the Tencent and PSTN corpora are released without training and
development partitions, we use the following procedure to define our own: we
first extract audio embeddings for each recording using OpenL3 [26], and
leverage the embedding as the audio feature for the Kennard-Stone algorithm
[27] to split 90% of each corpus into our training set and the remaining 10%
as our development set. As for the NISQA corpus, we include the LIVETALK and
FOR partition of the NISQA TEST corpus as our development set, and place the
remaining NISQA data into the training set. Note that we include the P.501
portion of the NISQA TEST data in our training set to enhance the domain
diversity of the training data.

The final evaluation test set provided by the challenge contains three sub-
partitions, Tencent, PSTN and TUB. The Tencent test set include 2,898 Chinese
speech clips with 799 clips from reverberant environments and 2099 clips from
environments without reverbration. The TUB eval dataset contains 434 German
and English utterances and the PSTN eval dataset contains 1,454 ten second
speech clips.

##  4 Experimental Results

In this section, we first present two sets of ablation studies to  (i) verify
the benefit of context-aware features and  (ii) highlight a cross-dataset
generalization error issue that we observed.  We present the individual
results of our final models along with the benchmarks against two recent works
on our development set, and attach the ConferencingSpeech 2022 challenge
results.

The challenge provides two baseline models. The models are trained on random
splits of the challenge datasets where the specification of train and
development split of the datasets are not available. As a result, we are
unable to compare our models directly with the baselines with the available
data. Hence, we benchmark the latest NISQA [21] model (NISQA v2.0) and the
DNSMOS [14] model on our development datasets to provide a fair comparison
with our proposed methods. We evaluate our results using average Pearson’s
correlation coefficient (PCC) and root mean squared error (RMSE).

Dataset |  NISQA[21] |  DNSMOS[14] | Model 1 | Model 2 | Model 3 | CCATMOS (Ensemble)
---|---|---|---|---|---|---
PCC | RMSE | PCC | RMSE | PCC | RMSE | PCC | RMSE | PCC | RMSE | PCC | RMSE
Tencent Dev | 0.719 | 0.953 | 0.737 | 0.888 | 0.872 | 0.526 | 0.874 | 0.516 | 0.882 | 0.502 | 0.901 | 0.452
PSTN Dev | 0.683 | 1.071 | 0.639 | 0.754 | 0.754 | 0.605 | 0.750 | 0.591 | 0.766 | 0.573 | 0.779 | 0.559
NISQA _TEST_LIVETALK | 0.778 | 0.617 | 0.580 | 0.943 | 0.708 | 0.737 | 0.747 | 0.720 | 0.731 | 0.718 | 0.779 | 0.646
NISQA _TEST_FOR | 0.891 | 0.418 | 0.586 | 1.327 | 0.763 | 0.619 | 0.732 | 0.681 | 0.804 | 0.634 | 0.827 | 0.541

Table 2: Results on our own division of development sets

###  4.1 Ablation Study

Dataset | w/o context | w/o PSTN Train
---|---|---
PCC | RMSE | PCC | RMSE
Tecent Dev | 0.858 | 0.564 | 0.863 | 0.544
PSTN Dev | 0.725 | 0.650 | 0.654 | 0.680
NISQA_TEST_LIVETALK | 0.698 | 0.810 | 0.699 | 0.767
NISQA_TEST_FOR | 0.756 | 0.642 | 0.761 | 0.588

Table 3: Ablation studies

####  4.1.1 Context-aware features

In Table 3, we perform ablation studies on the context-aware features by
replacing them with frame-wise inputs, while keeping the remaining model
configuration identical to Model 1. Compared to the results without context-
aware features in Table 3, Model 1 in Table 4 shows across the board
performance improvements with PCC scores up and RMSE down by an average of
0.015 and 0.045, respectively, across the development sets.

####  4.1.2 Generalization error across datasets

In our benchmarks of two previously published works [21, 14], we notice
significant performance inconsistencies across our development datasets. As
shown in Table 4, the NISQA model performs noticeably better on the NISQA test
datasets, whereas the DNSMOS struggles most on them (i.e. NISQA shows the
highest PCC at 0.891 on the NISQA FOR testset, but an average of around 0.702
on the two challenge testsets; DNSMOS scores below 0.586 PCC on both NISQA
testsets). This raises concerns about model generalization across varied
domains.

We conduct further ablation studies by removing the PSTN training data in the
training of Model 1, the result of which is shown in Table 3. We notice the
performance remains similar in the other three development sets but drops off
significantly in the PSTN development set, where PCC decreases from 0.754 to
0.654.

Although a thorough discussion of generalization error [28] in MOS prediction
is beyond the scope of this work, we highlight the fact that model performance
can be highly dataset sensitive in this task, even for corpora with outwardly
similar content.

###  4.2 Convolutional Context-aware Transformer

Moving on to our proposed models, the individual results for the three
selected models are presented in Table 4, we note that despite Model 3
outperforming Model 1 and Model 2 on all development datasets, the ensemble
prediction further improves overall robustness in both PCC and RMSE. Comparing
to the work of NISQA and DNSMOS, we are able to achieve significantly better
performance on the two challenge datasets, with PCC around 0.901 and RMSE
around 0.452 for the Tencent corpus and PCC around 0.779 and RMSE around 0.559
for the PSTN corpus. However, NISQA retains the best performance on its own
test corpus.

Finally, we present the results for the challenge evaluation test set in Table
4. The baseline model is a simplified version of the state-of-art NISQA
model[21]. This model uses a mel-spectral input to LSTM layers followed by
several fully connected layers. An average pooling layer then used to average
the predicted frame-level MOS and output the final predicted MOS score. The
metrics also include the epsilon-insensitive RMSE after a 3rd order polynomial
monotonic mapping (RMSE*3rd) [17, 29] which takes into account the confidence
interval of the individual MOS scores.

The results indicate that our proposed model shows better performance across
all three datasets under three different evaluation metrics. Overall, our CCAT
model, reaches a 0.697 PCC while the baseline model only reach 0.53 PCC,
indicating that the our model has a higher correlation to subjective speech
assessment ratings. The RMSE and RMSE*3rd over all eval datasets for our
proposed model is also lower than the baseline model. The proposed model
reaches a 0.57 RMSE and 0.401 RMSE*3rd compare to 0.768 RMSE and 0.497
RMSE*3rd from the baseline model. Therefore, we can conclude that our proposed
CCAT model shows promising results in the robust prediction of MOS scores
under varied conditions.

Dataset | Baseline 1 | CCATMOS (Ensemble)
---|---|---
PCC | RMSE |  | RMSE*3rd
---
PCC | RMSE |  | RMSE*3rd
---
Tecent Eval | 0.881 | 0.624 | 0.550 | 0.945 | 0.397 | 0.38
TUB Eval | 0.348 | 1.094 | 0.649 | 0.613 | 0.852 | 0.552
MS Eval | 0.361 | 0.585 | 0.293 | 0.532 | 0.459 | 0.271
Average | 0.530 | 0.768 | 0.497 | 0.697 | 0.570 | 0.401

Table 4: Results on the challenge evaluation test set (obtained from the
challenge host). Note that the “average” result is calculated without taking
the quantity of each test set into account. Also note the “MS Eval” result
released from the challenge does not match any of the evaluation test set
specification, here we assume this is the PSTN eval dataset.

##  5 Conclusions

Subjective quality assessment by human raters is the best method available for
assessing speech quality. However, it is time consuming and costly to conduct.
In this paper, we proposed a novel end-to-end non-intrusive speech quality
assessment model called convolutional context-aware transformer network or
CCAT. The proposed model architecture provides an accurate MOS prediction
across both real and simulated conditions. The overall performance of our
model is further improved by using hyper-parameter tuning and model ensembling
techniques.

We also highlight the issue of poor model generalization across MOS corpora.
We observe this behavior in our model as well as several other models from
recent literature trained on cross-domain dataset splits. We hypothesize that
this is likely a byproduct of the limited scale and variety of current MOS
prediction datasets and may be exacerbated by the inherently noisy nature of
subjective MOS ratings. As future work, we believe the robustness of the
proposed model could be further improved with a larger and more diverse
datasets as well as methods for cross-domain regularization.

## References

  * [1] D. S. Williamson, Y. Wang, and D. Wang, “Complex ratio masking for monaural speech separation,” _IEEE/ACM transactions on audio, speech, and language processing_ , vol. 24, no. 3, pp. 483–492, 2015.
  * [2] M. Stamenovic, N. Westhausen, L.-C. Yang, C. Jensen, and A. Pawlicki, “Weight, block or unit? exploring sparsity tradeoffs for speech enhancement on tiny neural accelerators,” _NeurIPS 2021 Workshop on Efficient Natural Language and Speech Processing_ , 2021.
  * [3] Y. Li, Y. Liu, and D. S. Williamson, “On loss functions for deep-learning based t60 estimation,” in _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2021, pp. 486–490.
  * [4] Y. Wang, R. J. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, Q. V. Le, Y. Agiomyrgiannakis, R. A. J. Clark, and R. A. Saurous, “Tacotron: Towards end-to-end speech synthesis,” in _INTERSPEECH_ , 2017.
  * [5] I.-T. Recommendation, “Perceptual evaluation of speech quality (pesq): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs,” _Rec. ITU-T P. 862_ , 2001.
  * [6] C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “An algorithm for intelligibility prediction of time–frequency weighted noisy speech,” _IEEE Transactions on Audio, Speech, and Language Processing_ , vol. 19, no. 7, pp. 2125–2136, 2011.
  * [7] J. M. Kates and K. H. Arehart, “The hearing-aid speech quality index (hasqi),” _Journal of the Audio Engineering Society_ , vol. 58, no. 5, pp. 363–381, 2010.
  * [8] T. Falk and W. Chan, “Single ended method for objective speech quality assessment in narrowband telephony applications,” _ITU-T_ , p. 563, 2004\.
  * [9] D.-S. Kim, “Anique: An auditory model for single-ended speech quality estimation,” _IEEE Transactions on Speech and Audio Processing_ , vol. 13, no. 5, pp. 821–831, 2005.
  * [10] T. H. Falk, C. Zheng, and W.-Y. Chan, “A non-intrusive quality and intelligibility measure of reverberant and dereverberated speech,” _IEEE Transactions on Audio, Speech, and Language Processing_ , vol. 18, no. 7, pp. 1766–1774, 2010.
  * [11] S.-W. Fu, Y. Tsao, H.-T. Hwang, and H.-M. Wang, “Quality-net: An end-to-end non-intrusive speech quality assessment model based on blstm,” _arXiv preprint arXiv:1808.05344_ , 2018.
  * [12] R. E. Zezario, S.-W. Fu, C.-S. Fuh, Y. Tsao, and H.-M. Wang, “Stoi-net: A deep learning based non-intrusive speech intelligibility assessment model,” in _2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)_. IEEE, 2020, pp. 482–486.
  * [13] Z. Zhang, P. Vyas, X. Dong, and D. S. Williamson, “An end-to-end non-intrusive model for subjective and objective real-world speech assessment using a multi-task framework,” in _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2021, pp. 316–320.
  * [14] C. K. Reddy, V. Gopal, and R. Cutler, “Dnsmos: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors,” in _ICASSP 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2021, pp. 6493–6497.
  * [15] I. Fedorov, M. Stamenovic, C. Jensen, L.-C. Yang, A. Mandell, Y. Gan, M. Mattina, and P. N. Whatmough, “Tinylstms: Efficient neural speech enhancement for hearing aids,” in _Proc. Interspeech 2020_ , 2020.
  * [16] I. Rec, “P. 808,” _Subjective evaluation of speech quality with a crowdsourcing approach. Geneva: International Telecommunication Union_ , 2018.
  * [17] I.-T. Recommendation, “Methods, metrics and procedures for statistical evaluation, qualification and comparison of objective quality prediction models,” _ITU-T Rec. P.1401_ , 2012.
  * [18] C.-C. Lo, S.-W. Fu, W.-C. Huang, X. Wang, J. Yamagishi, Y. Tsao, and H.-M. Wang, “Mosnet: Deep learning based objective assessment for voice conversion,” in _Proc. Interspeech 2019_ , 2019.
  * [19] A. Kumar and V. Ithapu, “A sequential self teaching approach for improving generalization in sound event recognition,” in _International Conference on Machine Learning_. PMLR, 2020, pp. 5447–5457.
  * [20] B. Patton, Y. Agiomyrgiannakis, M. Terry, K. Wilson, R. A. Saurous, and D. Sculley, “Automos: Learning a non-intrusive assessor of naturalness-of-speech,” _NIPS 2016 End-to-end Learning for Speech and Audio Processing Workshop_ , 2016.
  * [21] G. Mittag, B. Naderi, A. Chehadi, and S. Möller, “Nisqa: A deep cnn-self-attention model for multidimensional speech quality prediction with crowdsourced datasets,” _Interspeech 2021_ , Aug 2021. [Online]. Available: http://dx.doi.org/10.21437/Interspeech.2021-299
  * [22] R. N. Bracewell and R. N. Bracewell, _The Fourier transform and its applications_. McGraw-Hill New York, 1986, vol. 31999.
  * [23] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [24] J. Bergstra, B. Komer, C. Eliasmith, D. Yamins, and D. D. Cox, “Hyperopt: a python library for model selection and hyperparameter optimization,” _Computational Science & Discovery_, vol. 8, no. 1, p. 014008, 2015.
  * [25] G. Yi, W. Xiao, Y. Xiao, B. Naderi, S. Möller, G. Mittag, R. Cutler, Z. Zhang, D. S. Williamson, F. Chen _et al._ , “Conferencingspeech 2022 challenge evaluation plan.”
  * [26] J. Cramer, H.-H. Wu, J. Salamon, and J. P. Bello, “Look, listen, and learn more: Design choices for deep audio embeddings,” in _ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2019, pp. 3852–3856.
  * [27] R. W. Kennard and L. A. Stone, “Computer aided design of experiments,” _Technometrics_ , vol. 11, no. 1, pp. 137–148, 1969.
  * [28] Y. S. Abu-Mostafa, M. Magdon-Ismail, and H.-T. Lin, _Learning from data_. AMLBook New York, 2012, vol. 4.
  * [29] G. Mittag and S. Möller, “Non-intrusive speech quality assessment for super-wideband speech communication networks,” in _ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2019, pp. 7125–7129.


# Influence of experimental conditions on localized surface plasmon resonances
measurement by electron energy loss spectroscopy

Michal Horák<EMAIL_ADDRESS>Tomáš Šikola  CEITEC – Central
European Institute of Technology, Brno University of Technology, Purkyňova
123, 612 00 Brno, Czech Republic  Institute of Physical Engineering, Brno
University of Technology, Technická 2, 616 69 Brno, Czech Republic

###### Abstract

Scanning transmission electron microscopy (STEM) combined with electron energy
loss spectroscopy (EELS) has become a standard technique to map localized
surface plasmon resonances with a nanometer spatial and a sufficient energy
resolution over the last 15 years. However, no experimental work discussing
the influence of experimental conditions during the measurement has been
published up to now. We present an experimental study of the influence of the
primary beam energy and the collection semi-angle on the plasmon resonances
measurement by STEM-EELS. To explore the influence of these two experimental
parameters we study a series of gold rods and gold bow-tie and diabolo
antennas. We discuss the impact on experimental characteristics which are
important for successful detection of the plasmon peak in EELS, namely: the
intensity of plasmonic signal, the signal to background ratio, and the signal
to zero-loss peak ratio. We show that the best results are obtained using a
medium primary beam energy, in our case 120 keV, and an arbitrary collection
semi-angle, as it is not a critical parameter at this primary beam energy. Our
instructive overview will help microscopists in the field of plasmonics to
arrange their experiments.

###### keywords:

EELS, electron energy loss spectroscopy, localized surface plasmon resonance,
primary beam energy, collection semi-angle.

††journal: Elsevier

##  1 Introduction

Mapping of localized surface plasmon resonances (LSPR), collective
oscillations of free electrons in metallic nanostructures coupled to the local
electromagnetic field [1, 2, 3], with high spatial and energy resolution is
necessary to understand their origin and properties. The best spatial, energy,
and time resolution is achieved by electron beam spectroscopy [4, 5]. Scanning
transmission electron microscopy (STEM) combined with electron energy loss
spectroscopy (EELS) has become a standard technique to map LSPR with a
nanometer spatial and 10 meV to 100 meV energy resolution over the last 15
years [6, 7, 8, 9, 10].

Despite that many works dealing with EELS measurement of LSPR have been
published, there is no experimental work discussing the experimental
conditions during the measurement. Such an instructive overview has been
missing in literature making it especially difficult for the beginners in the
field. Experimental parameters of the STEM-EELS measurement, such as the
primary beam energy, the convergence semi-angle and the collection semi-angle,
may influence obtained results. Many publications include incomplete
information about these parameters or miss them completely. Moreover, even if
they are in the literature completely included, one can find several different
approaches. Some of them are summarized in Table 1. Primary beam energy is
used in the range from 60 keV to 300 keV. At 60 keV and 80 keV, microscopists
use the collection semi-angle smaller than the convergence semi-angle [11, 10,
12]. At all higher primary beam energies, the collection semi-angle similar to
the convergence semi-angle is used quite often [6, 13, 14, 15, 16, 17]. And
finally at 300 keV, the collection semi-angle larger than the convergence
semi-angle is sometimes used [18, 19, 20, 21]. All presented experimental
results are clear with no suspicion of any bad STEM-EELS measurement.
Consequently, there is no clear conclusion which experimental conditions are
optimal.

Table 1: Experimental conditions of STEM-EELS measurements of LSPR in the literature. Primary | Convergence | Collection |
---|---|---|---
beam energy | semi-angle | semi-angle | Reference
60 keV | 34 mrad | 10 mrad | [10]
80 keV | 11.4 mrad | 2 mrad | [11]
80 keV | 34 mrad | 18 mrad | [12]
100 keV | 7.5 mrad | 6 mrad | [6]
120 keV | 15 mrad | 17 mrad | [13]
200 keV | 9 mrad | 12 mrad | [22]
300 keV | 10 mrad | 6.6 mrad | [23]
300 keV | 13 mrad | 13 mrad | [14]
300 keV | 10 mrad | 10.4 mrad | [15]
300 keV | 15.1 mrad | 17.4 mrad | [16, 17]
300 keV | 15 mrad | 20.5 mrad | [18]
300 keV | 10 mrad | 20.5 mrad | [19, 20]
300 keV | 9.2 mrad | 20.7 mrad | [21]

In our contribution, we present the influence of these experimental
conditions, namely the primary beam energy and the convergence and the
collection semi-angle, on LSPR measurement by STEM-EELS while our figure of
merit is the intensity of plasmonic signal, the signal to background ratio,
and the signal to zero-loss peak ratio considering a limited dynamic range of
the spectrometer camera. Our goal is to find the optimal experimental
conditions and give an instructive overview which would help microscopists in
the field of plasmonics to arrange their experiments.

##  2 Theory

EELS utilizes an electron beam that interacts with the metallic nanoparticle
and excites the LSPR. In consequence, the energy of some electrons decreases
by the characteristic energy of the LSPR. As the result, we observe a peak in
the electron energy loss spectrum at the respect energy loss. Spatially-
resolved EELS further provides (relative) intensity of the near electric field
of a LSPR projected to the trajectory of the electron beam.

The field induced in the specimen by the probing electron acts back on the
electron and decreases its energy with the loss probability density
$\Gamma^{\mathrm{EELS}}(\vec{r_{\mathrm{e}}}(t),\omega)$ reading [4]

$\Gamma^{\mathrm{EELS}}=\frac{e}{\pi\hbar\omega}\int\mathrm{d}t\Re\left\\{\mathrm{e}^{-i\omega
t}\vec{v}\cdot\vec{E}_{\mathrm{el}}^{\mathrm{ind}}(\vec{r_{\mathrm{e}}}(t),\omega)\right\\},$
(1)

where $\hbar\omega$ is the loss energy, $\vec{r_{\mathrm{e}}}(t)$ represents
the electron trajectory, $\vec{v}$ is the electron velocity, and
$\vec{E}_{\mathrm{el}}^{\mathrm{ind}}(\vec{r_{\mathrm{e}}}(t),\omega)$ denotes
the induced parts of the electric field by the fast electron (for example, the
field of LSP). It is calculated by removing the free-space solution of the
electric field from the total computed electric field. Correspondence between
the loss probability and the electromagnetic local density of states projected
along the electron beam trajectory has been discussed [7, 24, 25].

Equation (1) tells us that the loss probability density measured by EELS
depends on the electron velocity which is directly related to the primary beam
energy in the STEM. However, electron losses are contributed not only by LSPR-
related losses proportional to the out-of-plane near field but also by
material-related losses proportional to the relative thickness of the sample
and relativistic effects like the Čerenkov radiation [26] dependent on the
speed of the electron. To separate the contribution of LSPR losses from other
losses, background subtraction is usually performed. The background is the EEL
spectrum recorded on a pure substrate far away from the plasmonic structure,
so it does not contain any LSPR-related signal. All losses have some angular
distribution which is very narrow at the energies in units of electronvolts
[27]. As the angular distribution of all losses is not exactly the same, the
choice of a proper collection semi-angle may play an important role in the
STEM-EELS experiment. Therefore, the primary beam energy and the collection
semi-angle are important experimental parameters.

Figure 1: Numerical simulation of LSPR in a gold rod (length 240 nm, width 80
nm, height 30 nm) on a 30 nm thick SiNx membrane: EEL spectra (a) for two
positions (5 nm outside the rod at the middle of its shorter and longer edge)
of the 300 keV electron beam followed by surface charge distribution (b) and
loss probability maps (c) for the longitudinal dipole mode at 1.08 eV and the
longitudinal quadrupole mode at 1.72 eV. The scalebar in (b) and (c) is 100 nm
long.

We have studied a series of gold rods to explore the influence of these two
experimental parameters. Plasmonic nanorods represent very simple objects
which have been studied many times by EELS [28, 29, 30]. Our gold rods with
the length of 240 nm, the width of 80 nm, and the height of 30 nm are situated
on a 30 nm thick silicon nitride membrane. First, we have described the
structure theoretically. Numerical simulations were performed with the MNPBEM
toolbox [31] which relies on the boundary element method (BEM) approach [32,
33]. The dielectric function of single-crystal gold was taken from Olmon et
al. [34] and the dielectric function of the silicon nitride membrane was set
to 4. The rod supports two main LSPR modes – the longitudinal dipole mode at
1.08 eV and the longitudinal quadrupole mode at 1.72 eV. Their spectral and
spatial characteristics are introduced in Figure 1.

Finally, to ensure that obtained results are not valid exclusively for rods,
we have included into our study several bow-tie and diabolo antennas, which
have been recently studied theoretically [35] and experimentally [15]. All
structures were of a similar size: wing angle of 90∘ and wing length around
250 nm. We have studied the transverse dipole mode in both types of antennas
(referred to as TD mode) and the longitudinal dipole mode in the bow-tie or
the longitudinal dipole antibonding mode in the diabolo, respectively (both
referred as LD mode in the following for simplicity), as they are well defined
by their spatial distribution and are the same for bow-tie and diabolo
antennas. These antennas support the TD mode (with the maximal loss
probability located at four outer corners of the antenna) around 0.8 eV and
the LD mode (with the maximal loss probability located in the gap/bridge of
the antenna) around 1.2 eV [15].

##  3 Experimental methods

Gold rods with the length of 240 nm, width of 80 nm, and height of 30 nm and
bow-tie and diabolo antennas with the wing angle of 90∘ and wing length of 250
nm were prepared using focused ion beam (FIB) lithography by gallium ions
(using dual beam FEI Helios) [19] of a 30 nm thick gold layer deposited by
magnetron sputtering (using Leica coater EM ACE600) directly on a 30 nm thick
silicon nitride membrane (purchased from Agar Scientific). We did not use any
adhesion layer as it affects and degrades the LSPR [36, 37]. The sample was
plasma cleaned in argon-oxygen plasma for 20 seconds (using plasma cleaner
Fishione 1020) to prevent the carbon contamination during the STEM-EELS
measurement [19].

EELS measurements were performed with TEM FEI Titan using a double focusing
Wien filter monochromator equipped with GIF Quantum spectrometer operated in a
monochromated scanning regime at the primary beam energy of 60 keV, 120 keV,
and 300 keV, the beam energies to which our microscope is tuned. Beam current
was set around 0.2 nA and the full-width at half-maximum of the zero-loss peak
(ZLP) was in range from 0.1 eV to 0.15 eV. We set the convergence semi-angle
to 10 mrad while the collection semi-angle was changed from 1.3 mrad to 20.5
mrad by changing the camera length of the microscope in the STEM mode. The
dispersion of the spectrometer was 0.01 eV/pixel. We recorded spectrum images
with the pixel size of 1 nm or 2 nm. Every pixel consists of 1 EEL spectrum
whose acquisition time was adjusted to use the maximal intensity range of CCD
camera in the spectrometer and avoid its overexposure.

Figure 2: Typical raw low-loss EEL spectrum (blue) decomposed into background
represented by the spectrum of pure SiNx membrane far away from the rod (red)
and the signal corresponding to the LSPR (green). We define the following
quantities: ZLP as the loss probability at 0 eV, spectrum as the loss
probability at the energy of LSPR, signal as the loss probability at the
energy of LSPR corresponding to extracted signal, and background as the loss
probability at the energy of LSPR corresponding to the subtracted background.

EEL spectra were integrated over rectangular areas consisting of tens of
pixels in the places of interest where the respect LSPR is significant.
Finally, the spectrum was divided by the integral intensity of the ZLP (the
energy window for integration was from -1 eV to +1 eV) so the counts were
transformed to a quantity proportional to the loss probability at 0.01 eV
intervals (referred as the loss probability in the following for simplicity).
To separate the raw low-loss EEL spectrum (referred to as the spectrum) to the
contribution of LSPR losses (referred to as the signal) from other losses
(referred to as the background) we have subtracted the EEL spectrum recorded
on a pure membrane far away from the gold rod (Figure 2).

EEL maps were calculated by dividing the map of integrated intensity at the
plasmon peak energy with the energy window of 0.1 eV by the map of the
integral intensity of the ZLP (energy window from -1 eV to +1 eV).

##  4 Results

We introduce several important experimental characteristics based on
definitions in the previous section and in Figure 2. To detect the peak
corresponding to a LSPR as easily as possible one needs to maximize the signal
itself (in principle the signal to noise ratio), to maximize the signal to
background ratio, and to maximize the signal to ZLP ratio as the spectrometer
camera has a limited dynamic range. The influence of the primary beam energy
and the collection semi-angle on these characteristics is presented in the two
following sections for rods. Finally, to ensure that obtained results are not
valid exclusively for rods, we have included to our study several bow-tie and
diabolo antennas, too.

###  4.1 Influence of the primary beam energy on LSPR signal from rods

In the first experiment we take a series of 3 rods and do the STEM-EELS
measurement at the primary beam energy of 300 keV, 120 keV, and 60 keV while
keeping the the collection semi-angle constant at 20.5 mrad.

Figure 3: EEL spectra of the same rod at different beam energies: (a) measured
raw EEL spectra and extracted signal; (b-d) STEM annular dark field (ADF)
images of the rod with marked area for integration of EEL spectra in (a)
recorded during STEM-EELS mapping at 300 keV (b), 120 keV (c), and 60 keV (d);
(e) simulated EEL spectra by BEM. Electron beam was situated 5 nm outside the
rod at the middle of its shorter edge and calculated loss probability density
was recalculated to loss probability at 0.01 eV energy intervals
(corresponding to the dispersion of the spectrometer in the experiment).

Figure 3(a) shows EEL spectra of one rod integrated over the same selected
area recorded at different beam energies. The areas of integration are marked
by colored squares in STEM annular dark field (ADF) micrographs of the rod in
Figure 3(b-d). We clearly see that the signal corresponding to the LSPR is the
strongest for the 60 keV electron beam and the weakest for the 300 keV
electron beam. This experimental finding is in a perfect agreement with the
theory represented by BEM simulation of EEL spectra shown in Figure 3(e).
Moreover, if we consider the second peak in experimental EEL spectra in Figure
3(a) at 1.76 eV corresponding to the longitudinal quadrupole mode, we see that
this mode is, in principle, not detected when using 300 keV electron beam, but
is clearly visible when using 60 keV or 120 keV electron beam. Consequently,
lower beam energies are better for observation of weaker plasmon modes. Based
on this observation, one can say that the lower the energy of primary electron
beam, the better.

If we consider measured raw EEL spectra [dashed lines in Figure 3(a)], we
clearly see that the peak at 1.08 eV corresponding to the longitudinal dipole
mode is the most noticeable when using 120 keV electron beam. In the case of
300 keV electron beam the background is enhanced by relativistic effects like
the Čerenkov radiation as the speed of the 300 keV electron is higher than the
speed of the light in the silicon nitride membrane with the refractive index
around 2 [38]. Therefore, the Čerenkov limit for silicon nitride is around 80
keV and the probability of Čerenkov photon excitation per unit path length of
the electron inside the silicon nitride specimen equals to 0.3 when the beam
energy is around 120 keV [26]. Considering this, the primary beam energy
should not exceed the theoretical Čerenkov limit for the membrane too much and
the probability of Čerenkov photon excitation per unit path length should be
below 0.4, which was introduced as the experimental limit for daily use [39].
On the other hand, the raw EEL spectra measured with a 60 keV electron beam
has the highest background in the lower energy loss region. This is caused by
a higher probability of scattering events as the inelastic mean free path
(IMFP) of electrons in the sample is smaller [27] or, in other words, the
relative thickness of the sample is larger. Consequently, the optimal primary
beam energy should not be too low to keep the relative thickness of the
supporting membrane as low as possible, which can be achieved either by
increasing the beam energy until the relativistic effects become too strong.

Figure 4: (a) Maps measured by EELS: thickness maps [the thickness of gold is
evaluated in units of inelastic mean free path (IMFP) of electrons in gold] of
the rod and EEL maps of the longitudinal dipole mode at 1.08 eV and the
longitudinal quadrupole mode at 1.76 eV recorded at different primary beam
energies (300 keV, 120 keV, and 60 keV). In the EEL maps the temperature color
scale is used, i.e. yellow corresponds to maximal values and blue corresponds
to minimal values. (b) Line profiles of the EEL maps of the longitudinal
dipole mode along the dashed black lines in (a). The zero position corresponds
to the middle of the rod.

Figure 4(a) shows relative thickness maps (i.e. the thickness of gold is
evaluated in units of inelastic mean free path of electrons in gold) and EEL
maps of the longitudinal dipole mode and the longitudinal quadrupole mode in
the same rod recorded at different primary beam energies. First, we will
inspect the relative thickness maps of the rods. The maximal relative
thickness of the gold in the area of the respect spectrum image reads 0.29 for
300 keV, 0.57 for 120 keV, and 0.77 for 60 keV. The relative thickness is thus
$2.66\times$ ($1.97\times$) larger for 60 keV (120 keV) primary electron beam
energy in comparison to the 300 keV electrons. The increased relative
thickness of the gold plays an important role when the EEL map is recorded. As
the whole map is recorded with the same acquisition time per spectrum, spectra
recorded at positions in the gold will have generally much lower signal in the
energy range of LSPR as more electrons will scatter inelastically.
Consequently, for lower beam energies one can expect more noisy EEL maps of
LSPR.

Now we focus on EEL maps in Figure 4(a). We easily observe two phenomena.
First, the EEL maps recorded at 60 keV are rather noisy. Second, the spatial
distribution of the LSPR seems to be more confined when measuring with lower
beam energies. However, the latter is just an optical illusion caused by the
color scale. Inspecting the line profiles in Figure 4(b) we clearly see that
the spatial distribution of the LSPR is similar for all three primary beam
energies. Consequently, the only difference in the EEL maps is in the signal
to noise ratio. Therefore, the optimal primary beam energy should be high
enough to measure the signal at positions in the metal with a good signal to
noise ratio. This is the easiest solution as it is usually not possible to
decrease the thickness of the metal or to adjust optimal exposure time for
every pixel. The former one would change the studied object. The latter one
would significantly prolonged the measuring time as some auto-exposure
procedure needs to be applied at each pixel of the spectrum image.

Figure 5: Signal (a), spectrum (b), signal to background ratio (c), and signal
to ZLP ratio (d) evaluated from a series of 3 rods for longitudinal dipole
(red) and quadrupole (green) mode at different primary beam energies. (e)
STEM-ADF micrographs of the evaluated series of rods.

Up to now, we have evaluated a single rod. To make a general conclusion we
have to evaluate the important characteristics — the signal, the signal to
background ratio, and the signal to ZLP ratio — quantitatively for a series of
rods. Figure 5 shows these quantities evaluated from a series of 3 rods [shown
in Figure 5(e)] for longitudinal dipole and quadrupole mode at primary beam
energies of 60 keV, 120 keV, and 300 keV. The signal [Figure 5(a)] is the
highest for the primary beam energy of 60 keV and decreases with increasing
the beam energy. The spectrum (i.e., signal plus background) [Figure 5(b)] is
high for the primary beam energies of 60 keV and 300 keV and the lowest for
the primary beam energy of 120 keV. The signal to background ratio [Figure
5(c)] is the highest for the primary beam energy of 120 keV and the lowest for
the primary beam energy of 300 keV. The signal to ZLP ratio [Figure 5(d)] is
rather constant for all three primary beam energies. Based on these
statistics, especially the signal to background ratio, the best results are
obtained using 120 keV electron beam and the worst results are obtained using
300 keV electron beam.

To conclude, in our case the optimal beam energy to measure LSPR in 30 nm
thick gold rods on a 30 nm thick silicon nitride membrane by STEM-EELS is 120
keV. We note that in the case of thicker plasmonic nanostructures using the
beam energy of 300 keV might be advantageous. On the other hand, in the case
of a thinner membrane and thinner plasmonic nanostructures using the beam
energy of 60 keV might be useful.

###  4.2 Influence of the collection semi-angle on LSPR signal from rods

In the second experiment we take a series of 3 rods and do the STEM-EELS
measurement while changing the the collection semi-angle from 1.3 mrad to 20.5
mrad. Note that the convergence semi-angle was constant for all experiments
reading 10 mrad. This experiment was repeated with a different rod series for
every primary beam energy, i.e. 300 keV, 120 keV, and 60 keV. The results are
summarized in Figure 6.

Figure 6: Signal (a,e,i), signal to background ratio (b,f,j), and signal to
ZLP ratio (c,g,k) evaluated from a series of 3 rods shown in STEM-ADF
micrographs (d,h,l) for longitudinal dipole (red) and quadrupole (green) mode
as a function of the collection semi-angle at different beam energies (300
keV, 120 keV, and 60 keV). The convergence semi-angle was set to 10 mrad.

At 300 keV, the signal [Figure 6(a)] is maximal for the largest collection
semi-angle in the case of the longitudinal dipole mode and rather constant
while changing the collection semi-angle for the longitudinal quadrupole mode.
The signal to background ratio [Figure 6(b)] and the signal to ZLP ratio
[Figure 6(c)] reach higher values for either the largest collection semi-
angle, or for the smallest collection semi-angle in the case of the
longitudinal dipole mode. However, in the case of the longitudinal quadrupole
mode, the highest values are reached for the smallest collection semi-angle
only. Consequently, at the primary beam energy of 300 keV the optimal
collection semi-angle should be preferentially much smaller than the
convergence semi-angle. In our case, we get the best characteristics for the
collection semi-angle of 1.3 mrad. However, we have to notice that the STEM-
EELS measurement with the collection semi-angle of 1.3 mrad is approximately
10times slower as we detect in the spectrometer just a small portion of
electrons.

At 120 keV, the signal [Figure 6(e)] is higher for smaller collection semi-
angles in the case of both modes. The signal to background ratio [Figure 6(f)]
reaches the highest value for the collection semi-angle equal to the
convergence semi-angle in the case of the longitudinal dipole mode and for the
smallest collection semi-angle in the case of the longitudinal quadrupole
mode, respectively. The signal to ZLP ratio [Figure 6(g)] is rather constant
while changing the collection semi-angle for both modes. Consequently, at the
primary beam energy of 120 keV the optimal collection semi-angle should be the
same or smaller than the convergence semi-angle. Note that the characteristics
are rather flat so at 120 keV the collection semi-angle is not a critical
parameter.

At 60 keV, the signal [Figure 6(i)] is rather constant while changing the
collection semi-angle for both modes. The signal to background ratio [Figure
6(j)] reaches the highest value for the collection semi-angle equal to the
convergence semi-angle for both modes. The signal to ZLP ratio [Figure 6(k)]
is rather constant while changing the collection semi-angle for both modes.
Consequently, at the primary beam energy of 60 keV the optimal collection
semi-angle should be equal to the convergence semi-angle.

To conclude, the collection semi-angle is generally not as critical parameter
as the primary beam energy. In the case of a high primary beam energy
(represented by 300 keV), the collection semi-angle should be preferentially
much smaller than the convergence semi-angle, but it is not a critical mistake
to use the collection semi-angle larger than the convergence semi-angle. In
the case of a medium primary beam energy (represented by 120 keV), the
collection semi-angle is not critical at all. Finally, in the case of a low
primary beam energy (represented by 60 keV), the collection semi-angle should
be preferentially the same as the convergence semi-angle.

###  4.3 Bow-tie and diabolo antennas

To ensure that the results are valid generally, we have done the experiments
with gold bow-tie and diabolo antennas, too. First, we have repeated the
primary beam energy experiment on 4 plasmonic antennas. Second, we have
repeated the collection semi-angle experiment at the primary beam energy of
300 keV for one bow-tie antenna. All structures were of a similar size: wing
angle of 90∘ and wing length in range from 230 nm to 250 nm. Such structures
support the TD mode around 0.8 eV and the LD mode around 1.15 eV.

Figure 7: Signal (a), spectrum (b), signal to background ratio (c), and signal
to ZLP ratio (d) evaluated from a series of 4 bow-tie and diabolo antennas for
the TD (blue) and the LD (orange) mode at different primary beam energies. (e)
STEM-ADF micrographs of the evaluated series of 3 bow-ties and a diabolo.

Figure 7 shows the signal, the spectrum, the signal to background ratio, and
the signal to ZLP ratio evaluated from a series of 4 bow-tie and diabolo
antennas [shown in Figure 7(e)] for the TD and the LD mode at primary beam
energies of 60 keV, 120 keV, and 300 keV. The signal [Figure 7(a)] is the
highest for the primary beam energy of 60 keV in the case of TD mode and
rather constant in the case of LD mode. The spectrum (i.e. signal plus
background) [Figure 7(b)] is high for the primary beam energies of 60 keV and
300 keV and the lowest for the primary beam energy of 120 keV. The signal to
background ratio [Figure 7(c)] is the highest for the primary beam energy of
120 keV. The signal to ZLP ratio [Figure 7(d)] is rather constant for all
three primary beam energies. Based on these statistics, especially the signal
to background ratio, the best results are obtained using 120 keV electron
beam.

Figure 8: Signal (a), signal to background ratio (b), and signal to ZLP ratio
(c) evaluated from a bow-tie for the TD (blue) and the LD (orange) mode as a
function of the collection semi-angle at the beam energy of 300 keV. The
convergence semi-angle was set to 10 mrad. Panels (d) and (e) show the STEM
micrographs of the bow-tie recorded as an annular dark field (ADF) image for
EELS collection semi-angle of 20.5 mrad (d) and as an annular bright field
(ABF) image for EELS collection semi-angle of 1.3 mrad (e) followed by EEL
maps of the TD mode at 0.8 eV and the LD mode at 1.13 eV. In the EEL maps the
temperature color scale is used, i.e. yellow corresponds to maximal values and
blue corresponds to minimal values.

Figure 8 shows the signal, the signal to background ratio, and the signal to
ZLP ratio evaluated from a bow-tie antenna at 300 keV primary beam energy
while changing the collection semi-angle from 1.3 mrad to 20.5 mrad. Note that
the convergence semi-angle was constant for all experiments reading 10 mrad.
The signal [Figure 8(a)] is maximal for the largest collection semi-angle. The
signal to background ratio [Figure 8(b)] and the signal to ZLP ratio [Figure
8(c)] reach the highest values for the smallest collection semi-angle. Note
that relatively high values are reached for the largest collection semi-angle,
too. Moreover, there are no significant differences in the EEL maps recorded
with the collection semi-angle 1.3 mrad [Figure 8(d)] and 20.5 mrad [Figure
8(e)]. Consequently, the optimal collection semi-angle should be
preferentially much smaller than the convergence semi-angle. In our case, we
get the best characteristics for the collection semi-angle of 1.3 mrad.

To conclude, the results of the measurement of bow-tie and diabolo antennas
have a good agreement with the results obtained for the rods presented in
previous sections in Figure 5 and Figure 6(a-c).

##  5 Conclusion

We have experimentally studied the influence of the primary beam energy and
the collection semi-angle on the localized surface plasmon resonances
measurement by STEM-EELS. We have discussed the impact on experimental
characteristics which are important to detect localized surface plasmon peaks
in EELS successfully, namely: the intensity of plasmonic signal, the signal to
background ratio, and the signal to zero-loss peak ratio considering a limited
dynamic range of the spectrometer camera.

The best results in terms of the best signal to background ratio are obtained
using a medium primary beam energy, in our case 120 keV. In the case of too
high primary beam energy, for example 300 keV, the relativistic effects in the
supporting membrane play a non-negligible role and lead to a higher intensity
of the background. On the other hand, using too low primary beam energy, for
example 60 keV, leads to a larger relative thickness of the sample so the
scattering probability is higher and this results into a higher intensity of
the background, too.

The collection semi-angle is generally not as critical parameter as the
primary beam energy. Using the primary beam energy of 300 keV, the collection
semi-angle should be preferentially much smaller than the convergence semi-
angle, but it is not a critical mistake to use the collection semi-angle
larger than the convergence semi-angle. In the case of a medium primary beam
energy (represented by 120 keV), the collection semi-angle is not a critical
parameter. Using a low primary beam energy (represented by 60 keV), the
collection semi-angle should be preferentially the same as the convergence
semi-angle.

## Acknowledgements

This work has been supported by the Ministry of Education, Youth and Sports of
the Czech Republic under the projects CzechNanoLab [project No. LM2018110,
2020-2022] supporting the CEITEC Nano Research Infrastructure and CEITEC 2020
[project No. LQ1601] and Brno University of Technology [project No.
FSI-S-17-4482]. M. H. acknowledges the support of Thermo Fisher Scientific and
CSMS scholarship 2019.

## References

  * [1] L. Novotny, N. van Hulst, Antennas for light, Nat. Photonics 5 (2011) 83–90.  doi:10.1038/nphoton.2010.237.
  * [2] J. A. Schuller, E. S. Barnard, W. Cai, Y. C. Jun, J. S. White, M. L. Brongersma, Plasmonics for extreme light concentration and manipulation, Nat. Mater. 9 (2010) 193–204.  doi:10.1038/nmat2630.
  * [3] K. L. Kelly, E. Coronado, L. L. Zhao, G. C. Schatz, The optical properties of metal nanoparticles: The influence of size, shape, and dielectric environment, J. Phys. Chem. B 107 (2003) 668–677.  doi:10.1021/jp026731y.
  * [4] F. J. García de Abajo, Optical excitations in electron microscopy, Rev. Mod. Phys. 82 (2010) 209–275.  doi:10.1103/RevModPhys.82.209.
  * [5] A. Losquin, T. T. A. Lummen, Electron microscopy methods for space-, energy-, and time-resolved plasmonics, Front. Phys. 12 (2016) 127301.  doi:10.1007/s11467-016-0605-2.
  * [6] J. Nelayah, M. Kociak, O. Stéphan, F. J. García de Abajo, M. Tencé, L. Henrard, D. Taverna, I. Pastoriza-Santos, L. M. Liz-Marzán, C. Colliex, Mapping surface plasmons on a single metallic nanoparticle, Nat. Physics 3 (2007) 348–353.  doi:10.1038/nphys575.
  * [7] M. Kociak, O. Stéphan, Mapping plasmons at the nanometer scale in an electron microscope, Chem. Soc. Rev. 43 (2014) 3865–3883.  doi:10.1039/C3CS60478K.
  * [8] C. Colliex, M. Kociak, O. Stéphan, Electron Energy Loss Spectroscopy imaging of surface plasmons at the nanometer scale, Ultramicroscopy 162 (2016) A1–A24.  doi:10.1016/j.ultramic.2015.11.012.
  * [9] Y. Wu, G. Li, J. P. Camden, Probing nanoparticle plasmons with electron energy loss spectroscopy, Chem. Rev. 118 (2018) 2994–3031.  doi:10.1021/acs.chemrev.7b00354.
  * [10] L. H. G. Tizei, H. Lourenço-Martins, P. Das, S. Y. Woo, L. Scarabelli, C. Hanske, L. M. Liz-Marzán, K. Watanabe, T. Taniguchi, M. Kociak, Monolayer and thin h–bn as substrates for electron spectro-microscopy analysis of plasmonic nanoparticles, Appl. Phys. Lett. 113 (2018) 231108.  doi:10.1063/1.5054751.
  * [11] B. Schaffer, W. Grogger, G. Kothleitner, F. Hofer, Comparison of EFTEM and STEM EELS plasmon imaging of gold nanoparticles in a monochromated TEM, Ultramicroscopy 110 (2010) 1087–1093.  doi:10.1016/j.ultramic.2009.12.012.
  * [12] O. Bitton, S. N. Gupta, L. Houben, M. Kvapil, V. Křápek, T. Šikola, G. Haran, Vacuum Rabi splitting of a dark plasmonic cavity mode revealed by fast electrons, Nat. Commun. 11 (2020) 487.  doi:10.1038/s41467-020-14364-3.
  * [13] S. Raza, N. Stenger, S. Kadkhodazadeh, S. V. Fischer, N. Kostesha, A.-P. Jauho, A. Burrows, M. Wubs, N. A. Mortensen, Blueshift of the surface plasmon resonance in silver nanoparticles studied with EELS, Nanophotonics 2 (2013) 131\.  doi:10.1515/nanoph-2012-0032.
  * [14] X. Zhou, A. Hörl, A. Trügler, U. Hohenester, T. B. Norris, A. A. Herzing, Effect of multipole excitations in electron energy-loss spectroscopy of surface plasmon modes in silver nanowires, J. Appl. Phys. 116 (2014) 223101\.  doi:10.1063/1.4903535.
  * [15] V. Křápek, A. Konečná, M. Horák, F. Ligmajer, M. Stöger-Pollach, M. Hrtoň, J. Babocký, T. Šikola, Independent engineering of individual plasmon modes in plasmonic dimers with conductive and capacitive coupling, Nanophotonics (2019).  doi:10.1515/nanoph-2019-0326.
  * [16] A. L. Koh, K. Bao, I. Khan, W. E. Smith, G. Kothleitner, P. Nordlander, S. A. Maier, D. W. McComb, Electron energy-loss spectroscopy (EELS) of surface plasmons in single silver nanoparticles and dimers: Influence of beam damage and mapping of dark modes, ACS Nano 3 (2009) 3015–3022.  doi:10.1021/nn900922z.
  * [17] A. L. Koh, A. I. Fernández-Domínguez, D. W. McComb, S. A. Maier, J. K. W. Yang, High-resolution mapping of electron-beam-excited plasmon modes in lithographically defined gold nanostructures, Nano Lett. 11 (2011) 1323–1330.  doi:10.1021/nl104410t.
  * [18] G. Haberfehlner, A. Trügler, F. P. Schmidt, A. Hörl, F. Hofer, U. Hohenester, G. Kothleitner, Correlated 3D nanoscale mapping and simulation of coupled plasmonic nanoparticles, Nano Lett. 15 (2015) 7726–7730.  doi:10.1021/acs.nanolett.5b03780.
  * [19] M. Horák, K. Bukvišová, V. Švarc, J. Jaskowiec, V. Křápek, T. Šikola, Comparative study of plasmonic antennas fabricated by electron beam and focused ion beam lithography, Sci. Rep. 8 (2018) 9640.  doi:10.1038/s41598-018-28037-1.
  * [20] M. Horák, V. Křápek, M. Hrtoň, A. Konečná, F. Ligmajer, M. Stöger-Pollach, T. Šamořil, A. Paták, Z. Édes, O. Metelka, J. Babocký, T. Šikola, Limits of Babinet’s principle for solid and hollow plasmonic antennas, Sci. Rep. 9 (2019) 4004.  doi:10.1038/s41598-019-40500-1.
  * [21] V. Křápek, A. L. Koh, L. Břínek, M. Hrtoň, O. Tomanec, R. Kalousek, S. A. Maier, T. Šikola, Spatially resolved electron energy loss spectroscopy of crescent-shaped plasmonic antennas, Opt. Express 23 (2015) 11855–11867.  doi:10.1364/OE.23.011855.
  * [22] C. Cherqui, Y. Wu, G. Li, S. C. Quillin, J. A. Busche, N. Thakkar, C. A. West, N. P. Montoni, P. D. Rack, J. P. Camden, D. J. Masiello, STEM/EELS imaging of magnetic hybridization in symmetric and symmetry-broken plasmon oligomer dimers and all-magnetic fano interference, Nano Lett. 16 (2016) 6668–6676.  doi:10.1021/acs.nanolett.6b03504.
  * [23] F. Ligmajer, M. Horák, T. Šikola, M. Fojta, A. Daňhel, Silver amalgam nanoparticles and microparticles: A novel plasmonic platform for spectroelectrochemistry, J. Phys.l Chem. C 123 (2019) 16957–16964.  doi:10.1021/acs.jpcc.9b04124.
  * [24] F. J. García de Abajo, M. Kociak, Probing the photonic local density of states with electron energy loss spectroscopy, Phys. Rev. Lett. 100 (2008) 106804\.  doi:10.1103/PhysRevLett.100.106804.
  * [25] U. Hohenester, H. Ditlbacher, J. R. Krenn, Electron-energy-loss spectra of plasmonic nanoparticles, Phys. Rev. Lett. 103 (2009) 106801.  doi:10.1103/PhysRevLett.103.106801.
  * [26] M. Horák, M. Stöger-Pollach, The Čerenkov limit of Si, GaAs and GaP in electron energy loss spectrometry, Ultramicroscopy 157 (2015) 73 – 78\.  doi:10.1016/j.ultramic.2015.06.005.
  * [27] R. Egerton, Electron Energy-Loss Spectroscopy in the Electron Microscope, Springer, Boston, 2011.  doi:10.1007/978-1-4419-9583-4.
  * [28] G. W. Bryant, F. J. García de Abajo, J. Aizpurua, Mapping the plasmon resonances of metallic nanoantennas, Nano Lett. 8 (2008) 631–636.  doi:10.1021/nl073042v.
  * [29] X. Zhou, T. B. Norris, A. Hörl, A. Trügler, U. Hohenester, A. A. Herzing, Effect of multipole excitations in electron energy-loss spectroscopy of surface plasmon modes in silver nanowires, J. Appl. Phys. 116 (2014) 223101\.  doi:10.1063/1.4903535.
  * [30] M. Bosman, L. Zhang, H. Duan, S. F. Tan, C. A. Nijhuis, C. Qiu, J. K. W. Yang, Encapsulated annealing: Enhancing the plasmon quality factor in lithographically–defined nanostructures, Sci. Rep. 4 (2014) 5537.  doi:10.1038/srep05537.
  * [31] J. Waxenegger, A. Trügler, U. Hohenester, Plasmonics simulations with the MNPBEM toolbox: Consideration of substrates and layer structures, Comput. Phys. Commun 193 (2015) 138–150.  doi:10.1016/j.cpc.2015.03.023.
  * [32] F. J. García de Abajo, A. Howie, Relativistic electron energy loss and electron-induced photon emission in inhomogeneous dielectrics, Phys. Rev. Lett. 80 (1998) 5180–5183.  doi:10.1103/PhysRevLett.80.5180.
  * [33] F. J. García de Abajo, A. Howie, Retarded field calculation of electron energy loss in inhomogeneous dielectrics, Phys. Rev. B 65 (2002) 115418.  doi:10.1103/PhysRevB.65.115418.
  * [34] R. L. Olmon, B. Slovick, T. W. Johnson, D. Shelton, S.-H. Oh, G. D. Boreman, M. B. Raschke, Optical dielectric function of gold, Phys. Rev. B 86 (2012) 235147\.  doi:10.1103/PhysRevB.86.235147.
  * [35] M. Hrtoň, A. Konečná, M. Horák, T. Šikola, V. Křápek, Plasmonic antennas with electric, magnetic, and electromagnetic hot spots based on Babinet’s principle (2019).  arXiv:1904.01397.
  * [36] T. G. Habteyes, S. Dhuey, E. Wood, D. Gargas, S. Cabrini, P. J. Schuck, A. P. Alivisatos, S. R. Leone, Metallic adhesion layer induced plasmon damping and molecular linker as a nondamping alternative, ACS Nano 6 (2012) 5702–5709.  doi:10.1021/nn301885u.
  * [37] S. J. Madsen, M. Esfandyarpour, M. L. Brongersma, R. Sinclair, Observing plasmon damping due to adhesion layers in gold nanostructures using electron energy loss spectroscopy, ACS Photonics 4 (2017) 268–274.  doi:10.1021/acsphotonics.6b00525.
  * [38] H. R. Philipp, Optical properties of silicon nitride, J. Electrochem. Soc. 120 (1973) 295–300.  doi:10.1149/1.2403440.
  * [39] M. Stöger-Pollach, Optical properties and bandgaps from low loss EELS: Pitfalls and solutions, Micron 39 (2008) 1092–1110.  doi:10.1016/j.micron.2008.01.023.