# Using multiple ASR hypotheses to boost i18n NLU performance

Charith Peris Gokmen Oz Khadige Abboud Venkata sai Varada Prashan Wanigasekara
Haidar Khan

Alexa AI, Amazon
Cambridge MA
{perisc, ogokmen, abboudk, vnk, wprasha<EMAIL_ADDRESS>

###### Abstract

Current voice assistants typically use the best hypothesis yielded by their
Automatic Speech Recognition (ASR) module as input to their Natural Language
Understanding (NLU) module, thereby losing helpful information that might be
stored in lower-ranked ASR hypotheses. We explore the change in performance of
NLU associated tasks when utilizing five-best ASR hypotheses when compared to
status quo for two language datasets, German and Portuguese. To harvest
information from the ASR five-best, we leverage extractive summarization and
joint extractive-abstractive summarization models for Domain Classification
(DC) experiments while using a sequence-to-sequence model with a pointer
generator network for Intent Classification (IC) and Named Entity Recognition
(NER) multi-task experiments. For the DC full test set, we observe significant
improvements of up to 7.2% and 15.5% in micro-averaged F1 scores, for German
and Portuguese, respectively. In cases where the best ASR hypothesis was not
an exact match to the transcribed utterance (mismatched test set), we see
improvements of up to 6.7% and 8.8% micro-averaged F1 scores, for German and
Portuguese, respectively. For IC and NER multi-task experiments, when
evaluating on the mismatched test set, we see improvements across all domains
in German and in 17 out of 19 domains in Portuguese (improvements based on
change in SeMER scores). Our results suggest that the use of multiple ASR
hypotheses, as opposed to one, can lead to significant performance
improvements in the DC task for these non-English datasets. In addition, it
could lead to significant improvement in the performance of IC and NER tasks
in cases where the ASR model makes mistakes.

##  1 Introduction

Recent years have seen a dramatic increase in the adoption of intelligent
voice assistants such as Amazon Alexa, Apple Siri and Google Assistant. As use
cases expand, these assistants are expected to process ever more complex user
utterances and perform many different tasks. Some of the key components that
enable the performance of these tasks are housed within the spoken language
understanding (SLU) system; one being the Automatic Speech Recognition (ASR)
module which transcribes the users’ vocal sound wave into text and another
being the Natural Language Understanding module which performs a variety of
downstream tasks that help identify the actions requested by the user (Ram et
al., 2018; Gao et al., 2018). These modules perform in tandem and are crucial
for the successful processing of user utterances. Typical ASR models generate
multiple hypotheses for an input audio signal, that are ranked by their
confidence scores (Li et al., 2020). However, only the top ranked hypothesis
(referred to hereafter as the ASR 1-best) is usually processed by the NLU
module for downstream tasks (Li et al., 2020).

Three major tasks performed by the NLU module are Domain Classification (DC),
Intent Classification (IC) and Named Entity Recognition (NER). DC predicts the
domain relevant to the utterance (Weather, Shopping, Music etc.) and IC
extracts actions requested by users (some examples are, buy an item, play a
song or set a reminder). NER is focused on identifying and extracting entities
from user requests (names, dates, locations, etc.). Current NLU models usually
take in the ASR 1-best hypothesis as input to perform NLU recognition (Li et
al., 2020). However, the highest-scored ASR hypothesis is not always correct
and, at times, can lead to downstream failures including incorrect NLU
hypotheses. These errors can be mitigated by utilizing multiple top-ranked ASR
hypotheses (ASR n-best hypotheses) in NLU modeling, which have a higher
likelihood of containing the correct hypothesis. Even in the case of all
n-best hypotheses being incorrect, the NLU models may be capable of recovering
the correct hypothesis by integrating the information contained within the
n-best hypotheses. Hence, the use of multiple hypotheses should help obtain
firmer predictions from ASR modules for their corresponding NLU module and
result in improved performance.

In this study we focus on two non-English internal datasets, German and
Portuguese, and evaluate the use of ASR n-best hypotheses for improving NLU
modeling within these contexts. Given that the ASR models we use in this
experiment produce a maximum of five (or less) hypotheses per input utterance,
we utilize all available hypotheses (referred to hereafter as the ASR 5-best)
for our work. We leverage two BERT-based summarization models (Devlin et al.,
2019; Liu, 2019; Liu and Lapata, 2019) and a sequence-to-sequence model with a
pointer generator network (Rongali et al., 2020) to extract the information
from the ASR 5-best hypotheses. We show that using multiple hypotheses, as
opposed to just one, can significantly improve the overall performance of DC,
and the performance of IC and NER in cases where the ASR model makes mistakes.
We describe relevant work in Section 2 and present a description of our data
set and opportunity cost analysis in Section 3. In Section 4 we describe the
architecture of our models. In Section 5, we present our experimental results
followed by our conclusions in Section 6.

##  2 Related work

Using deep learning models for summarization has been an active area of
research in the recent past. Two popular types in current literature have been
extractive summarization and abstractive summarization. Extractive
summarization systems summarize by identifying and concatenating the most
important sentences in a document whereas abstractive summarization systems
conceptualize the task as a sequence-to-sequence problem and generate the
summary by paraphrasing sections of the source document. Extensive work has
been done on extractive summarization (Liu, 2019; Cheng and Lapata, 2016;
Nallapati et al., 2016a; Narayan et al., 2018b; Dong et al., 2018; Zhang et
al., 2018; Zhou et al., 2018) and abstractive summarization (Narayan et al.,
2018a; See et al., 2017; Rush et al., 2015; Nallapati et al., 2016b) used in
isolation. Furthermore, studies have shown improvement in summary quality when
extractive and abstractive objectives have been used in combination (Liu and
Lapata, 2019; Gehrmann et al., 2018; Li et al., 2018).

Liu (2019) proposed a simple, yet powerful, variant of BERT for extractive
summarization in which they modified the input sequence of BERT from its
original two sentences to multiple sentences. They used multiple
classification tokens ([CLS]) combined with interval segment embeddings to
distinguish multiple sentences within a document. They appended several
summarization specific layers (either a simple classifier, a transformer or an
LSTM) on top of the BERT outputs to capture document level features relevant
for extracting summaries. Following this work, Liu and Lapata (2019) proposed
a model that comprises of the pre-trained BERT extractive summarization model
(Liu, 2019) as the encoder and a decoder which consists of a 6-layered
transformer (Vaswani et al., 2017). The encoder was fine-tuned in two stages,
first on the extractive summarization task and then again on an abstractive
summarization task resulting in a joint extractive-abstractive model that
showed improved performance on summarization tasks.

The utilization of multiple ASR hypotheses for improved NLU model performance
across DC, IC tasks was first introduced by Li et al. (2020). They proposed
the use of 5-best ASR hypotheses to train a BiLSTM language model, instead of
using a single 1-best hypothesis selected using either majority vote, highest
confidence score or a reranker. They explored two methods to integrate the
n-best hypothesis: a basic concatenation of hypotheses text and a hypothesis
embedding concatenation using max/avg pooling. The results show 14%-25%
relative gains in both DC and IC accuracy.

In our work, we explore the performance improvement offered by utilizing the
ASR 5-best hypotheses in previously unexplored languages, German and
Portuguese. We also differ from previous studies due to our use of the
superior BERT-based extractive (Liu, 2019) and joint extractive-abstractive
(Liu and Lapata, 2019) summarization models to extract a summary hypothesis
for the DC task, from the ASR 5-best.

Voice assistants traditionally handle IC and NER tasks using semantic parsing
components which typically comprise of statistical slot-filling systems for
simple queries and, in more recent time, shift-reduce parsers (Gupta et al.,
2018; Einolghozati et al., 2019) for more complex utterances. Rongali et al.
(2020) proposed a unified architecture based on sequence-to-sequence models
and pointer generator networks to handle both simple and complex IC and NER
tasks with which they achieve state-of-the-art results. In this work, we use a
model that expands this approach to consume the 5-best ASR hypotheses and
evaluate its performance on IC/NER tasks for the two language datasets
considered.

##  3 Data

Our experiments focus on two non-English internal datasets; German and
Portuguese. We run all utterances in each language through one language-
specific ASR model and take the top-ranked ASR hypothesis for each utterance
as ASR 1-best and all available hypotheses for each utterance (a maximum of
five in our models) as ASR 5-best. In addition, we also obtain a human
transcribed version of each utterance. For German, we use 1.48 million
utterances from 21 domains for training and validation. We split the data
randomly within each domain, with 85% used for training and 15% for
validation. An independent set of 193K utterances are used for testing. Within
the independent test set we find 17K utterances where the ASR 1-best did not
match the transcribed utterance exactly and mark them as the “mismatched” test
set. (Table 1). For Portuguese, we use 890K utterances from 19 domains for
training and validation, split the same way as with German. Another 247K
utterances are used for testing. We find 41K utterances within test, where the
ASR 1-best did not match the transcribed utterance exactly, and mark them as
the mismatched test set (Table 1).

Table 1: Total data set sizes in terms of utterance counts

Language | Train | Validation | Test (full) | Test (mismatched)
---|---|---|---|---
German | 1,255,402 | 221,543 | 192,697 | 16,672
Portuguese | 756,148 | 133,438 | 246,638 | 40,896

###  3.1 Opportunity Cost Measurement

Li et al. (2020) showed improvement in NLU model performance on English (en-
US) upon utilizing the ASR 5-best hypotheses instead of only ASR 1-best.
However, the impact of this on non-English languages has not yet been
explored. To understand the opportunity of improvement that the ASR 5-best
hypotheses can lend to NLU model performance in German and Portuguese
datasets, we analyze the ASR 5-best hypotheses in comparison to the ground-
truth human transcribed data for each of the considered language datasets.
First, we calculate the number of exact matches to the transcribed utterance
occurring in each of the top 5-best hypotheses. It should be mentioned that
each ASR hypothesis is different from the others and only one hypothesis (if
at all) can match the transcribed utterance. Next we compute the amount of
exact matches found in the nth-best hypothesis set, as a fraction of the
volume of exact matches found at 1-best. The results are shown in Table 2. We
find that the amount of exact matches that occur in 2-5 best hypotheses,
compared to the volume of exact matches that occur in the top-ranked
hypothesis, is large for Portuguese (30.16%) and German (20.83%) (see Table
2). This gives an indication of the opportunity present in using hypotheses
beyond ASR 1-best for each language dataset.

In Table 3, we further illustrate the use of the ASR 5-best hypotheses by
showing three possible cases of stored information that we want our NLU model
to extract; selecting the best matching hypothesis (first and second rows) and
combining hypotheses (third row).

Table 2: Exact Matches to the transcribed utterance found in ASR n-best as a
percentage of Exact Matches found in ASR 1-best

n | Portuguese (%) | German (%)
---|---|---
2 | 16.55 | 10.26
3 | 7.1 | 5.01
4 | 3.92 | 3.33
5 | 2.59 | 2.23
total | 30.16 | 20.83

Table 3: Illustrative examples in English that compares the 3-best ASR
hypotheses to the transcribed utterance

Transcription | 1- best hypothesis | 2-best hypothesis | 3-best hypothesis
---|---|---|---
buy movie mystery | buy movie mystery |  buy my tree | but move my tree
who is nelson | how is my son | who is nelson | how samsung
play music | pull music | pull news |  play my muse

##  4 Experimental Setup

###  4.1 DC models

For our DC experiments, we compare performance across the following
classification models:

  * •

Baseline – A BERT-based classification baseline model with MLP classifier
trained on the transcribed utterance and tested on the ASR 1-best

  * •

BSUMEXT– A BERT-based extractive summarization model trained and tested on the
ASR 5-best

  * •

BSUMEXTABS– A BERT-based joint extractive and abstractive summarization model
trained and tested on the ASR 5-best

Standard testing on transcribed utterances underestimates the combined ASR and
NLU errors. In order to avoid this our test sets exclude transcribed
utterances and thus reflect the real situation.

In Section 3, we described the simple extractive summarization model proposed
by Liu (2019). We adapt their extractive summarization model to take the ASR
5-best hypotheses as input and output a probability score per domain based on
a summarized hypothesis. Figure 1 shows the architecture of BSUMEXT with ASR
5-best input. The task of the BSUMEXT model is to create an extractive summary
by picking from the class assigned to each hypothesis. This summary is then
fed into a multi-layer perceptron classifier to perform the DC task. As in the
case of Liu (2019), vanilla BERT is modified to include multiple [CLS]
symbols. Each symbol is used to obtained features of each of the ASR n-best
hypotheses preceding it. Alternating hypotheses fed into the model are
assigned a segment embedding (E_A or E_B), based on whether it is an even or
odd numbered hypothesis. For example for a sentence “play music” :

⬇

1ASR 1-best: play muse [E_A]

2ASR 2-best: play mu chick [E_B]

3ASR 3-best: play news [E_A]

4ASR 4-best: play mus [E_B]

5ASR 5-best: play my sick [E_A]

The model then takes the [CLS] representation of each ASR 5-best utterance and
performs multi-headed attention to obtain the summary hypothesis.

For the BSUMEXTABS model, the BERT encoder is fine-tuned on an abstractive
summarization task and then further fine-tuned on the extractive summarization
task. In this model the summary hypothesis fed into the multi-layer perceptron
classifier, is generated token by token in a sequence-to-sequence fashion.
Similar to Liu and Lapata (2019), a decoupled fine-tuning schedule which
separates the optimizers of the encoder and the decoder is used.

We trained each of our models for up to 30 epochs and use the best performing
model, based on validation metrics, for evaluating the independent test set.

Figure 1: A schematic of the architecture of the BSUMEXT

###  4.2 IC/NER models

We compare the following models for the IC and NER tasks:

  * •

Baseline – A BERT-based classification baseline model trained on the
transcribed utterance and tested on the ASR 1-best

  * •

BERT_S2S_NBEST_PTR – A BERT-based sequence-to-sequence model which employs a
pointer generator network, trained on the ASR 5-best + transcribed utterance
and tested on ASR 5-best

Instead of a typical sequence tagging problem, Rongali et al. (2020) propose a
unified architecture to handle IC and NER tasks as a sequence generation
problem. We build upon that approach. BERT_S2S_NBEST_PTR is a sequence-to-
sequence model augmented with a pointer generator network which functions as a
self-attention mechanism. We expand the architecture proposed by Rongali et
al. (2020) to include multiple input queries. The model task is to generate
target words which can be either intent or slot delimiters or words that are
from the source sequences. The pointer generator network enables the model to
generate pointers to the source sequences (instead of using a large vocabulary
of tokens) within the target sequence. An example of a source sequence with
two ASR hypotheses and a target sequence looks as follows (we use spaces to
delimit hypotheses and _&_ to delimit separate tokens within an utterance):

⬇

1Source: ply_&_madonna play_&_mad_&_owner

2Target: PlaySongIntent( @ptr1_0 ArtistName( @ptr0_1 )ArtistName
)PlaySongIntent

where @ptr0_1, for example, is a pointer to the second word “madonna” in the
first utterance of the source query. One advantage of using pointers instead
of the actual tokens is the smaller target vocabulary required for the
decoder, resulting in a more light-weight model.

The architecture consists of a pre-trained BERT encoder and a transformer
decoder (Devlin et al., 2019; Vaswani et al., 2017). The decoder is augmented
with a pointer generator network that functions as a self-attention mechanism.
Figure 2 shows the high-level architecture. The Bert encoder processes each
ASR hypothesis separately. The encoder hidden states over all ASR hypotheses
are then concatenated and passed to the decoder. The decoder hidden states are
used to update the attention mechanism and the tagging vocabulary and pointer
distributions (see Rongali et al. (2020) for detailed descriptions). These
probability distributions of tags and pointers are used to determine the next
word and tag that is output by the decoder. The model is trained by minimizing
sequence cross entropy loss over the training set.

These models are domain-specific multi-task models which handle both IC and
NER tasks simultaneously. We trained one model per domain with all models
trained for up to 50 epochs. The best performing model based on validation
metrics was used for evaluating the independent test set.

Figure 2: A schematic of the sequence-to-sequence model with attention. Each
ASR hypothesis is encoded separately. The encoder hidden states are then
concatenated and passed to the decoder to have a cross-attention between
encoder and decoder outputs over all ASR hypotheses.

##  5 Results and Discussion

###  5.1 Evaluation

We measure the success of our DC experiments by comparing both micro- and
macro-averaged F1 scores of our experimental models to those of the baseline
model. Micro- and macro-averaged F1 scores are defined as

$F1_{micro}=\frac{2\times P\times R}{P+R}$

(1)

$F1_{macro}=\frac{1}{n}\sum_{i}F1_{i}=\frac{1}{n}\sum_{i}\frac{2\times
P_{i}\times R_{i}}{P_{i}+R_{i}}$

(2)

where $P$ and $R$ are overall precision and recall respectively and $P_{i}$
and $R_{i}$ are the within class precisions and recalls respectively. We also
calculate the relative change in error of each experimental model run with
respect to baseline as shown in equation 3. Note that “lower-is-better” for
this metric. In addition to these metrics calculated on the full test data
set, we also calculate these metrics on the mismatched test set utterances
where the ASR 1-best did not match the transcribed utterance.

$\Delta_{err}=100\times\frac{((100-F1_{experiment})-(100-F1_{baseline}))}{(100-F1_{baseline}))}$

(3)

For the IC and NER experiments, we use Semantic Error Rate (SemER) (Su et al.,
2018) as our metric of choice. SemER is defined as follows:

$SemER=\frac{D+I+S}{C+D+S}$

(4)

where D=deletion, I=insertion, S=substitution and C=correct-slots. The Intent
is treated as a slot in this metric and Intent error, considered as a
substitution. We use the relative change in SemER with respect to the baseline
model (equation 5), both overall and per domain in order to evaluate the
success of our models. Note that “lower-is-better” for relative change in
SemER as well.

$\Delta_{sem}=100\times\frac{(SemER_{experiment}-SemER_{baseline})}{SemER_{baseline}}$

(5)

###  5.2 DC experiments

Table 4: Evaluation on the full and mismatched test sets for DC. Relative
change in error rate ($\Delta_{err}$) measured against baseline for each
metric is shown in each succeeding column (negative is good).

| Full set | Mismatched set
---|---|---
Model |  | f1_micro
---
($\Delta_{err}$)
| f1_macro
---
($\Delta_{err}$)
| f1_micro
---
($\Delta_{err}$)
| f1_macro
---
($\Delta_{err}$)
German
BSUMEXT | -1.60% | -4% | -5.40% | -12%
BSUMEXTABS | -7.20% | -3.90% | -6.70% | -2.30%
Portuguese
BSUMEXT | -12.60% | 4.90% | -6.30% | -0.30%
BSUMEXTABS | -15.50% | -7.30% | -8.80% | -7.40%

Table 4 describes the performance of all the models defined in Section 4.1 on
the full test set and the mismatched test set (see Section 3 and Table 1). The
full test set enables us to understand the general performance improvement
that can be achieved by using summarization models. Although utilizing the
full ASR 5-best hypotheses might offer some improvement even in cases where
the ASR 1-best hypothesis is an exact match to the transcribed utterance, much
more value-add is expected when using the ASR 5-best hypotheses in cases where
there is a mismatch between the transcribed utterance and ASR 1-best. To study
this use case, we use the mismatched test set.

We observed that a majority of F1 scores across all models for German exceeded
their corresponding values in Portuguese. Our opportunity cost analysis showed
that exact matches between the transcribed utterance and ASR 2-5-best for
Portuguese are higher than for German (see Section 3.1). This suggests that
the German ASR model tends to perform better than the Portuguese ASR model. In
this light, the smaller gains in relative change in error observed for German
when compared to Portuguese are likely due to the German ASR model being
superior and therefore leaving smaller room for improvement.

Figure 3 displays the relative changes of each model against the baseline for
each dataset. When considering micro-averaged F1 scores, the BSUMEXT and
BSUMEXTABS models out-perform the baseline in all cases, with the later out-
performing the former. This shows that the use of ASR 5-best hypotheses can
significantly improve overall classification for both language datasets. The
BSUMEXTABS models also consistently out-perform the baseline on macro-averaged
F1 scores, showing improvement in mean within-class classification scores as
well. This suggests that BSUMEXTABS with additional fine-tuning on the
abstractive task, is in general more successful at creating a firmer
hypothesis for DC than the pure extractive summarization of BSUMEXT. For
Portuguese, even with the relatively large percentage of exact matches
available for extraction within its ASR 2-5 hypotheses (see Section 3),
BSUMEXTABS consistently outperforms BSUMEXT across all metrics and datasets.

Figure 3: Relative change in error rate measured against baseline for each
metric on full and unmatched test sets for DC experiments. Table 5: Joint
evaluation on full and mismatched test sets for IC/NER tasks. $\Delta_{sem}$
(%) is the relative change in SemER against baseline for each domain (negative
is good).

German
---
| S2S_NBEST_PTR
Domain | Full Set $\Delta_{sem}$ (%) | Mismatched Set $\Delta_{sem}$ (%)
domain A | 14.79 | -14.16
domain B | 30.33 | -10.27
domain C | 25.25 | -5.83
domain D | 95.41 | -7.3
domain E | 16.38 | -12.68
domain F | 12.54 | -18.9
domain G | -33.51 | -23.2
domain H | 7.41 | -14.2
domain I | 12.96 | -25.2
domain J | 15.27 | -3.72
domain K | 32.02 | -7.42
domain L | 89.45 | -18.63
domain M | 643.85 | -15.95
domain N | 1.06 | -7.21
domain O | -34.8 | -25.02
domain P | 26.52 | -8.74
domain Q | 8.47 | -6.13
domain R | 69.35 | -13.76
domain S | 19.07 | -2.12
domain T | -4.25 | -10.93
domain U | 1.92 | -7.33
Overall | 19.17 | -11.64
Portuguese
| S2S_NBEST_PTR
Domain | Full Set $\Delta_{sem}$ (%) | Mismatched Set $\Delta_{sem}$ (%)
domain A | 2.89 | -14.11
domain B | 18.88 | -7.94
domain C | 46.86 | -14.7
domain D | 4.3 | 3.16
domain E | -12.54 | -30.65
domain F | 5.87 | -18.89
domain G | 6.56 | -3.7
domain H | 24.64 | -2.57
domain I | 71.12 | -24.42
domain J | -7.69 | -10.03
domain K | 19.16 | -5.45
domain L | 11.15 | -9.97
domain M | 3.54 | -10.58
domain N | 48.85 | -10.15
domain O | -30.38 | -59.98
domain P | 6.84 | -12.29
domain Q | 0.11 | -15.66
domain R | 20.49 | -8.94
domain V | 1533.33 | 47.62
Overall | 106.58 | -8.09

###  5.3 IC and NER experiments

Table 5 describes the performance of all the models defined in Section 4.2 on
domain-level data from the full test set and the mismatched test set. As with
the DC experiments, we use the full test set to understand the general overall
performance improvement, and use the mismatched test set to identify
improvement in cases where the ASR 1-best hypothesis is not an exact match to
the transcribed utterance.

When evaluating the BERT_S2S_NBEST_PTR model, we find that it tends improve
performance specifically on the mismatched test set. For German, we find
improved performance across every domain on the mismatched test set (see Table
5) with an overall SemER improvement of 11.6% against baseline. However, we
only observe improvement in three domains on the full set, while other domains
show degradation in SemER. It is also interesting to note that the domains
that improve also had low utterance counts. For Portuguese, testing on the
mismatched test set yields improved performance across 17 out of 19 domains
(see Table 5) with an overall SemER improvement of 8.1% against baseline,
while we see only three domains show improvement on the full test set. Our
results suggest that the ASR 1-best hypothesis works well for IC/NER tasks.
The noise added by additional hypotheses seem to degrade results in the
general use case. However, the additional hypotheses tend to be very helpful
in cases where the ASR model makes mistakes (i.e. mismatched set data where
the ASR 1-best is not an exact match to the transcribed utterance).

Our full test set results show that the baseline model appears to be a better
choice for the IC/NER tasks. However, if we could detect user utterances where
the ASR model might have made a mistake in its top hypothesis, the ASR outputs
(i.e. the set of all hypotheses) of these utterances could be channeled to a
separate NLU model such as BERT_S2S_NBEST_PTR, that could build a better
hypothesis than the baseline and improve overall IC/NER performance.

We analyzed the confidence scores of our ASR models on the full and mismatched
test set hypotheses to explore the possibility of detecting a mismatched set
ASR output. For each ASR output we obtain the mean confidence score across all
available hypotheses. We then compare the frequency distributions of the mean
confidence scores in the full and mismatched test sets. Figure 4 shows the
resulting distributions for two example domains for each language dataset. We
find that the full set shows a strong peak at high confidence scores while the
mismatched set shows a more uniform distribution. The pronounced difference in
distribution shape suggests that a thresholding mechanism based on the
confidence score output by the ASR model (or a simple classifier trained on
ASR outputs and scores) might be used to predict mismatched test set outputs
with good confidence. Leveraging such a mechanism might enable the use of a
second model such as BERT_S2S_NBEST_PTR to improve performance in these
mismatched cases, and in turn improve overall IC/NER performance.

Figure 4: Frequency distributions of mean confidence score across all
available hypotheses for each data point in full and mismatched test sets. We
show results for only two domains for each language due to space limitations.
The distributions show similar shape across all domains within each language
dataset.

##  6 Conclusions and future work

In this study, we explore the benefits of using ASR 5-best hypotheses for the
NLU tasks in the German and Portuguese datasets. We explore several models to
perform DC and IC/NER tasks and evaluate their performance against baseline
models that use ASR 1-best. We find significant overall improvement in
performance for the DC task. We also find significant improvement in
performance of the jointly evaluated IC/NER tasks in cases where the ASR
1-best hypothesis is not an exact match to the transcribed utterance. For the
DC task, our results suggest that the use of ASR 5-best helps produce better
hypotheses and thereby greater improvements in the case of slight lower
quality ASR models.

Our next steps will include exploring how different data splits based on ASR
confidence scores might affect the sequence-to-sequence model performance.
Furthermore, we will explore performance improvements in IC and NER tasks,
using different model architectures and training schedules. We will also
expand our study to a larger set of languages in order to understand how the
use of multiple ASR hypotheses might affect languages with different lexical
distributions. Languages which use multiple scripts (Japanese, Hindi, Arabic
etc.) or which are more opaque and likely to have heterographs (e.g., “serial,
“cereal”) and those that have less standardized spelling systems (Hindi etc)
are more likely to have ASR errors. They may have different levels of
improvement with the use of ASR 5-best hypotheses and we hope to analyze this
in our future work.

## Acknowledgments

We thank Saleh Soltan for extensive discussions on the BERTSUM model
architecture and implementation as well as for creating the BERT embeddings
and making them available for our use. We thank Xinyue Liu for valuable
contributions to discussions on baselines and model optimization. We thank
Mukund Harakere Sridhar for help with pretraining the original encoders,
extensions of which were used in this work. We also thank Karolina Owczarzak,
Chengwei Su and Wael Hamza for helpful discussions and advice on this work.

## References

  * Cheng and Lapata (2016) Jianpeng Cheng and Mirella Lapata. 2016.  Neural summarization by extracting sentences and words.  _CoRR_ , abs/1603.07252.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dong et al. (2018) Yue Dong, Yikang Shen, Eric Crawford, Herke van Hoof, and Jackie Chi Kit Cheung. 2018.  BanditSum: Extractive summarization as a contextual bandit.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3739–3748, Brussels, Belgium. Association for Computational Linguistics.
  * Einolghozati et al. (2019) Arash Einolghozati, Panupong Pasupat, Sonal Gupta, Rushin Shah, Mrinal Mohit, Mike Lewis, and Luke Zettlemoyer. 2019.  Improving semantic parsing for task oriented dialog.  _CoRR_ , abs/1902.06000.
  * Gao et al. (2018) Ge Gao, Eunsol Choi, Yejin Choi, and Luke Zettlemoyer. 2018.  Neural metaphor detection in context.  _CoRR_ , abs/1808.09653.
  * Gehrmann et al. (2018) Sebastian Gehrmann, Yuntian Deng, and Alexander Rush. 2018.  Bottom-up abstractive summarization.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 4098–4109, Brussels, Belgium. Association for Computational Linguistics.
  * Gupta et al. (2018) S. Gupta, Rushin Shah, Mrinal Mohit, A. Kumar, and M. Lewis. 2018.  Semantic parsing for task oriented dialog using hierarchical representations.  In _EMNLP_.
  * Li et al. (2020) Mingda Li, Weitong Ruan, Xinyue Liu, Luca Soldaini, W. Hamza, and Chengwei Su. 2020\.  Improving spoken language understanding by exploiting asr n-best hypotheses.  _ArXiv_ , abs/2001.05284.
  * Li et al. (2018) Wei Li, Xinyan Xiao, Yajuan Lyu, and Yuanzhuo Wang. 2018.  Improving neural abstractive document summarization with explicit information selection modeling.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1787–1796, Brussels, Belgium. Association for Computational Linguistics.
  * Liu (2019) Yang Liu. 2019.  Fine-tune BERT for extractive summarization.  _CoRR_ , abs/1903.10318.
  * Liu and Lapata (2019) Yang Liu and Mirella Lapata. 2019.  Text summarization with pretrained encoders.  _CoRR_ , abs/1908.08345.
  * Nallapati et al. (2016a) Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2016a.  Summarunner: A recurrent neural network based sequence model for extractive summarization of documents.  _CoRR_ , abs/1611.04230.
  * Nallapati et al. (2016b) Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Çağlar GuÌ‡lçehre, and Bing Xiang. 2016b.  Abstractive text summarization using sequence-to-sequence RNNs and beyond.  In _Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning_ , pages 280–290, Berlin, Germany. Association for Computational Linguistics.
  * Narayan et al. (2018a) Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018a.  Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1797–1807, Brussels, Belgium. Association for Computational Linguistics.
  * Narayan et al. (2018b) Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018b.  Ranking sentences for extractive summarization with reinforcement learning.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1747–1759, New Orleans, Louisiana. Association for Computational Linguistics.
  * Ram et al. (2018) Ashwin Ram, Rohit Prasad, Chandra Khatri, Anu Venkatesh, Raefer Gabriel, Qing Liu, Jeff Nunn, Behnam Hedayatnia, Ming Cheng, Ashish Nagar, Eric King, Kate Bland, Amanda Wartick, Yi Pan, Han Song, Sk Jayadevan, Gene Hwang, and Art Pettigrue. 2018.  Conversational AI: the science behind the alexa prize.  _CoRR_ , abs/1801.03604.
  * Rongali et al. (2020) Subendhu Rongali, Luca Soldaini, Emilio Monti, and Wael Hamza. 2020.  Don’t parse, generate! a sequence to sequence architecture for task-oriented semantic parsing.  _Proceedings of The Web Conference 2020_.
  * Rush et al. (2015) Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015.  A neural attention model for abstractive sentence summarization.  _CoRR_ , abs/1509.00685.
  * See et al. (2017) Abigail See, Peter J. Liu, and Christopher D. Manning. 2017.  Get to the point: Summarization with pointer-generator networks.  _CoRR_ , abs/1704.04368.
  * Su et al. (2018) Chengwei Su, Rahul Gupta, Shankar Ananthakrishnan, and Spyros Matsoukas. 2018.  A re-ranker scheme for integrating large scale NLU models.  _CoRR_ , abs/1809.09605.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _CoRR_ , abs/1706.03762.
  * Zhang et al. (2018) Xingxing Zhang, Mirella Lapata, Furu Wei, and Ming Zhou. 2018.  Neural latent extractive document summarization.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 779–784, Brussels, Belgium. Association for Computational Linguistics.
  * Zhou et al. (2018) Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018\.  Neural document summarization by jointly learning to score and select sentences.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 654–663, Melbourne, Australia. Association for Computational Linguistics.


i}\chi)\right\rVert_{L^{2}(Y;\mathbb{C}^{3})}^{2}=\sum_{k\in\mathbb{Z}^{3}}|a_{k}\otimes\left(2\pi{\rm
i}k+{\rm i}\chi\right)|^{2},$

and therefore

$\left\lVert(\operatorname{sym}\nabla+{\rm
i}X_{\chi})\boldsymbol{u}\right\rVert_{L^{2}(Y;\mathbb{C}^{3})}^{2}=\sum_{k\in\mathbb{Z}^{3}}|a_{k}\odot\left(2\pi{\rm
i}k+{\rm i}\chi\right)|^{2}.$

Combining this with the inequality
$|\boldsymbol{a}\odot\boldsymbol{b}|\geq|\boldsymbol{a}||\boldsymbol{b}|/\mathchoice{{\hbox{$\displaystyle\sqrt{2\,}$}\lower
0.4pt\hbox{\vrule
height=6.44444pt,depth=-5.15558pt}}}{{\hbox{$\textstyle\sqrt{2\,}$}\lower
0.4pt\hbox{\vrule
height=6.44444pt,depth=-5.15558pt}}}{{\hbox{$\scriptstyle\sqrt{2\,}$}\lower
0.4pt\hbox{\vrule
height=4.51111pt,depth=-3.6089pt}}}{{\hbox{$\scriptscriptstyle\sqrt{2\,}$}\lower
0.4pt\hbox{\vrule height=3.22221pt,depth=-2.57779pt}}},$ we infer (A.5)–(A.6).
∎

######  Proposition A.10.

There is an extension operator mapping $\boldsymbol{u}\in H_{\\#}^{1}(Y_{\rm
stiff};\mathbb{C}^{3})$ to an element of $H_{\\#}^{1}(Y;\mathbb{C}^{3})$ (for
which we keep the same notation $\boldsymbol{u}$) such that

$\left\lVert\boldsymbol{u}\right\rVert_{H^{1}(Y;\mathbb{C}^{3})}\leq
C\left\lVert\boldsymbol{u}\right\rVert_{H^{1}(Y_{\rm
stiff};\mathbb{C}^{3})},\qquad\quad\bigl{\lVert}\bigl{(}\operatorname{sym}\nabla+{\rm
i}X_{\chi}\bigr{)}\boldsymbol{u}\bigr{\rVert}_{L^{2}(Y;\mathbb{C}^{3\times
3})}\leq C\bigl{\lVert}\bigr{(}\operatorname{sym}\nabla+{\rm
i}X_{\chi}\bigr{)}\boldsymbol{u}\bigr{\rVert}_{L^{2}(Y_{\rm
stiff};\mathbb{C}^{3\times 3})},$ (A.7)

for all $\chi\in Y$, where the constant $C$ depends only on $Y_{\rm stiff}$.

###### Proof.

This is a straightforward consequence of an analogous result for the extension
operator applied to quasiperiodic functions. For the related construction, see
[48]. ∎

######  Proposition A.11.

There exists a constant $C>0$ such that for every $\boldsymbol{u}\in
H_{\\#}^{1}(Y_{\rm stiff};\mathbb{C}^{3})$, $\chi\in
Y^{\prime}\setminus\\{0\\}$ we have the following estimate:

$\left\lVert\boldsymbol{u}\right\rVert_{H^{1}(Y_{\rm
stiff};\mathbb{C}^{3})}\leq
C|\chi|^{-1}\bigl{\lVert}\bigl{(}\operatorname{sym}\nabla+{\rm
i}X_{\chi}\bigr{)}\boldsymbol{u}\bigr{\rVert}_{L^{2}(Y_{\rm
stiff};\mathbb{C}^{3\times 3})}.$ (A.8)

###### Proof.

By Proposition A.10, the function $\boldsymbol{u}$ is extended to
$\boldsymbol{u}\in H_{\\#}^{1}(Y;\mathbb{C}^{3})$. Combining (A.5) and (A.7)
yields

$\left\lVert\boldsymbol{u}\right\rVert_{H^{1}(Y_{\rm
stiff};\mathbb{C}^{3})}\leq\left\lVert\boldsymbol{u}\right\rVert_{H^{1}(Y;\mathbb{C}^{3})}\leq
C|\chi|^{-1}\bigl{\lVert}\bigl{(}\operatorname{sym}\nabla+{\rm
i}X_{\chi}\bigr{)}\boldsymbol{u}\bigr{\rVert}_{L^{2}(Y;\mathbb{C}^{3\times
3})}\leq C|\chi|^{-1}\bigl{\lVert}\bigl{(}\operatorname{sym}\nabla+{\rm
i}X_{\chi}\bigr{)}\boldsymbol{u}\bigr{\rVert}_{L^{2}(Y_{\rm
stiff};\mathbb{C}^{3\times 3})}.$

∎

######  Proposition A.12.

There exists a constant $C>0$ such that for all $\boldsymbol{u}\in
H_{\\#}^{1}(Y_{\rm stiff};\mathbb{C}^{3})$, $\chi\in Y^{\prime}$ one has

$\displaystyle\left\lVert\boldsymbol{u}-\frac{1}{|Y_{\rm stiff}|}\int_{Y_{\rm
stiff}}\boldsymbol{u}\right\rVert_{H^{1}(Y_{\rm stiff};\mathbb{C}^{3})}\leq
C\bigl{\lVert}\bigl{(}\operatorname{sym}\nabla+{\rm
i}X_{\chi}\bigr{)}\boldsymbol{u}\bigr{\rVert}_{L^{2}(Y_{\rm
stiff};\mathbb{C}^{3\times 3})},$ (A.9)
$\displaystyle\left\lVert\boldsymbol{u}-\frac{1}{|\Gamma|}\int_{\Gamma}\boldsymbol{u}\right\rVert_{H^{1/2}(\Gamma;\mathbb{C}^{3})}\leq
C\bigl{\lVert}\bigl{(}\operatorname{sym}\nabla+{\rm
i}X_{\chi}\bigr{)}\boldsymbol{u}\bigr{\rVert}_{L^{2}(Y_{\rm
stiff};\mathbb{C}^{3\times 3})}.$ (A.10)

###### Proof.

The bound (A.9) is deduced from (A.6) and Proposition A.10 similar to how
Proposition A.11 was established. To prove (A.10), note first that by (A.9)
and the continuity of traces, one has

$\begin{split}\left\lVert\boldsymbol{u}-\frac{1}{|\Gamma|}\int_{\Gamma}\boldsymbol{u}\right\rVert_{L^{2}(\Gamma;\mathbb{C}^{3})}\leq\left\lVert\boldsymbol{u}-\frac{1}{|Y_{\rm
stiff}|}\int_{Y_{\rm
stiff}}\boldsymbol{u}\right\rVert_{L^{2}(\Gamma;\mathbb{C}^{3})}&\leq
C\left\lVert\boldsymbol{u}-\frac{1}{|Y_{\rm stiff}|}\int_{Y_{\rm
stiff}}\boldsymbol{u}\right\rVert_{H^{1}(Y_{\rm
stiff};\mathbb{C}^{3})}\\\\[3.99994pt] &\leq
C\left\lVert\left(\operatorname{sym}\nabla+{\rm
i}X_{\chi}\right)\boldsymbol{u}\right\rVert_{L^{2}(Y_{\rm
stiff};\mathbb{C}^{3\times 3})}.\end{split}$ (A.11)

Second, from Proposition A.8 and the trace inequality, one has

$\displaystyle\left\|\boldsymbol{u}-\frac{1}{|\Gamma|}\int_{\Gamma}\boldsymbol{u}\right\|_{H^{1/2}(\Gamma;\mathbb{C}^{3})}$
(A.12) $\displaystyle\hskip 56.9055pt\leq
C\left(\left\lVert\left(\operatorname{sym}\nabla+{\rm
i}X_{\chi}\right)\left(\boldsymbol{u}-\frac{1}{|\Gamma|}\int_{\Gamma}\boldsymbol{u}\right)\right\rVert_{L^{2}(Y_{\rm
stiff};\mathbb{C}^{3\times
3})}+\left\lVert\boldsymbol{u}-\frac{1}{|\Gamma|}\int_{\Gamma}\boldsymbol{u}\right\rVert_{L^{2}(\Gamma;\mathbb{C}^{3})}\right)$
$\displaystyle\hskip 56.9055pt\leq
C\left(\left\lVert\left(\operatorname{sym}\nabla+{\rm
i}X_{\chi}\right)\boldsymbol{u}\right\rVert_{L^{2}(Y_{\rm
stiff};\mathbb{C}^{3\times
3})}+|\chi|\left\lvert\frac{1}{|\Gamma|}\int_{\Gamma}\boldsymbol{u}\right\rvert_{L^{2}(\Gamma;\mathbb{C}^{3})}+\left\lVert\boldsymbol{u}-\frac{1}{|\Gamma|}\int_{\Gamma}\boldsymbol{u}\right\rVert_{L^{2}(\Gamma;\mathbb{C}^{3})}\right).$

The bound (A.10) now follows from (A.12) by using (A.11), (A.8), and the
continuity of traces. ∎

###  A.3 Well-posedness and regularity of elasticity boundary value problems

Here we state some results regarding the properties of the weak solution to
the boundary value problem

$-\operatorname{div}\left(\mathbb{A}\operatorname{sym}\nabla\boldsymbol{u}\right)=\boldsymbol{f}\
\ \mbox{ on $\Omega$},\qquad\ \
(\mathbb{A}\operatorname{sym}\nabla\boldsymbol{u}){\boldsymbol{n}}|_{\partial\Omega}+\boldsymbol{u}=\boldsymbol{g}\
\ \mbox{ on $\partial\Omega$},$ (A.13)

where $\Omega\subset\mathbb{R}^{3}$ is a bounded Lipschitz domain with the
exterior normal ${\boldsymbol{n}},$ $\boldsymbol{f}\in
L^{2}(\Omega;\mathbb{C}^{3})$, $\boldsymbol{g}\in
H^{-1/2}(\partial\Omega;\mathbb{C}^{3}),$ and the tensor of material
properties $\mathbb{A}\in L^{\infty}(\Omega;\mathbb{R}^{3\times 3\times
3\times 3})$ satisfies the following assumptions (cf. Assumption 2.1).

######  Assumption A.13.

  * •

Uniform positive-definiteness and uniform boundedness on symmetric matrices:
there exists $\nu>0$ such that
$\nu|\boldsymbol{\xi}|^{2}\leq\mathbb{A}(x)\boldsymbol{\xi}:\boldsymbol{\xi}\leq\nu^{-1}|\boldsymbol{\xi}|^{2}$
for all $\boldsymbol{\xi}\in\mathbb{R}^{3\times
3},\boldsymbol{\xi}^{\top}=\boldsymbol{\xi},$ $x\in\Omega.$

  * •

Material symmetries:
$[\mathbb{A}]_{ijkl}=[\mathbb{A}]_{jikl}=[\mathbb{A}]_{klij},$
$i,j,k,l\in\\{1,2,3\\}.$

The weak form of this problem is stated in the following definition.

######  Definition A.14 (Robin boundary problem for elasticity).

For given functions $\boldsymbol{f}\in H^{-1}(\Omega;\mathbb{C}^{3})$,
$\boldsymbol{g}\in H^{-1/2}(\partial\Omega;\mathbb{C}^{3}),$ find
$\boldsymbol{u}\in H^{1}(\Omega;\mathbb{C}^{3})$ such that

$\int_{\Omega}\mathbb{A}\operatorname{sym}\nabla\boldsymbol{u}:\overline{\operatorname{sym}\nabla\boldsymbol{v}}+\int_{\partial\Omega}\boldsymbol{u}\cdot\overline{\boldsymbol{v}}=\int_{\Omega}\boldsymbol{f}\cdot\overline{\boldsymbol{v}}+\int_{\partial\Omega}\boldsymbol{g}\cdot\overline{\boldsymbol{v}}\qquad\forall\boldsymbol{v}\in
H^{1}(\Omega;\mathbb{C}^{3}).$

Notice that the map
$\boldsymbol{v}\mapsto\int_{\partial\Omega}\boldsymbol{g}\cdot\overline{\boldsymbol{v}}$
is a bounded linear functional on $H^{1}(\Omega;\mathbb{C}^{3})$. Also, the
form

$(\boldsymbol{u},\boldsymbol{v})\mapsto\int_{\Omega}\mathbb{A}\operatorname{sym}\nabla\boldsymbol{u}:\overline{\operatorname{sym}\nabla\boldsymbol{v}}+\int_{\partial\Omega}\boldsymbol{u}\cdot\overline{\boldsymbol{v}},$

is coercive on $H^{1}(\Omega;\mathbb{C}^{3})$ by Proposition A.5. Thus by the
Lax-Milgram lemma, there exists a unique weak solution $\boldsymbol{u}\in
H^{1}(\Omega;\mathbb{C}^{3})$ of (A.13) in the sense of Definition A.14, and

$\lVert\boldsymbol{u}\rVert_{H^{1}(\Omega;\mathbb{C}^{3})}\leq
C\bigl{(}\lVert\boldsymbol{f}\rVert_{H^{-1}(\Omega;\mathbb{C}^{3})}+\lVert\boldsymbol{g}\rVert_{H^{-1/2}(\partial\Omega;\mathbb{C}^{3})}\bigr{)}.$

The following two lemmata can be found in, e.g., [42].

######  Lemma A.15 (Regularity of the solution of Robin problem).

Let $\Omega\subset\mathbb{R}^{3}$ be a bounded domain with boundary
$\partial\Omega$ of class $C^{1,1}$. Let $\boldsymbol{f}\in
L^{2}(\Omega;\mathbb{C}^{3})$, $\boldsymbol{g}\in
H^{1/2}(\partial\Omega;\mathbb{C}^{3})$ and, in addition to Assumption A.13,
$\mathbb{A}\in\mathcal{C}^{0,1}(\overline{\Omega};\mathbb{R}^{3\times 3\times
3\times 3})$. Then the unique weak solution to the problem (A.13) belongs to
$H^{2}(\Omega;\mathbb{C}^{3}),$ and

$\lVert\boldsymbol{u}\rVert_{H^{2}(\Omega;\mathbb{C}^{3})}\leq
C\bigl{(}\lVert\boldsymbol{f}\rVert_{L^{2}(\Omega;\mathbb{C}^{3})}+\lVert\boldsymbol{g}\rVert_{H^{1/2}(\partial\Omega;\mathbb{C}^{3})}\bigr{)}.$

We are also interested in the solution to the Dirichlet boundary value problem

$\left\\{\begin{array}[]{ll}-\operatorname{div}\left(\mathbb{A}\operatorname{sym}\nabla\boldsymbol{u}\right)=\boldsymbol{f}&\mbox{on
$\Omega$},\\\\[1.49994pt] \boldsymbol{u}=\boldsymbol{g}&\mbox{ on
$\partial\Omega$},\end{array}\right.$ (A.14)

where $\boldsymbol{f}\in H^{-1}(\Omega;\mathbb{C}^{3})$, $\boldsymbol{g}\in
H^{1/2}(\partial\Omega;\mathbb{C}^{3})$. Applying the Lax-Milgram lemma, we
infer that it has a unique weak solution $\boldsymbol{u}\in
H^{1}(\Omega;\mathbb{C}^{3})$.

######  Lemma A.16 (Regularity of the solution of Dirichlet problem).

Suppose that $\Omega\subset\mathbb{R}^{3}$ be a bounded domain with $C^{1,1}$
boundary $\partial\Omega,$ $\boldsymbol{f}\in L^{2}(\Omega;\mathbb{C}^{3})$,
$\boldsymbol{g}\in H^{3/2}(\partial\Omega;\mathbb{C}^{3}),$ and
$\mathbb{A}\in\mathcal{C}^{0,1}(\overline{\Omega};\mathbb{R}^{3\times 3\times
3\times 3})$. Then the solution to (A.14) belongs to
$H^{2}(\Omega;\mathbb{C}^{3}),$ and

$\lVert\boldsymbol{u}\rVert_{H^{2}(\Omega;\mathbb{C}^{3})}\leq
C\bigl{(}\lVert\boldsymbol{f}\rVert_{L^{2}(\Omega;\mathbb{C}^{3})}+\lVert\boldsymbol{g}\rVert_{H^{3/2}(\partial\Omega;\mathbb{C}^{3})}\bigr{)}.$

where the constant $C$ depends only on $\Omega$ and the
$\mathcal{C}^{0,1}$-norm of $\mathbb{A}$.

####  A.3.1 Trace extension lemma

A version of the following theorem can be found in [8].

######  Theorem A.17.

Let $\Omega\subset\mathbb{R}^{3}$ be a bounded domain with $C^{0,1}$ boundary
$\partial\Omega.$ Let $\boldsymbol{g}_{0}\in
H^{1}(\partial\Omega;\mathbb{C}^{3})$, $\boldsymbol{g}_{1}\in
L^{2}(\partial\Omega;\mathbb{C}^{3})$. Then there exists $\boldsymbol{u}\in
H^{2}(\Omega;\mathbb{C}^{3})$ such that
$\partial_{{\boldsymbol{n}}}\boldsymbol{u}=\boldsymbol{g}_{1}$ and
$\boldsymbol{u}=\boldsymbol{g}_{0}$ on $\partial\Omega$ if and only if

$\nabla_{\partial\Omega}\boldsymbol{g}_{0}+\boldsymbol{g}_{1}\otimes{\boldsymbol{n}}\in
H^{1/2}(\partial\Omega;\mathbb{C}^{3}),$ (A.15)

where $\nabla_{\partial\Omega}$ is the tangential gradient.

This leads us to the trace extension lemma.

######  Lemma A.18.

Let $\Omega\subset\mathbb{R}^{3}$ be a bounded domain with boundary
$\partial\Omega$ of class $C^{1,1}$ and
$\mathbb{A}\in\mathcal{C}^{0,1}(\overline{\Omega};\mathbb{R}^{3\times 3\times
3\times 3})$. Let $\boldsymbol{g}\in H^{1/2}(\partial\Omega;\mathbb{C}^{3})$.
Then there exists $\boldsymbol{u}\in H^{2}(\Omega;\mathbb{C}^{3})$ such that

$(\mathbb{A}\operatorname{sym}\nabla\boldsymbol{u}){\boldsymbol{n}}=\boldsymbol{g}\quad\mbox{
on $\partial\Omega$, }\quad\qquad\boldsymbol{u}=0\quad\mbox{ on
$\partial\Omega$. }$ (A.16)

###### Proof.

The first step is to note that for $\boldsymbol{u}\in
H^{2}(\Omega;\mathbb{C}^{3})$ such that $\boldsymbol{u}|_{\Gamma}=0$ one has

$\nabla\boldsymbol{u}|_{\partial\Omega}=\partial_{{\boldsymbol{n}}}\boldsymbol{u}|_{\partial\Omega}\otimes{\boldsymbol{n}}.$
(A.17)

Indeed, for an arbitrary point on $\partial\Omega$ consider an arbitrary
vector
${\boldsymbol{v}}=\alpha_{1}{\boldsymbol{\tau}}_{1}+\alpha_{2}{\boldsymbol{\tau}}_{2}+\alpha_{3}{\boldsymbol{n}}$,
where ${\boldsymbol{\tau}}_{1},{\boldsymbol{\tau}}_{2},{\boldsymbol{n}}$ is an
orthonormal basis of vectors ${\boldsymbol{\tau}}_{1},{\boldsymbol{\tau}}_{2}$
tangential on $\partial\Omega$ and the vector ${\boldsymbol{n}}$ is normal to
$\partial\Omega$. Then

$\nabla\boldsymbol{u}|_{\partial\Omega}\cdot{\boldsymbol{v}}=\alpha_{1}\partial_{{\boldsymbol{\tau}}_{1}}\boldsymbol{u}|_{\partial\Omega}+\alpha_{2}\partial_{{\boldsymbol{\tau}}_{2}}\boldsymbol{u}|_{\partial\Omega}+\alpha_{3}\partial_{{\boldsymbol{n}}}\boldsymbol{u}|_{\partial\Omega}=\alpha_{3}\partial_{{\boldsymbol{n}}}\boldsymbol{u}|_{\partial\Omega},$

due to the fact that
$\partial_{{\boldsymbol{\tau}}_{1}}\boldsymbol{u}|_{\partial\Omega}=\partial_{{\boldsymbol{\tau}}_{2}}\boldsymbol{u}=0|_{\partial\Omega}$.
But, since $\alpha_{3}={\boldsymbol{v}}\cdot{\boldsymbol{n}}$, one has

$\bigl{(}\nabla\boldsymbol{u}|_{\partial\Omega}\bigr{)}{\boldsymbol{v}}=\left({\boldsymbol{v}}\cdot{\boldsymbol{n}}\right)\partial_{{\boldsymbol{n}}}\boldsymbol{u}|_{\partial\Omega}=\bigl{(}\left(\partial_{{\boldsymbol{n}}}\boldsymbol{u}|_{\partial\Omega}\otimes{\boldsymbol{n}}\right)\bigr{)}{\boldsymbol{v}},$

and (A.17) follows. With this in hand, the first equation in (A.16) becomes
$(\mathbb{A}\left(\partial_{{\boldsymbol{n}}}\boldsymbol{u}|_{\partial\Omega}\odot{\boldsymbol{n}}\right)){\boldsymbol{n}}=\boldsymbol{g}.$
We proceed by introducing the new variable
$\boldsymbol{\omega}:=\partial_{{\boldsymbol{n}}}\boldsymbol{u}|_{\partial\Omega}$
and consider the problem of finding $\boldsymbol{\omega}$ such that

$\bigl{(}\mathbb{A}\left(\boldsymbol{\omega}\odot{\boldsymbol{n}}\right)\bigr{)}{\boldsymbol{n}}=\boldsymbol{g}.$
(A.18)

The second step of the proof consists in showing that there exists a unique
solution $\boldsymbol{\omega}\in H^{1/2}(\partial\Omega;\mathbb{C}^{3})$ to
(A.18). Note that this is an algebraic equation for every
$x\in\partial\Omega$. For a fixed point $x\in\partial\Omega$, we introduce an
operator $L_{x}:\mathbb{C}^{3}\to\mathbb{C}^{3}$ as follows:
$L_{x}\boldsymbol{\omega}:=(\mathbb{A}\left(\boldsymbol{\omega}\odot{\boldsymbol{n}}(x)\right)){\boldsymbol{n}(x)}.$
It is symmetric and positive definite:

$\displaystyle\langle
L_{x}\boldsymbol{\omega},\boldsymbol{w}\rangle_{\mathbb{C}^{3}}$
$\displaystyle=\bigl{\langle}\bigl{(}\mathbb{A}\left(\boldsymbol{\omega}\odot{\boldsymbol{n}}(x)\right)\bigr{)}{\boldsymbol{n}}(x),\boldsymbol{w}\bigr{\rangle}_{\mathbb{C}^{3}}=\mathbb{A}\left(\boldsymbol{\omega}\odot{\boldsymbol{n}}(x)\right):\left(\boldsymbol{w}\odot{\boldsymbol{n}}(x)\right)=\langle
L_{x}\boldsymbol{w},\boldsymbol{\omega}\rangle_{\mathbb{C}^{3}},$
$\displaystyle\langle
L_{x}\boldsymbol{\omega},\boldsymbol{\omega}\rangle_{\mathbb{C}^{3}}$
$\displaystyle=\mathbb{A}\left(\boldsymbol{\omega}\odot{\boldsymbol{n}}(x)\right):\left(\boldsymbol{\omega}\odot{\boldsymbol{n}}(x)\right)\geq\nu|\boldsymbol{\omega}\odot{\boldsymbol{n}}(x)|^{2}\geq\frac{\nu}{2}|\boldsymbol{\omega}|^{2}|{\boldsymbol{n}}(x)|^{2}=\frac{\nu}{2}|\boldsymbol{\omega}|^{2},$
(A.19)

where we have used Assumption A.13. Thus, the problem
$L_{x}\boldsymbol{\omega}(x)=\boldsymbol{g}(x)$ has a unique solution for a.e.
$x\in\partial\Omega$. It follows from (A.19) that
$\det\left(L_{x}\right)\geq\alpha>0$, uniformly in $x\in\partial\Omega$. By
$C^{0,1}$ regularity of both the material coefficients and the normal
${\boldsymbol{n}}$, we know that both $\det\left(L_{x}\right)$ and
$\left(\det\left(L_{x}\right)\right)^{-1}$ are of class $C^{0,1}$. Then, by
virtue of Cramer’s rule, the function
$\boldsymbol{\omega}(x):=\left(L_{x}\right)^{-1}\boldsymbol{g}(x)$ belongs to
$H^{1/2}(\partial\Omega;\mathbb{C}^{3})$. Thus, we have reduced the problem
(A.16) to finding $\boldsymbol{u}\in H^{2}(\Omega;\mathbb{C}^{3})$ such that
$\partial_{{\boldsymbol{n}}}\boldsymbol{u}=\boldsymbol{\omega}$ on
$\partial\Omega,$ $\boldsymbol{u}=0$ on $\partial\Omega$, where
$\boldsymbol{\omega}$ is the solution of (A.18). To complete the proof, it
remains to check the validity of the condition (A.15) and apply Theorem A.17,
which is possible as $\boldsymbol{\omega}\otimes{\boldsymbol{n}}\in
H^{1/2}(\partial\Omega;\mathbb{C}^{3})$ due to the
$C^{0,1}(\partial\Omega;\mathbb{R}^{3})$ regularity of ${\boldsymbol{n}}$. ∎

###  A.4 Auxiliary proofs

###### Proof of Theorem 4.3.

The definition of $\Pi_{\chi}^{\rm stiff}$ can be restated as follows:
$\Pi_{\chi}^{\rm stiff}\boldsymbol{g}:=\boldsymbol{G}+\boldsymbol{u},$ where
$\boldsymbol{u}\in H_{\\#}^{1}(Y_{\rm stiff};\mathbb{C}^{3}),$
$\boldsymbol{u}|_{\Gamma}=0$ is such that

$a_{0,\chi}^{\rm stiff}(\boldsymbol{u},\boldsymbol{v})=-a_{0,\chi}^{\rm
stiff}(\boldsymbol{G},\boldsymbol{v})\qquad\forall\boldsymbol{v}\in
H_{\\#}^{1}(Y_{\rm stiff};\mathbb{C}^{3}),\quad\boldsymbol{v}|_{\Gamma}=0,$

and $\boldsymbol{G}\in H_{\\#}^{1}(Y_{\rm stiff};\mathbb{C}^{3})$ is an
extension of $\boldsymbol{g}\in H^{1/2}(\Gamma;\mathbb{C}^{3})$ satisfying the
bound $\left\lVert\boldsymbol{G}\right\rVert_{H^{1}}\leq
C\left\lVert\boldsymbol{g}\right\rVert_{H^{1/2}(\Gamma;\mathbb{C}^{3})},$ see
Proposition A.3. Recall that $a_{0,\chi}^{\rm stiff}:H_{\\#}^{1}(Y_{\rm
stiff};\mathbb{C}^{3})\times H_{\\#}^{1}(Y_{\rm
stiff};\mathbb{C}^{3})\to\mathbb{R}$ is the sesquilinear form associated with
the operator $\mathcal{A}_{0,\chi}^{\rm stiff}$. We aim to find an expansion
of the solution $\boldsymbol{u}\in H_{\\#}^{1}(Y_{\rm stiff};\mathbb{C}^{3})$,
$\boldsymbol{u}|_{\Gamma}=0,$ to

$\displaystyle\int_{Y_{\rm stiff}}\mathbb{A}_{\rm
stiff}\left(\operatorname{sym}\nabla+{\rm
i}X_{\chi}\right)\boldsymbol{u}(y):\left(\operatorname{sym}\nabla+{\rm
i}X_{\chi}\right)\boldsymbol{v}(y)dy$ (A.20) $\displaystyle=-\int_{Y_{\rm
stiff}}\mathbb{A}_{\rm stiff}\left(\operatorname{sym}\nabla+{\rm
i}X_{\chi}\right)\boldsymbol{G}(y):\left(\operatorname{sym}\nabla+{\rm
i}X_{\chi}\right)\boldsymbol{v}(y)dy\qquad\forall\boldsymbol{v}\in
H_{\\#}^{1}(Y_{\rm stiff};\mathbb{C}^{3}),\ \boldsymbol{v}|_{\Gamma}=0.$

in the form

$\boldsymbol{u}=\boldsymbol{u}_{0}+\sum_{n=1}^{\infty}\boldsymbol{u}_{n},$
(A.21)

where ${\boldsymbol{u}}_{n}|_{\Gamma}=0$ for all $n,$ the $H^{1}$ norm of
$\boldsymbol{u}_{n}$ is of order $|\chi|^{n}$, while the error of
approximation

$\boldsymbol{u}_{\rm
error,n}:=\boldsymbol{u}-\boldsymbol{u}_{0}-\sum_{k=1}^{n}\boldsymbol{u}_{k}$

is of order $|\chi|^{n+1}$. The leading-order term $\boldsymbol{u}_{0}$
corresponds to the harmonic lift in the case $\chi=0$. With this apriori
intention, we plug the expansion (A.21) into (A.20) and equate the terms which
would be of order $\mathcal{O}(1)$ to obtain

$\displaystyle\int_{Y_{\rm stiff}}\mathbb{A}_{\rm
stiff}\operatorname{sym}\nabla\boldsymbol{u}_{0}(y):\operatorname{sym}\nabla\boldsymbol{v}(y)dy$
$\displaystyle=-\int_{Y_{\rm stiff}}\mathbb{A}_{\rm
stiff}\operatorname{sym}\nabla\boldsymbol{G}(y):\operatorname{sym}\nabla\boldsymbol{v}(y)dy\qquad\forall\boldsymbol{v}\in
H_{\\#}^{1}(Y_{\rm stiff};\mathbb{C}^{3}),\quad\boldsymbol{v}|_{\Gamma}=0.$

We recognise here the definition of the order-zero lift operator $\Pi_{0}^{\rm
stiff}$, namely $\Pi_{0}^{\rm
stiff}\boldsymbol{g}=\boldsymbol{u}_{0}+\boldsymbol{G}=:\widetilde{\boldsymbol{u}}.$
It is clear that

$\left\lVert\boldsymbol{u}_{0}\right\rVert_{H^{1}(Y_{\rm
stiff};\mathbb{C}^{3})}\leq
C\left\lVert\boldsymbol{G}\right\rVert_{H^{1}(Y_{\rm
stiff};\mathbb{C}^{3})}\leq
C\left\lVert\boldsymbol{g}\right\rVert_{H^{1/2}(\Gamma;\mathbb{C}^{3})}.$

We proceed to define $\boldsymbol{u}_{1}$ as the solutions to

$\displaystyle\int_{Y_{\rm stiff}}\mathbb{A}_{\rm
stiff}\operatorname{sym}\nabla\boldsymbol{u}_{1}(y):\operatorname{sym}\nabla\boldsymbol{v}(y)dy$
(A.22) $\displaystyle=$ $\displaystyle-\int_{Y_{\rm stiff}}\mathbb{A}_{\rm
stiff}{\rm
i}X_{\chi}\widetilde{\boldsymbol{u}}(y):\operatorname{sym}\nabla\boldsymbol{v}(y)dy-\int_{Y_{\rm
stiff}}\mathbb{A}_{\rm
stiff}\operatorname{sym}\nabla\widetilde{\boldsymbol{u}}(y):{\rm
i}X_{\chi}\boldsymbol{v}(y)dy\qquad\forall\boldsymbol{v}\in H_{\\#}^{1}(Y_{\rm
stiff};\mathbb{C}^{3}),\ \boldsymbol{v}|_{\Gamma}=0.$

Clearly, one has $\left\lVert\boldsymbol{u}_{1}\right\rVert_{H^{1}(Y_{\rm
stiff};\mathbb{C}^{3})}\leq
C|\chi|\left\lVert\boldsymbol{g}\right\rVert_{H^{1/2}(\Gamma;\mathbb{C}^{3})}.$
The next-order term of the expansion is defined by

$\displaystyle\int_{Y_{\rm stiff}}\mathbb{A}_{\rm
stiff}\operatorname{sym}\nabla\boldsymbol{u}_{2}(y):\operatorname{sym}\nabla\boldsymbol{v}(y)dy=$
$\displaystyle-\int_{Y_{\rm stiff}}\mathbb{A}_{\rm stiff}{\rm
i}X_{\chi}\widetilde{\boldsymbol{u}}(y):{\rm
i}X_{\chi}\boldsymbol{v}(y)dy-\int_{Y_{\rm stiff}}\mathbb{A}_{\rm stiff}{\rm
i}X_{\chi}\boldsymbol{u}_{1}(y):\operatorname{sym}\nabla\boldsymbol{v}(y)dy$
$\displaystyle-\int_{Y_{\rm stiff}}\mathbb{A}_{\rm
stiff}\operatorname{sym}\nabla\boldsymbol{u}_{1}(y):{\rm
i}X_{\chi}\boldsymbol{v}(y)dy\qquad\forall\boldsymbol{v}\in H_{\\#}^{1}(Y_{\rm
stiff};\mathbb{C}^{3}),\ \boldsymbol{v}|_{\Gamma}=0.$

We see that $\left\lVert\boldsymbol{u}_{2}\right\rVert_{H^{1}(Y_{\rm
stiff};\mathbb{C}^{3})}\leq
C|\chi|^{2}\left\lVert\boldsymbol{g}\right\rVert_{H^{1/2}(\Gamma;\mathbb{C}^{3})}.$
Continuing this process by induction for $n\geq 3$, we define
$\boldsymbol{u}_{n}$ by the recurrence relation

$\displaystyle\int_{Y_{\rm stiff}}$ $\displaystyle\mathbb{A}_{\rm
stiff}\operatorname{sym}\nabla\boldsymbol{u}_{n}(y):\operatorname{sym}\nabla\boldsymbol{v}(y)dy=-\int_{Y_{\rm
stiff}}\mathbb{A}_{\rm stiff}{\rm i}X_{\chi}\boldsymbol{u}_{n-2}(y):{\rm
i}X_{\chi}\boldsymbol{v}(y)dy$ $\displaystyle-\int_{Y_{\rm
stiff}}\mathbb{A}_{\rm stiff}{\rm
i}X_{\chi}\boldsymbol{u}_{n-1}(y):\operatorname{sym}\nabla\boldsymbol{v}(y)dy-\int_{Y_{\rm
stiff}}\mathbb{A}_{\rm
stiff}\operatorname{sym}\nabla\boldsymbol{u}_{n-1}(y):{\rm
i}X_{\chi}\boldsymbol{v}(y)dy\qquad\forall\boldsymbol{v}\in H_{\\#}^{1}(Y_{\rm
stiff};\mathbb{C}^{3}),\ \boldsymbol{v}|_{\Gamma}=0,$

so that $\left\lVert\boldsymbol{u}_{n}\right\rVert_{H^{1}(Y_{\rm
stiff};\mathbb{C}^{3})}\leq
C|\chi|^{n}\left\lVert\boldsymbol{g}\right\rVert_{H^{1/2}(\Gamma;\mathbb{C}^{3})}$
for all $n\in\mathbb{N}.$ The error term

$\boldsymbol{u}_{\rm
error,n}:=\boldsymbol{u}-\sum_{k=0}^{n}\boldsymbol{u}_{k},\quad
n\in\mathbb{N},$

satisfies

$\displaystyle\int_{Y_{\rm stiff}}$ $\displaystyle\mathbb{A}_{\rm
stiff}\left(\operatorname{sym}\nabla+{\rm i}X_{\chi}\right)\boldsymbol{u}_{\rm
error,n}(y):\left(\operatorname{sym}\nabla+{\rm
i}X_{\chi}\right)\boldsymbol{v}(y)dy$ $\displaystyle=-\int_{Y_{\rm
stiff}}\mathbb{A}_{\rm stiff}{\rm i}X_{\chi}\boldsymbol{u}_{n}(y):{\rm
i}X_{\chi}\boldsymbol{v}(y)dy-\int_{Y_{\rm stiff}}\mathbb{A}_{\rm stiff}{\rm
i}X_{\chi}\boldsymbol{u}_{n-1}(y):{\rm i}X_{\chi}\boldsymbol{v}(y)dy$
$\displaystyle-\int_{Y_{\rm stiff}}\mathbb{A}_{\rm stiff}{\rm
i}X_{\chi}\boldsymbol{u}_{n}(y):\operatorname{sym}\nabla\boldsymbol{v}(y)dy-\int_{Y_{\rm
stiff}}\mathbb{A}_{\rm
stiff}\operatorname{sym}\nabla\boldsymbol{u}_{n}(y):{\rm
i}X_{\chi}\boldsymbol{v}(y)dy\qquad\forall\boldsymbol{v}\in H_{\\#}^{1}(Y_{\rm
stiff};\mathbb{C}^{3}),\ \boldsymbol{v}|_{\Gamma}=0.$

It follows that $\lVert\boldsymbol{u}_{\rm error,n}\rVert_{H^{1}(Y_{\rm
stiff};\mathbb{C}^{3})}\leq
C|\chi|^{n+1}\left\lVert\boldsymbol{g}\right\rVert_{H^{1/2}(\Gamma;\mathbb{C}^{3})}.$
∎

###### Proof of Lemma 5.1.

The proof is by contradiction. Assume that for all $n\in\mathbb{N}$ there
exist $\chi_{n}\in Y^{\prime}$ and
$\boldsymbol{f}_{n}\in\widehat{\mathcal{H}}_{\chi_{n}}^{\rm stiff(soft)}$ such
that $\lVert\boldsymbol{f}_{n}\rVert_{H^{1}(Y_{\rm
stiff(soft)};\mathbb{C}^{3})}=1$ and
$\lim_{n\to\infty}\lVert\boldsymbol{f}_{n}\rVert_{L^{2}(Y_{\rm
stiff(soft)};\mathbb{C}^{3})}=0$. From this, it is clear that

$\boldsymbol{f}_{n}\xrightharpoonup{H^{1}(Y_{\rm
stiff(soft)};\mathbb{C}^{3})}0.$ (A.23)

We proceed by defining
$\boldsymbol{g}_{n}:=\boldsymbol{f}_{n}|_{\Gamma}\in\widehat{\mathcal{E}}_{\chi_{n}}$,
i.e., $\boldsymbol{f}_{n}=\widehat{\Pi}_{\chi_{n}}^{\rm
stiff(soft)}\boldsymbol{g}_{n}.$ Since
$\lVert\boldsymbol{f}_{n}\rVert_{H^{1}(Y_{\rm
stiff(soft)};\mathbb{C}^{3})}=1$, due to the continuity of the trace operator,
we conclude that the sequence $(\boldsymbol{g}_{n})_{n\in\mathbb{N}}$ is
bounded in $H^{1/2}(\Gamma;\mathbb{C}^{3})$ and there exists
$\boldsymbol{g}\in\mathcal{E}$ such that (on a subsequence)
$\boldsymbol{g}_{n}\xrightharpoonup{H^{1/2}(\Gamma;\mathbb{C}^{3})}\boldsymbol{g}.$
Furthermore, due to Proposition 3.18, the sequence
$\left(\lVert\boldsymbol{g}_{n}\rVert_{H^{1/2}(\Gamma;\mathbb{C}^{3})}\right)_{n\in\mathbb{N}}$
is bounded below by a $\chi$-independent $C>0:$

$\lVert\boldsymbol{g}_{n}\rVert_{H^{1/2}(\Gamma;\mathbb{C}^{3})}\geq C.$
(A.24)

However, due to the compactness of the trace operator acting from
$H^{1}(Y_{\rm stiff(soft)};\mathbb{C}^{3})$ to $L^{2}(\Gamma;\mathbb{C}^{3})$,
from (A.23) we infer that
$\boldsymbol{g}_{n}\xrightarrow{L^{2}(\Gamma;\mathbb{C}^{3})}0,$ hence
$\boldsymbol{g}=0.$ Next, we note that, on the one hand,

$\langle\Lambda_{\chi_{n}}^{\rm
stiff}\boldsymbol{g}_{n},\boldsymbol{g}_{n}\rangle_{L^{2}(\Gamma;\mathbb{C}^{3})}+\langle\boldsymbol{g}_{n},\boldsymbol{g}_{n}\rangle_{L^{2}(\Gamma;\mathbb{C}^{3})}\leq\left(C|\chi|^{2}+1\right)\lVert\boldsymbol{g}_{n}\rVert_{L^{2}(\Gamma;\mathbb{C}^{3})}^{2},$

where we use the fact that
$\boldsymbol{g}_{n}\in\widehat{\mathcal{E}}_{\chi_{n}}$. On the other hand,
one has

$\displaystyle\langle\Lambda_{\chi_{n}}^{\rm
stiff}\boldsymbol{g}_{n},\boldsymbol{g}_{n}\rangle_{L^{2}(\Gamma;\mathbb{C}^{3})}+\langle\boldsymbol{g}_{n},\boldsymbol{g}_{n}\rangle_{L^{2}(\Gamma;\mathbb{C}^{3})}$
$\displaystyle=\lambda_{\chi_{n}}^{\rm
stiff}(\boldsymbol{g}_{n},\boldsymbol{g}_{n})+\lVert\boldsymbol{g}_{n}\rVert_{L^{2}(\Gamma;\mathbb{C}^{3})}^{2}$
$\displaystyle\geq C\left(\left\lVert\left(\operatorname{sym}\nabla+{\rm
i}X_{\chi_{n}}\right)\Pi_{\chi_{n}}^{\rm
stiff}\boldsymbol{g}_{n}\right\rVert_{L^{2}(Y_{\rm
stiff};\mathbb{C}^{3})}^{2}+\lVert\boldsymbol{g}_{n}\rVert_{L^{2}(\Gamma;\mathbb{C}^{3})}^{2}\right)$
$\displaystyle\geq C\left\lVert\Pi_{\chi_{n}}^{\rm
stiff}\boldsymbol{g}_{n}\right\rVert_{H^{1}(Y_{\rm
stiff};\mathbb{C}^{3})}^{2}\geq
C\left\lVert\boldsymbol{g}_{n}\right\rVert_{H^{1/2}(\Gamma;\mathbb{C}^{3})}^{2},$

where we use the coercivity of the sesquilinear form (43), Proposition (A.8),
and the continuity of the trace operator. It follows that

$\left\lVert\boldsymbol{g}_{n}\right\rVert_{H^{1/2}(\Gamma;\mathbb{C}^{3})}\leq
C\lVert\boldsymbol{g}_{n}\rVert_{L^{2}(\Gamma;\mathbb{C}^{3})}\to 0,$

which contradicts (A.24). ∎

## Acknowledgements

KC and AVK have been supported by EPSRC (Grants EP/L018802/2, EP/V013025/1.)
IV and JŽ have been supported by the Croatian Science Foundation under Grant
agreement No. IP-2018-01-8904 (Homdirestroptcm) and under Grant agreement No.
IP-2022-10-5181 (HOMeOS).

## Data availability statement

This manuscript has no associated data.

## References

  * [1] M. S. Agranovich, 2006. On a mixed Poincaré–Steklov type spectral problem in a Lipschitz domain. Russian J. Math. Phys. 13(3), 239-244.
  * [2] G. Allaire, 1992. Homogenization and two-scale convergence, SIAM J. Math. Anal. 23, 1482–1518.
  * [3] T. Arbogast, J. Douglas, U. Hornung, 1990. Derivation of the double porosity model of single phase flow via homogenisation theory. SIAM J. Math. Anal. 21(4), 823–836.
  * [4] M. Sh. Birman, T. A. Suslina, 2004. Second order periodic differential operators. Threshold properties and homogenisation. St. Petersburg Math. J. 15(5), 639–714.
  * [5] M. Sh. Birman, T. A. Suslina, 2006. Averaging of periodic elliptic differential operators taking a corrector into account. St. Petersburg Math. J. 17(6), 897–973.
  * [6] M. Sh. Birman, M. Z. Solomjak.  Spectral Theory of Selfadjoint Operators in Hilbert Space.  Mathematics and its Applications (Soviet Series). D. Reidel Publishing Co., Dordrecht, 1987.
  * [7] L. Brillouin, 1953. Wave Propagation in Periodic Structures. Dover Publications, New York.
  * [8] A. Buffa, G. Geymonat, 2001. On traces of functions in $W^{2,p}(\Omega)$ for Lipschitz domains in $\mathbb{R}^{3}.$ C. R. Acad. Sci. Paris Sér. I Math. 332(8), 699–704.
  * [9] M. Bužančić, K Cherednichenko, I. Velčić, J. Žubrinić, 2022. Spectral and evolution analysis of composite elastic plates with high contrast. J. Elast. 152, 79–177.
  * [10] F. Capolino, 2009. Theory and Phenomena of Metamaterials. Taylor & Francis.
  * [11] K. Cherednichenko, S. Cooper, 2015. Homogenisation of the system of high-contrast Maxwell equations. Mathematika 61(2), 475–500.
  * [12] K. Cherednichenko, S. Cooper, 2016. Resolvent estimates for high-contrast homogenisation problems. Arch. Rational Mech. Anal. 219(3), 1061–1086.
  * [13] K. Cherednichenko, S. Cooper, 2018. Asymptotic behaviour of the spectra of systems of Maxwell equations in periodic composite media with high contrast. Mathematika 64(2), 583–605.
  * [14] K. Cherednichenko, S. D’Onofrio, 2023. Order-sharp norm-resolvent homogenisation estimates for Maxwell equations on periodic singular structures: the case of non-zero current and the general system. arXiv: 2007.04836, 25 pp.
  * [15] K. D. Cherednichenko, Yu. Yu. Ershova, A. V. Kiselev, 2019. Time-dispersive behaviour as a feature of critical contrast media, SIAM J. Appl. Math. 79(2), 690–715.
  * [16] K. D. Cherednichenko, Yu. Yu. Ershova, A. V. Kiselev, 2020.  Effective behaviour of critical-contrast PDEs: micro-resonances, frequency conversion, and time dispersive properties. I. Comm. Math. Phys. 375, 1833–1884.
  * [17] K. D. Cherednichenko, Yu. Yu. Ershova, A. V. Kiselev, 2023. Norm-resolvent convergence for Neumann Laplacians on manifolds thinning to graphs. arXiv: 2205.04397, 24 pp.
  * [18] K. Cherednichenko, Y. Ershova, A. Kiselev, S. Naboko, 2019. Unified approach to critical-contrast homogenisation with explicit links to time-dispersive media. Trans. Moscow Math. Soc. 80(2), 295–342.
  * [19] K. Cherednichenko, Y. Ershova, A. Kiselev, V. A. Ryzhov, L. O. Silva, 2023. Asymptotic analysis of operator families and applications to resonant media. Oper. Theory: Adv. Appl. 291, 239–311.
  * [20] K. D. Cherednichenko, Yu. Yu. Ershova, S. N. Naboko, 2021. Functional model for generalised resolvents and its application to time-dispersive media. arXiv: 2111.05387, 24 pp.
  * [21] K. D. Cherednichenko, A. V. Kiselev, 2017. Norm-resolvent convergence of one-dimensional high-contrast periodic problems to a Kronig-Penney dipole-type model. Comm. Math. Phys. 349(2), 441–480.
  * [22] K. Cherednichenko, A. Kiselev, L. Silva, 2022. Operator-norm resolvent asymptotic analysis of continuous media with low-index inclusions. Math. Notes 111(3–4), 373–387.
  * [23] K. Cherednichenko, I. Velčić, 2022. Sharp operator-norm asymptotics for linearised elastic plates with rapidly oscillating periodic properties. J. London Math. Soc. 105(3), 1634–1680.
  * [24] K. Cherednichenko, I. Velčić, J. Žubrinić, 2023. Operator-norm resolvent estimates for thin elastic periodically heterogeneous rods in moderate contrast. Calc. Var. Partial Differential Equations 62(5), Paper No. 147, 72 pp.
  * [25] S. Cooper, I. Kamotski, V. P. Smyshlyaev, 2023. Uniform asymptotics for a family of degenerating variational problems and multiscale approximations with error estimates. arXiv: 2307:13151, 69 pp.
  * [26] V. Derkach, 2015. Boundary triples, Weyl functions, and the Kreĭn formula, in Operator Theory: Living Reference Work (Springer, Basel). https://doi.org/10.1007/978-3-0348-0692-3_32-1
  * [27] P. Exner, O. Post, 2005. Convergence of spectra of graph-like thin manifolds. J. Geom. Phys. 54(1), 77–115.
  * [28] A. Figotin, P. Kuchment, 1998. Spectral properties of classical waves in high-contrast periodic media. SIAM J. Appl. Math 58(2), 683–702,
  * [29] A. Figotin, J. H. Schenker, 2005. Spectral analysis of time dispersive and dissipative systems, J. Stat. Phys. 118(1–2), 199–263.
  * [30] A. Figotin, J. H. Schenker, 2007. Hamiltonian structure for dispersive and dissipative dynamical systems. J. Stat. Phys. 128(4), 969–1056.
  * [31] L. Friedlander, 2002. On the density of states of periodic media in the large coupling limit. Comm. Partial Differential Equations 27(1–2), 355–380.
  * [32] G. Griso, 2004. Error estimate and unfolding for periodic homogenization. Asymptot. Anal. 40(3–4), 269–286.
  * [33] G. Grubb, 1984. Singular Green operators and their spectral asymptotics. Duke Math. J. 51(3), 477–528.
  * [34] T. Kato, 1995. Perturbation Theory for Linear Operators. Springer-Verlag, Berlin.
  * [35] C. E. Kenig, F. Lin, Z. Shen, 2012. Convergence rates in $L^{2}$ for elliptic homogenization problems. Arch. Rational Mech. Anal. 203(3), 1009–1036.
  * [36] S. G. Krein, Ju. I. Petunin, E. M. Semenov, 1982. Interpolation of Linear Operators. Transl. Math. Monogr. 54, American Mathematical Society, Providence, RI.
  * [37] P. Kuchment, 1993. Floquet Theory for Partial Differential Equations. Birkhäuser.
  * [38] P. Kuchment, H. Zeng, 2001. Convergence of spectra of mesoscopic systems collapsing onto a graph. J. Math. Anal. Appl. 258(2), 671–700.
  * [39] P. Kuchment, H. Zeng, 2004. Asymptotics of spectra of Neumann Laplacians in thin domains. Contemporary Mathematics 327, Amer. Math. Soc., Providence, RI, 199–213.
  * [40] P. D. Lax, R. S. Phillips, 1967.  Scattering Theory.  Pure and Applied Mathematics, Vol. 26. Academic Press, New York-London.
  * [41] Y.-S. Lim, 2023. A high-contrast composite with annular inclusions: Norm-resolvent asymptotics. arXiv: 2212.10206, 54 pp.
  * [42] W. McLean, 2000. Strongly Elliptic Systems and Boundary Integral Equations. Cambridge University Press, Cambridge.
  * [43] G. W. Milton, A. V. Cherkaev, 1995. Which elasticity tensors are realizable? J. Eng. Mater. Technol. 117(4), 483–493.
  * [44] P. W. Milonni, 2005. Fast Light, Slow Light and Left-Handed Light. Taylor & Francis, New York.
  * [45] D. Mitrea, 2018. Distributions, Partial Differential Equations, and Harmonic Analysis. Springer, New York.
  * [46] M. Neumark, 1940. Spectral functions of a symmetric operator. (Russian) Bull. Acad. Sci. URSS. Ser. Math. 4, 277–318.
  * [47] M. Neumark, 1943 Positive definite operator functions on a commutative group. (Russian) Bull. Acad. Sci. URSS Ser. Math. 7, 237–244.
  * [48] O. A. Oleĭnik, A. S. Shamaev, G. A. Yosifian, 1992. Mathematical Problems in Elasticity and Homogenization. North-Holland Publishing Co., Amsterdam.
  * [49] M. A. Pakhnin, T. A. Suslina, 2013. Operator error estimates for the homogenization of the elliptic Dirichlet problem in a bounded domain. St. Petersburg Math. J. 24(6), 949–976.
  * [50] O. Post, 2012. Spectral Analysis on Graph-Like Spaces, Lecture Notes in Mathematics 2039, Springer.
  * [51] V. Ryzhov, 2007.  Functional model of a class of non-selfadjoint extensions of symmetric operators.  In: Operator theory, analysis and mathematical physics. Oper. Theory Adv. Appl. 174, 117–158.
  * [52] V. Ryzhov, 2020. Linear operators and operator functions associated with spectral boundary value problems. In: Analysis as a Tool in Mathematical Physics. Oper. Theory: Adv. Appl. 276, 576–626.
  * [53] J. Selden, 2005. Periodic operators in high-contrast media and the integrated density of states function. Comm. Partial Differential Equations 30(7–9), 1021–1037.
  * [54] M. Schechter, 1960. A generalization of the problem of transmission. Ann. Scuola Norm. Sup. Pisa 14(3), 207–36.
  * [55] V. P. Smyshlyaev, 2009. Propagation and localization of elastic waves in highly anisotropic periodic composites via two-scale homogenization. Mech. Mater. 41(4), 434–447.
  * [56] A. V. Štraus, 1954. Generalised resolvents of symmetric operators. (Russian) _Izv. Akad. Nauk SSSR, Ser. Mat._ 18, 51–86.
  * [57] A. V. Štrauss, 1970. Extensions and generalized resolvents of a non-densely defined symmetric operator. (Russian) Izv. Akad. Nauk SSSR Ser. Mat. 34, 175–202.
  * [58] A. V. Štraus, 1999. Functional models and generalized spectral functions of symmetric operators. St. Petersburg Math. J. 10(5), 733–784.
  * [59] T. A. Suslina, 2018. Spectral approach to homogenization of elliptic operators in a perforated space. Rev. Math. Phys. 30(8), 1840016, 57 pp.
  * [60] C. Tretter, 2008. Spectral Theory of Block Operator Matrices and Applications. World Scientific.
  * [61] H. Triebel, 1978. Interpolation Theory, Function Spaces, Differential Operators. North-Holland.
  * [62] V. G. Veselago, 1968. The electrodynamics of substances with simultaneously negative values of $\varepsilon$ and $\mu.$ Soviet Phys. Uspekhi 10, 509–514.
  * [63] V. V. Zhikov, 1989. Spectral approach to asymptotic diffusion problems. Differential equations 25(1), 33-39.
  * [64] V. V. Zhikov, 2000. On an extension of the method of two-scale convergence and its applications, Sb. Math. 191(7), 973–1014.
  * [65] V. Zhikov, S. Pastukhova, 2005. On operator estimates for some problems in homogenization theory. Russian J. Math. Phys. 12(4), 515–524.
  * [66] V. V. Zhikov, S. E. Pastukhova, 2013. On gaps in the spectrum of the operator of elasticity theory on a high contrast periodic structure. J. Math. Sci. 188(3), 227–240.