11institutetext: ITMO University,
49 Kronverksky Pr., St. Peterburg, 197101

# Combination of abstractive and extractive approaches for summarization of
long scientific texts

Tretyak Vladislav<EMAIL_ADDRESS>Stepanov Denis
<EMAIL_ADDRESS>
###### Abstract

In this research work, we present a method to generate summaries of long
scientific documents that uses the advantages of both extractive and
abstractive approaches. Before producing a summary in an abstractive manner,
we perform the extractive step, which then is used for conditioning the
abstractor module. We used pre-trained transformer-based language models, for
both extractor and abstractor. Our experiments showed that using extractive
and abstractive models jointly significantly improves summarization results
and ROUGE scores.

##  1 Introduction

####

The language modeling task in the general case is a process of learning the
joint probability function of sequences of words in a natural language.
Statistical Language Modeling, or Language Modeling, is the development of
probabilistic models that are able to predict the next word in the sequence
given the words that precede, it also known as context. Language models could
operate on different sequence levels. Small language models work with
sequences of chars and words. While the big language model works with
sentences. But most common language model operates with sequences of words.

####

The language model could be used standalone to predict words according to
context but usually, the language model is used to solve more challenging
tasks. For example, it helps to solve a large range of natural language tasks,
such as machine translation, natural language understanding, speech
recognition, information retrieval, text summarization. In other words,
language models are used in most real-world natural language processing
applications and the quality of such applications mostly depends on the
language model’s performance. That’s why language modeling task plays a major
role in natural language processing and artificial intelligence research.

####

The first neural language model used feed-forward architecture. One of the
main features of using neural language models is getting representation vector
of words sequences. These word vectors usually called embedding vector [11]
and embeddings for similar words located closer to each other in dimension
space, also having similar representations. After successful usage of feed-
forward networks, recurrent neural networks [5] achieved better results in
language modeling tasks because of its ability to take into account the
position of words in sentences and producing contextual word embeddings [13].
Long Short-Term Memory networks [6] allows the language model to learn the
relevant context of longer sequences than feed-forward or RNN’s, because of
its more sophisticated memory mechanism. Next, the attention mechanism [6]
made improvement in language modeling tasks with a combination of sequence to
sequence framework [18]. The attention mechanism improves memory mechanism of
recurrent neural networks by giving the ability for the decoder network to
look at the whole context.

####

The next big step in the language modeling task was developing transformer[19]
architecture, with novel self-attention mechanism, that helps the model to use
the context of the sentence more efficiently. Such models could take into
account both left and right context of the sequence as it is implemented in
BERT model [4] and only left context like in GPT model [14]. Transformer-based
LM have own disadvantages, one of them is a limited receptive field. It means
that the transformer could only proceed sequences that have limited length,
while recurrent neural networks could work with unlimited sequences. This
issue partially solved by the Transformer-XL model [3], which could work with
continuous sequences of texts like recurrent neural networks. Despite of this
disadvantage, models like GPT with large receptive field and trained on large
amount of data are capable of capturing long range dependencies.

####

For this work, we propose method that uses both extractive and abstractive
approaches for summarization task. Our work improves previous approach [17],
by using pre-trained LM instead of training it from scratch. In this research
work, we used arxiv dataset as an great example of long scientific documents
and in order to compare our work with previous approaches. We split training
process in two steps. First, we train extractive model as a classification
task, that simply selects which sentences should be selected into summary.
Second, we use extracted summary together with different article sections as
conditioning for generating abstractive summary. Adding extractive summary
into conditioning part is crucial, for generating target summary. Also, we
made experiments with different variants of conditioning and found the best
combination for it. According to our experiments, extracted summary +
introduction and conclusion of the paper performs the best.

####

The contributions of this work are:

  * •

We show that combination of extractive and abstractive approaches improves
quality of produced summary, while using pre-trained transformer-based
language models

  * •

As a result of applying proposed model, improved ROUGE metric on arxiv dataset

##  2 Related Work

Automatic summarization is the process of shortening a set of data
computationally, to create a summary that represents the most important or
relevant information within the original content. Summarization is one of the
main tasks in Natural Language understanding. There are two main approaches
for creating a summary. Extractive summarization aims at identifying the
information that is then extracted and grouped together to form an extractive
summary. It means that, while we train a machine learning model, it solves the
classification task. Which information should be included in the summary.
Abstractive summary generation rewrites the entire document by building
internal semantic representation, and then a summary is created using natural
language processing. As an example of the ML model, it receives a source
document and generates a summary word by word. Without copying text from
source document. The first summarization systems were using extractive
techniques, as it is much easier than abstractive. (Hsu et al. 2018) [7]
combined both extractive and abstractive approaches for creating summaries.
Authors proposed a unified model, that combines sentence-level and word-level
attention to take advantage of two approaches. Also, the authors created a
novel inconsistency loss function in order to be sure that the proposed model
is mutually beneficial to both extractive and abstractive summarization.
Another attempt of using both extractive and abstractive models together was
proposed in [1] paper. The authors proposed a model that uses BERT to perform
extractive summarization with the LSTM Pointer network that selects which
sentences should be included in the summary. Then, the extracted summary is
fed into attention based seq2seq with the copying mechanism for handling out-
of-vocabulary (OOV) words. This part of the model is responsible for
paraphrasing of the extracted summary and producing an abstractive summary.
They used reinforcement learning technique to maximize the ROUGE metric
directly instead of maximizing log-likelihood for the next word given the
previous ground truth words. Both extractive and abstractive models are
trained end2end. In work [9] authors took pre-trained BERT model and first
trained it on extractive task using CNN/Daily Mail datasets. Then, they
continue training the same model but doing abstractive
summarization.(Subramanian et al. 2019) [17] used arxiv dataset for
summarizing long scientific texts and uses the similar approach to ours. Their
model consists of two parts: first is a hierarchical representation of the
document that points out to sentences in the document in order to construct an
extractive summary, second is a transformer language model that conditions on
the extractive summary, and some part of the paper text as well.

##  3 Proposed model

Our model consist of two components. 1) Extractive model classifier that
choose which sentences from source text should be included in summary. 2)
Abstractive model that uses condition text to produce abstractive summary.
During research work, we propose a model, that combines advantages of
extractive and abstractive approaches for creating summaries. The proposed
model consists of two parts: the first part is the extractive summarization
model, second is the abstractive model. A similar approach was used in [17]]
paper. The difference is that the extractive and abstractive models are
replaced by pre-trained transformer-based LMs.The motivation for this is that
pre-trained language model in most cases shows a significant increase in
performance. In the previous approach, the transformer model was used as the
abstractive model. Instead of it, autoregressive pre-trained LM was used.
During research work, a list of experiments for a conditional generation was
performed using an extractive summary to generate abstractive summaries. The
training is performed in two steps. First, the extractive model is trained.
Second, training of the abstractive model that uses extractive summary to
produce better results

###  3.1 Dataset description

As it was already mentioned, the dataset from arxiv.org was used for training.
This dataset contains long sequences of texts from the scientific domain, with
ground truth summaries. The structure of the dataset contains such information
as: “article id”, “abstract text”, “article text”, “labels”, “section name”,
“sections”. The field “article text” holds full text of a paper, “section
name” is a list of paper’s sections, “sections” contains full text of paper
divided into sections. That means, we could identify which text is from the
introduction section, which is from the abstract section or conclusion. This
advantage was used in experiments. The dataset statistics are described in
table 1.

Table 1: Arxiv dataset statistics. num. of docs |  avg. doc. length (words) |  avg. summary length (words)
---|---|---
215.00 |  4938 |  220

####

Both for validation and test parts we took 5 percent of documents. During the
preprocessing step, too long and too short papers were removed, also papers
without abstracts, and text of paper were removed. Also, we replaced some
LaTeX markup with special tokens such as: [math], [graph], [table], [equation]
in order to help model recognize special tokens, other LaTeX source code we
cleared. Also, we removed all irrelevant chars and exclude all not Latin
letters. This preprocessing pipeline was applied for both extractive and
abstractive tasks. We used common approach for creating dataset for training
extractive model.

####

First, we create list of sentence pairs, ”abstract sentence” $->$ ”sentence
from paper”, every sentence from abstract are matched with every sentence from
paper text. Between every pair, we compute the ROUGE metric. In particular, we
compute: ROUGE-1, ROUGE-2, ROUGE-L, and take the average value of f-score.
After that, we got a scoring value for each pair of sentences, the higher the
better. We choose two pairs with the highest scores, this is our positive
examples. Then we randomly sample two sentences from the paper text and mark
such pairs as negative examples. After completing these steps, we got a
dataset that contains a list of sentence pairs and labels. Then, we save the
dataset for extractive summarization in a separate file.

####

For the abstractive summarization task, we took our best model that solves
extractive summarization tasks and infer it on the dataset, in other words, we
generate summaries for each paper in the dataset with our best extractive
model. Then we make all necessary preprocessing and save the model generated
summaries with corresponding papers and abstracts. These are all steps, that
were done with the dataset.

###  3.2 Extractive model

First, we make experiments with extractive models. We used three architectures
for extractive summarization models: BERT [4], RoBERTa [10], and ELECTRA [2].
ELECTRA has BERT’s architecture but the pre-training phase is different. In
the original paper, the authors proposed a new scheme with two models: the
generator and language model that are trained jointly. Generator process
sentences and corrupts some of the tokens in it. Corruption denotes the
process of replacing tokens to some other tokens that could fit into the
context of the sentence. In turn, the language model predicts which tokens
were replaced by the generator. This scheme of pre-training improves the
performance of the language model and reduces the number of training
iterations, computing resources. All used extractive models were pre-trained
on a large corpus of texts and we took “base” variants, we did not check
“large” variants because of long training and hardware constraints. It is
highly probable that “large” variants could show even better results. For each
of the extractive models, we construct special input of tokens, equation 1.

$input_{ext}=concat(token_{cls},seq_{gt},token_{cls},seq_{candidate})$ (1)

Where, $x_{ext}$ denotes input to extractive model, $concat()$ denotes
concatenation function, $token_{cls}$ denotes special classification token
that was originaly used in BERT model, $seq_{gt}$ denotes sentence from ground
truth summary, $seq_{candidate}$ corresponds to candidate sentence.

####

We used Word Piece tokenizer [15] for both BERT and ELECTRA models as in
original papers, with vocabulary size 30525. RoBERTa uses BPE [16] tokenizer
in original implementation and does not have [CLS] token in its vocabulary,
that’s why we manually add it before training. The key benefit of using
subword tokenizers is a small dataset size and decreasing out-of-vocabulary
cases. Also, special tokens were added: [math], [graph], [table], [equation].
That were extracted by regular expressions while pre-processing.

####

All extractive models were trained using cross-entropy loss as an optimization
function. We used the learning rate of 1e-5, model input 512, batch size 24.
Also, we applied gradient clipping [12] technique to make the training process
more stable. We applied distributed training using two GPUs. The training
process took approximately 3-4 days. The architecture of the proposed models
is described in Figure 1, below. The scheme remains the same only input to
model changes and pre-trained LM replaced by one of BERT, ELECTRA, RoBERTa.

Figure 1: Architecture of extractive model

####

Classification head in the figure above denotes a block that contains a stack
of linear layers with an output of size 1 with sigmoid activation. We used
ROUGE metrics for evaluation quality of summaries. To evaluate the
summarization model, we firstly inference it on the test set. We make a list
of pairs: sentence from an abstract sentence from paper and score every pair
with the model. Then we use only candidates that have the highest score. After
this process, we got the extractive summary that we use with ground truth
abstract to calculate ROUGE scores. All scores are averaged between papers. We
used ROUGE-1, ROUGE-2, and ROUGE-L as our main evaluation metric for
summaries.

####

All proposed models used the same set of training hyperparameters and other
settings were the same. From the table above we could conclude that the BERT
model achieves the highest result among all other models. With equal set of
hyperparameters BERT model achieves better results. In the table above, Oracle
denotes the ROUGE score between the ground truth abstract and extracted
summary that includes the most relevant sentences from the text of a paper.
The Oracle scores in the table indicate the limit for extractive models to get
the best summary, according to ROUGE metric. In order to get more coherent
text, we perform paraphrasing of extracted summaries. To get paraphrased
sentences we apply the back-translation technique. For this, we used pre-
trained transformer LM that was trained to translate sentences from english to
german and backward. First, we translate the extractive summary into the
German language using a pre-trained transformer LM and then back to the
English language. Those paraphrased summaries are used later during
experiments with condition model. The results of extractive models are
presented in table 2.

Table 2: Extractrive model results on arxiv dataset. Ext denotes extractive. Model |  Type |  Rouge-1 |  Rouge-2 |  Rouge-L
---|---|---|---|---
BERT |  Ext |  f1: 45.4 |  f1: 20.9 |  f1: 33.7
RoBERTa |  Ext |  f1: 44.3 |  f1: 20.1 |  f1: 32.4
ELECTRA |  Ext |  f1: 41.8 |  f1: 17 |  f1: 29.3

###  3.3 Abstractive model

####

After finishing experiments with extractive summarization, we made experiments
with abstractive approaches. For abstractive summarization, we used pre-
trained autoregressive language models: GPT-2 and BART [8]. We also made
experiments with conditional generation for both models. During conditional
generation, we give to the model some context, according to which it generates
text. We used a condition summary that was extracted by the best extractive
model, table 2. Also, we made conditioning on paraphrased summaries. We used
an input size of 1024, GPT-base variant. The input is defined by equation 2.

$input_{abs}=concat(t_{c},t_{s})*mask_{segment}$ (2)

Where we concatenate $t_{c}$ conditioning text with target summary $t_{s}$ and
apply a segment mask $mask_{segment}$ that identifies which part of the input
is condition and which is target text.

####

The experiments with the BART model were performed in a similar way. But there
are some differences in model input. BART consists of two parts, the first one
is an encoder which uses BERT-like architecture. The second part is a decoder
that consists of stacked transformer decoders, similar architecture to GPT.
These two parts are connected so that encoder output is fed into the decoder.
In such architecture decoder uses hidden states that were produced by encoder.
Encoder and decoder are trained end to end. It means that during backward
pass, we update weights of decoder and encoder. In such a scenario, we fed the
conditioning part into BART’s encoder and target text into the decoder during
the training process. We propose different conditioning scenarios. We made
conditioning: on the extractive summary, on the introduction of paper,
introduction concatenated with conclusion and introduction concatenated with
the extractive summary and with the conclusion. We made the assumption that
both the introduction and conclusion concentrate the most valuable information
for generating summaries. Because usually in introdiction author describes the
problem itself, some details about the proposed method, novelty. In
conclusion, authors usually make some recap of what was done, conclusion of
results. We did not apply conditioning on long texts to GPT-2, because of
restrictions in the input size. The condition part plus target text in most
cases will not fit into 1024 input size, that’s why we only made conditioning
on summaries extracted by models. The experiments results with GPT-2 and BART
are presented in table 3, below.

Table 3: Proposed model results on arxiv dataset Model |  Type |  Rouge-1 |  Rouge-2 |  Rouge-L
---|---|---|---|---
Our approaches
BERT |  Ext |  f1: 45.4 |  f1: 20.9 |  f1: 33.7
RoBERTa |  Ext |  f1: 44.3 |  f1: 20.1 |  f1: 32.4
ELECTRA |  Ext |  f1: 41.8 |  f1: 17 |  f1: 29.3
BART conditioned on extractive summary |  Mix |  f1: 42.5 |  f1: 24.6 |  f1: 29.2
BART conditioned on introduction |  Mix |  f1: 40.3 |  f1: 21.9 |  f1: 27.3
BART conditioned on introduction with conclusion |  Mix |  f1: 41.6 |  f1: 23.7 |  f1: 28.1
BART conditioned on introduction + extractive summary + conclusion |  Mix |  f1: 45.3 |  f1: 25.1 |  f1: 36.2
GPT2 with condition on extractive summary |  Mix |  f1: 40.1 |  f1: 15.8 |  f1: 27.7
GPT2 with condition on extractive paraphrased summary |  Mix |  f1: 17.9 |  f1: 3.5 |  f1: 12.2
GPT2 w/o conditioning |  Abs |  f1: 17.2 |  f1: 10.2 |  f1: 13.1
Previous approaches
(Subramanian S. et al. (2019)) |  Mix |  f1: 42.43 |  f1: 15.24 |  f1: 24.08
(Cohan A. et al. (2018)) |  Mix |  f1: 35.80 |  f1: 11.05 |  f1: 31.8
Gold extractive
Gold extractive |  Oracle |  f1: 47.3 |  f1: 23.17 |  f1: 36

####

From the table above we could conclude that extractive summary plays a crucial
role in generating abstractive summaries. Removing the extractive summary from
the condition part leads to a decreasing ROUGE score. Also, the assumption
that both introduction and conclusion holds the most relevant information,
confirmed. It’s obvious that extractive summary has a bigger impact than
introduction with conclusion, because extractive summary already holds a lot
of relevant (according to ROUGE score) sentences. Also, we investigate that
ROUGE-2 and ROUGE-L scores outperforms the best model and outperforms Oracle,
because during the abstractive summarization model could produce words that
could not be presented in the source document. The best model that uses BERT
as extractor and BART as abstractor is presented in Fig. 2. First BERT
performs extractive summarization of the article, extracted summary
concatenates with the introduction and conclusion of the paper. This setup
shows the best performance according to the ROUGE metric and outperforms the
previous approach that was applied to arxiv dataset.

Figure 2: Proposed method for generating summaries of long texts

##  4 Conclusion

####

The novel improvements was proposed that uses both extractive and abstractive
approaches, as an extracted model BERT model was used, as an abstractive model
BART model was used. During research work, comparison analysis of different
architectures for extractive and abstractive summarization approaches were
done. Also, we make experiments with conditioning on different parts of the
source document to produce an abstractive summary. Conditioning on extractive
summary, introduction and conclusion of the paper shows the best ROUGE score.
The assumption about conditioning on paraphrased summary failed, during
paraphrasing source sentence is changed so that it becomes irrelevant to
ground truth summary. Evaluation results, that were obtained during the
research, outperforms previously applied algorithms for arxiv dataset in all
ROUGE-1, ROUGE-2 and ROUGE-L metrics 45.3, 25.1 and 36.2 correspondingly.
Also, experiments showed that using the advantages of extractive and
abstractive approaches improves the quality of the produced summary,
extractive summarization plays a crucial part in generating abstractive
summaries.

####

As a future improvement of the proposed architecture, end-to-end learning
could be applied using both extractor and abstractor. This feature potentially
could improve the quality of abstractive summaries. Also, the proposed
architecture could be tested for other summarization datasets.

## References

  * [1] Bae, S., Kim, T., Kim, J., Lee, S.g.: Summary level training of sentence rewriting for abstractive summarization. arXiv preprint arXiv:1909.08752 (2019)
  * [2] Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555 (2020)
  * [3] Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q.V., Salakhutdinov, R.: Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860 (2019)
  * [4] Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
  * [5] Gal, Y., Ghahramani, Z.: A theoretically grounded application of dropout in recurrent neural networks. In: Advances in neural information processing systems. pp. 1019–1027 (2016)
  * [6] Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural computation 9(8), 1735–1780 (1997)
  * [7] Hsu, W.T., Lin, C.K., Lee, M.Y., Min, K., Tang, J., Sun, M.: A unified model for extractive and abstractive summarization using inconsistency loss. arXiv preprint arXiv:1805.06266 (2018)
  * [8] Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., Zettlemoyer, L.: Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)
  * [9] Liu, Y., Lapata, M.: Text summarization with pretrained encoders. arXiv preprint arXiv:1908.08345 (2019)
  * [10] Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
  * [11] Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems. pp. 3111–3119 (2013)
  * [12] Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: International conference on machine learning. pp. 1310–1318 (2013)
  * [13] Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettlemoyer, L.: Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018)
  * [14] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
  * [15] Schuster, M., Nakajima, K.: Japanese and korean voice search. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 5149–5152. IEEE (2012)
  * [16] Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909 (2015)
  * [17] Subramanian, S., Li, R., Pilault, J., Pal, C.: On extractive and abstractive neural document summarization with transformer language models. arXiv preprint arXiv:1909.03186 (2019)
  * [18] Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in neural information processing systems. pp. 3104–3112 (2014)
  * [19] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in neural information processing systems. pp. 5998–6008 (2017)


# Semi-insulating 4H-SiC lateral bulk acoustic wave resonators

B. Jiang<EMAIL_ADDRESS>N. Opondo  S. Bhave  OxideMEMS Lab, Purdue
University, West Lafayette, Indiana 47907, USA

###### Abstract

Silicon carbide (SiC) excels in its outstanding mechanical properties, which
are widely studied in Microelectromechanical systems (MEMS). Recently, the
mechanical tuning of color centers in 4H-SiC has been demonstrated, broadening
its application in quantum spintronics. The strain generated in a mechanical
resonator can be used to manipulate the quantum state of the color center
qubit. This work reports a lateral overtone mechanical resonator fabricated
from a semi-insulating (SI) bulk 4H-SiC wafer. An aluminum nitride (AlN)
piezoelectric transducer on SiC is used to drive the resonance. The resonator
shows a series of modes with quality factors (Q) above 3000. An acoustic
reflector positioned at the anchor shows a 22% improvement in the Q at 300 MHz
resonance and suppresses the overtone modes away from it. This monolithic SiC
resonator allows optical access to the SiC color centers from both sides of
the wafer, enabling convenient setup in quantum measurements.

††preprint: AIP/123-QED

Silicon carbide (SiC) is known for its hardness, elasticity at high
temperatures, high thermal conductivity, and chemical inertness, making it
suitable for the next generation MEMS components. Recently, the color centers
in SiC, such as divacancies and silicon vacancies, have been demonstrated as
viable platforms for quantum sensing and computation Son  _et al._ (2020). In
particular, researchers have been able to use surface acoustic wave (SAW) in
SiC resonators to control the spin states inside the acoustic cavity Whiteley
_et al._ (2019). MEMS researchers have usually used SiC-on-Insulator (SiCOI)
technology, which involves epitaxially grown SiC on an oxide layer. For
example, the 3C-SiC Lateral Overtone Bulk Acoustic Wave Resonator (LOBAR)
devices developed by Gong et al.Gong, Kuo, and Piazza (2012) show outstanding
resonance performance and are easy to fabricate at wafer scale. However, SI
4H-SiC, a polytype that cannot be grown epitaxially, stands out as a unique
candidate for its optical properties Falk  _et al._ (2013) and its potential
as the host for single quantum emitters Castelletto  _et al._ (2014).

Recently, Lekavicius et al. introduced a diamond lamb wave resonator
Lekavicius, Oo, and Wang (2019) and created bright NV centers in it, a similar
system to color centers in SiC. However, their devices are not integrated with
a piezoelectric transducer, and therefore cannot be mechanically actuated. We
have previously fabricated cantilever resonators made from SI 4H-SiC Jiang
_et al._ (2019) resonating near 10 MHz and showed that the process preserves
the fluorescent quality of the divacancies. The color center spin’s mechanical
Rabi frequency increases linearly with the frequency of the mechanical
resonance MacQuarrie  _et al._ (2015), so higher-frequency resonators are
desired for fast quantum control. In this work, we report the development of
300 MHz acoustic wave resonators actuated by AlN piezoelectric layer on our
SiC platform.

A 3D illustration for the device we made is shown in FIG. 1. We design the
LOBAR devices based on a $200\text{\,}\mathrm{\SIUnitSymbolMicro m}$ thick
4H-SiC wafer. The LOBAR has an AlN transducer in the center, which has
electrically alternating inter digitated transducer (IDT) fingers generating a
laterally vibrating acoustic wave. The LOBAR generates overtones modes in the
frequency spectrum, because of the SiC wings extending beyond the IDT-covered
area on both sides. We also place an acoustic reflector Harrington and
Abdolvand (2011), an arc-shaped trench through the bulk wafer, centering at
the LOBAR’s anchor point. The leaking acoustic wave from the anchor incident
on the trench will be reflected back into the resonator body, thus reducing
the acoustic energy loss. For a reflector positioned at $R_{ref}$ from the
tether, the total wave propagating length is $2R_{ref}$. When
$R_{ref}=\lambda/2+n\cdot\lambda(n=0,1,2,\ldots$ and $\lambda$ is the acoustic
wavelength), the highest Qs are achieved due to interference.

For quantum applications, when a laser is introduced to excite the color
center spins in the SiC, our device allows laser access to both surfaces of
the SiC LOBAR, illustrated in FIG. 1. Compared with other MEMS resonators on
SiC and diamond Whiteley  _et al._ (2019); MacQuarrie  _et al._ (2015); Maity
_et al._ (2020); Chen  _et al._ (2019), which are unreleased and allow only
optical access from one side, our resonators would extend the sensing
flexibility of spins. Optical access on one side will allow the sensing
proximity to a measurand on the other, which is important in nanoscale sensing
Thiel  _et al._ (2019).

Figure 1:  1/4th of the LOBAR resonator connected to the anchor area. The
white lines indicate the mechanical wave leaking from the anchor, met by the
reflector. The laser used for quantum measurement can be applied from both the
upper and the lower surface of the LOBAR’s extending wings.

To find the appropriate design parameters, we performed a 2D COMSOL
Multiphysics modal analysis based on the design shown in FIG. 2. Because of
the simulation and fabrication limitations, we set the LOBAR length as
$1600\text{\,}\mathrm{\SIUnitSymbolMicro m}$, width as
$300\text{\,}\mathrm{\SIUnitSymbolMicro m}$, and the thickness as
$10\text{\,}\mathrm{\SIUnitSymbolMicro m}$, while $R_{tether}$ as
$90\text{\,}\mathrm{\SIUnitSymbolMicro m}$ and $R_{ref}$ as
$60\text{\,}\mathrm{\SIUnitSymbolMicro m}$. The distance between 2 IDT fingers
is $20\text{\,}\mathrm{\SIUnitSymbolMicro m}$, leading to a resonance
frequency around 300 MHz.

Figure 2:  Mode shape of the LOBAR device, shown as the displacement in COMSOL
simulation. The IDT fingers in the center region create the alternative
mechanical vibration to both sides.

We fabricate the SiC LOBAR devices from a
$500\text{\,}\mathrm{\SIUnitSymbolMicro m}$ thick SI 4H-SiC wafer purchased
from Cree Inc, with resistivity larger than
$1\text{\times}{10}^{6}\text{\,}\mathrm{\SIUnitSymbolOhm}\text{${\cdot}$}\mathrm{cm}$.
The wafer is thinned down to $200\text{\,}\mathrm{\SIUnitSymbolMicro m}$ by
chemical mechanical polishing (CMP). The detailed process after thinning is
shown in FIG. 3(a): (1)-(3) a piezoelectric stack consisting of
$1\text{\,}\mathrm{\SIUnitSymbolMicro m}$ AlN sandwiched between 100 nm
Molybdenum (Mo) layer is deposited by OEM Group Inc. Optical lithography and
dry etch are performed to define the piezoelectric transducers. (4) The key
fabrication step involves a dual-mask followed by front-backside etch,
enabling a wafer-scale dry release of the LOBARs. For the hardmask of the SiC
etch, we deposit $4\text{\,}\mathrm{\SIUnitSymbolMicro m}$ thick Nickel (Ni)
on the backside Jiang  _et al._ (2019). At the same time, we use electron beam
evaporation to form an Indium Tin Oxide (ITO) thin layer (600 nm) as the etch
mask for the frontside of SiC, since it can be dissolved by wet etch without
damaging the metal electrodes. (5) The ITO layer is patterned by photoresist,
then etched by Cl2 based plasma, to define the release window for LOBARs. (6)
From the backside, the SiC is dry etched with SF6 and Ar plasma. The etch rate
is about $15\text{\,}\mathrm{\SIUnitSymbolMicro m}$ per hour. The etch is
stopped when it reaches $190\text{\,}\mathrm{\SIUnitSymbolMicro m}$, leaving a
$10\text{\,}\mathrm{\SIUnitSymbolMicro m}$ SiC membrane. (7) Turning over the
wafer, we perform another dry etch on the frontside for
$10\text{\,}\mathrm{\SIUnitSymbolMicro m}$ until the LOBAR device is released.
(8) Finally, we dissolve the ITO with 1:5=HCl:H2O diluted hydrochloric acid
and leave the backside Ni mask on as it does not affect our measurement. The
acid does not attack the AlN transducer stack. Concurrently, a through wafer
$200\text{\,}\mathrm{\SIUnitSymbolMicro m}$ acoustic reflector is defined
using the frontside $10\text{\,}\mathrm{\SIUnitSymbolMicro m}$ and
$190\text{\,}\mathrm{\SIUnitSymbolMicro m}$ deep etches, shown in FIG. 3(b).

Figure 3:  (a) Fabrication process of the SiC LOBAR resonator. The cross-
section is along the longitudinal direction of the LOBAR. (b) Fabrication of
the reflectors. The figures correspond to step (6) and (7) in (a). The cross-
section is along the transverse direction of the LOBAR. Compared with the
previous acoustic reflectors Harrington and Abdolvand (2011); Mansoorzare  _et
al._ (2018) which were only defined by frontside etch, our front-backside etch
allows a reflector through the whole wafer thickness. It may be used in future
developments in anchor loss reduction.

The final devices are inspected with Hitachi S-4800 Field Emission SEM. The
images are shown in FIG. 4. The SiC LOBAR is released using our front-backside
etch. One challenge in the fabrication is that the main LOBAR release window
is much larger than the reflector, resulting in different etch rates in these
two regions. When the backside etch reaches
$190\text{\,}\mathrm{\SIUnitSymbolMicro m}$ in the LOBAR area, the reflector
etch on the backside is shallower. We overcome the issue by an over-etch on
the frontside with an ITO hardmask. ITO shows selectivity of 1:50 over SiC,
and survives the extensive etch on the frontside. FIG. 4(b) shows the
piezoelectric transducer defined on top of SiC LOBAR. Even though the dry etch
of AlN with BCl3 stops on the SiC layer, it leaves a rough surface due to
heavy Cl^- ion bombardment.

Figure 4:  (a) Backside of the LOBAR. The reflectors go through the whole
wafer and have openings on both sides of the wafer. The LOBAR is
$1600\text{\,}\mathrm{\SIUnitSymbolMicro m}$ long and
$300\text{\,}\mathrm{\SIUnitSymbolMicro m}$ wide. (b) Frontside of a released
LOBAR. The enlarged picture shows SiC surface damage during the AlN etch.

The resonators are characterized on a Cascade Microtech 11971B Probe Station
with a Keysight N5230A PNA-L Network Analyzer and 2 Cascade Microtech ACP40
GSG probes at room temperature and under atmosphere conditions. In FIG. 5, we
compare the transmission measurements of LOBARs with and without the
reflectors. The Qs are calculated with the Lorentzian peak fitting method. The
piezoelectric transducer couples to multiple overtones in the resonator. We
define the target mode of the LOBAR as the mode with maximum coupling with the
transducer. We realize a 22% improvement on Q in the target LOBAR mode with
the reflector. In FIG. 5(c), the LOBAR with the reflector has increased Qs
around 300 MHz, while the Qs of other overtone modes quickly decline and
vanish below the measurement noise floor outside a 100 MHz wide range.
Meanwhile, the LOBAR without the reflector displays lower Qs across a wider
frequency spectrum.

As it is discussed in FIG. 1, for a designed
$R_{ref}$=$60\text{\,}\mathrm{\SIUnitSymbolMicro m}$, the Q is maximized when
the wavelength $\lambda$=$40\text{\,}\mathrm{\SIUnitSymbolMicro m}$ and n=1.
With an acoustic velocity of 12500 m/s Chinone  _et al._ (1989), this
wavelength corresponds to a frequency of 300 MHz in 4H SiC. Other modes are
affected by the miss-matched interference and suffer a Q loss. A LOBAR has
multiple modes with similar Qs and piezoelectric coupling. Appropriate design
and positioning of the acoustic reflector enhance the target mode and
attenuate the adjacent modes’ Qs. This Q boost will enhance the strain-spin
coupling in the color center applications in SiC. Our design will also
simplify spur-free RF LOBAR oscillators Kourani, Lu, and Gong (2020).

The SI 4H-SiC resonators’ Qs are similar to the previous 3C-SiC LOBARs Gong,
Kuo, and Piazza (2012) and the SCHBAR in diamond Chen  _et al._ (2019), but
still lower than the LOBARs made by Ziaei-Moayyed et al.Ziaei-Moayyed  _et
al._ (2011). Future fabrication improvement to increase the Q will include
reducing the surface roughness in AlN etch on the frontside and reducing the
dimples, and improving the etch depth uniformity on backside and increasing
the sidewall smoothness.

Figure 5:  2-port transmission measurements of: (a) LOBAR without the
reflector. (b) LOBAR with the reflector, $R_{ref}=1.5\lambda$. The reflector
is frequency-sensitive, suppressing the resonances away from 300 MHz. (c) Q
measured in both devices’ overtone modes. The circled two modes are the marked
ones and shown in figure insets in (a) and (b) respectively.

In conclusion, we have introduced a monolithic high-frequency lateral bulk
acoustic wave resonator based on SI 4H-SiC, and improved its mechanical
performance by a reflector design. Front-backside etch process provides unique
flexibility to produce other types of resonators Milanovic, Last, and Pister
(2003); Hamelin  _et al._ (2019); Gokhale  _et al._ (2020); Lukin, Guidry, and
Vučković (2020). In the future, we shall explore the mechanical coupling to
the color centers using our devices, and study the effect of a laterally
vibrating acoustic wave on the quantum system. Furthermore, since no other
substrate material is needed and the optical accessibility is on both sides,
we can combine this process with the development of 2D material on SiC
Borysiuk  _et al._ (2014) and study the acoustic effect on the interface
between the two materials.

This work was conducted at Birck Nanotechnology Center in Purdue University.
The research was supported by NSF RAISE-TAQS Award #1839164. The CMP was
completed at NOVASiC, and the AlN deposition was performed by Plasma-Therm.

The data that support the findings of this study are openly available in
Zenodo at https://doi.org/10.5281/zenodo.4568410.

## References

  * Son  _et al._ (2020) N. T. Son, C. P. Anderson, A. Bourassa, K. C. Miao, C. Babin, M. Widmann, M. Niethammer, J. Ul Hassan, N. Morioka, I. G. Ivanov, F. Kaiser, J. Wrachtrup, and D. D. Awschalom, “Developing silicon carbide for quantum spintronics,” Applied Physics Letters 116, 190501 (2020).
  * Whiteley  _et al._ (2019) S. J. Whiteley, G. Wolfowicz, C. P. Anderson, A. Bourassa, H. Ma, M. Ye, G. Koolstra, K. J. Satzinger, M. V. Holt, F. J. Heremans, A. N. Cleland, D. I. Schuster, G. Galli, and D. D. Awschalom, “Spin–phonon interactions in silicon carbide addressed by Gaussian acoustics,” Nature Physics 15, 490–495 (2019).
  * Gong, Kuo, and Piazza (2012) S. Gong, N. Kuo, and G. Piazza, “GHz High-Q Lateral Overmoded Bulk Acoustic-Wave Resonators Using Epitaxial SiC Thin Film,” Journal of Microelectromechanical Systems 21, 253–255 (2012).
  * Falk  _et al._ (2013) A. L. Falk, B. B. Buckley, G. Calusine, W. F. Koehl, V. V. Dobrovitski, A. Politi, C. A. Zorman, P. X.-L. Feng, and D. D. Awschalom, “Polytype control of spin qubits in silicon carbide,” Nature Communications 4, 1819 (2013).
  * Castelletto  _et al._ (2014) S. Castelletto, B. C. Johnson, V. Ivády, N. Stavrias, T. Umeda, A. Gali, and T. Ohshima, “A silicon carbide room-temperature single-photon source,” Nature Materials 13, 151–156 (2014).
  * Lekavicius, Oo, and Wang (2019) I. Lekavicius, T. Oo, and H. Wang, “Diamond Lamb wave spin-mechanical resonators with optically coherent nitrogen vacancy centers,” Journal of Applied Physics 126 (2019), 10.1063/1.5124307.
  * Jiang  _et al._ (2019) B. Jiang, N. Opondo, G. Wolfowicz, P. Yu, D. D. Awschalom, and S. A. Bhave, “SiC Cantilevers for Generating Uniaxial Stress,” in  _2019 20th International Conference on Solid-State Sensors, Actuators and Microsystems Eurosensors XXXIII (TRANSDUCERS EUROSENSORS XXXIII)_ (2019) pp. 1655–1658.
  * MacQuarrie  _et al._ (2015) E. R. MacQuarrie, T. A. Gosavi, A. M. Moehle, N. R. Jungwirth, S. A. Bhave, and G. D. Fuchs, “Coherent control of a nitrogen-vacancy center spin ensemble with a diamond mechanical resonator,” Optica 2, 233–238 (2015).
  * Harrington and Abdolvand (2011) B. P. Harrington and R. Abdolvand, “In-plane acoustic reflectors for reducing effective anchor loss in lateral-extensional MEMS resonators,” Journal of Micromechanics and Microengineering 21, 85021 (2011).
  * Maity  _et al._ (2020) S. Maity, L. Shao, S. Bogdanović, S. Meesala, Y.-I. Sohn, N. Sinclair, B. Pingault, M. Chalupnik, C. Chia, L. Zheng, K. Lai, and M. Lončar, “Coherent acoustic control of a single silicon vacancy spin in diamond,” Nature Communications 11, 193 (2020).
  * Chen  _et al._ (2019) H. Chen, N. F. Opondo, B. Jiang, E. R. Macquarrie, R. S. Daveau, S. A. Bhave, and G. D. Fuchs, “Engineering Electron-Phonon Coupling of Quantum Defects to a Semiconfocal Acoustic Resonator,” Nano Letters 19, 7021–7027 (2019).
  * Thiel  _et al._ (2019) L. Thiel, Z. Wang, M. A. Tschudin, D. Rohner, I. Gutiérrez-Lezama, N. Ubrig, M. Gibertini, E. Giannini, A. F. Morpurgo, and P. Maletinsky, “Probing magnetism in 2D materials at the nanoscale with single-spin microscopy,” Science 364, 973–976 (2019).
  * Mansoorzare  _et al._ (2018) H. Mansoorzare, S. Moradian, S. Shahraini, R. Abdolvand, and J. Gonzales, “Achieving the Intrinsic Limit of Quality Factor in VHF Extensional-Mode Block Resonators,” in  _2018 IEEE International Frequency Control Symposium (IFCS)_ (2018) pp. 1–4.
  * Chinone  _et al._ (1989) Y. Chinone, S. Ezaki, F. Fujita, and K. Matsumoto, “Applications of High Purity SiC Prepared by Chemical Vapor Deposition BT - Amorphous and Crystalline Silicon Carbide II,” (Springer Berlin Heidelberg, Berlin, Heidelberg, 1989) pp. 198–206.
  * Kourani, Lu, and Gong (2020) A. Kourani, R. Lu, and S. Gong, “A Wideband Oscillator Exploiting Multiple Resonances in Lithium Niobate MEMS Resonator,” IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 67, 1854–1866 (2020).
  * Ziaei-Moayyed  _et al._ (2011) M. Ziaei-Moayyed, S. D. Habermehl, D. W. Branch, P. J. Clews, and R. H. Olsson, “Silicon carbide lateral overtone bulk acoustic resonator with ultrahigh quality factor,” Proceedings of the IEEE International Conference on Micro Electro Mechanical Systems (MEMS) , 788–792 (2011).
  * Milanovic, Last, and Pister (2003) V. Milanovic, M. Last, and K. S. J. Pister, “Laterally actuated torsional micromirrors for large static deflection,” IEEE Photonics Technology Letters 15, 245–247 (2003).
  * Hamelin  _et al._ (2019) B. Hamelin, J. Yang, A. Daruwalla, H. Wen, and F. Ayazi, “Monocrystalline Silicon Carbide Disk Resonators on Phononic Crystals with Ultra-Low Dissipation Bulk Acoustic Wave Modes,” Scientific Reports 9, 18698 (2019).
  * Gokhale  _et al._ (2020) V. J. Gokhale, B. P. Downey, D. S. Katzer, N. Nepal, A. C. Lang, R. M. Stroud, and D. J. Meyer, “Epitaxial bulk acoustic wave resonators as highly coherent multi-phonon sources for quantum acoustodynamics,” Nature Communications 11, 2314 (2020).
  * Lukin, Guidry, and Vučković (2020) D. M. Lukin, M. A. Guidry, and J. Vučković, “Integrated Quantum Photonics with Silicon Carbide: Challenges and Prospects,” PRX Quantum 1, 20102 (2020).
  * Borysiuk  _et al._ (2014) J. Borysiuk, J. Sołtys, J. Piechota, S. Krukowski, J. M. Baranowski, and R. Stȩpniewski, “Structural defects in epitaxial graphene layers synthesized on C-terminated 4H-SiC (0001¯) surface-Transmission electron microscopy and density functional theory studies,” Journal of Applied Physics 115, 54310 (2014).