# A Deeper Analysis for Improving Identifiability in Transformers

First Author
Affiliation / Address line 1
Affiliation / Address line 2
Affiliation / Address line 3
email@domain
&Second Author
Affiliation / Address line 1
Affiliation / Address line 2
Affiliation / Address line 3
email@domain
Rishabh Bhardwaj1, Navonil Majumder1, Soujanya Poria1, Eduard Hovy2
1 Singapore University of Technology and Design, Singapore
2 Carnegie Mellon University, Pittsburgh, PA, USA
<EMAIL_ADDRESS>
{navonil_majumder<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
# A Deeper Analysis for Improving Identifiability in Transformers without
Compromising their Performance in Diverse Text Classification Tasks

First Author
Affiliation / Address line 1
Affiliation / Address line 2
Affiliation / Address line 3
email@domain
&Second Author
Affiliation / Address line 1
Affiliation / Address line 2
Affiliation / Address line 3
email@domain
Rishabh Bhardwaj1, Navonil Majumder1, Soujanya Poria1, Eduard Hovy2
1 Singapore University of Technology and Design, Singapore
2 Carnegie Mellon University, Pittsburgh, PA, USA
<EMAIL_ADDRESS>
{navonil_majumder<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
# Improving Identifiability in Transformers without
Compromising their Performance in Varied Text Classification Tasks

First Author
Affiliation / Address line 1
Affiliation / Address line 2
Affiliation / Address line 3
email@domain
&Second Author
Affiliation / Address line 1
Affiliation / Address line 2
Affiliation / Address line 3
email@domain
Rishabh Bhardwaj1, Navonil Majumder1, Soujanya Poria1, Eduard Hovy2
1 Singapore University of Technology and Design, Singapore
2 Carnegie Mellon University, Pittsburgh, PA, USA
<EMAIL_ADDRESS>
{navonil_majumder<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
# More Identifiable yet Equally Performant Transformers
for Text Classification

First Author
Affiliation / Address line 1
Affiliation / Address line 2
Affiliation / Address line 3
email@domain
&Second Author
Affiliation / Address line 1
Affiliation / Address line 2
Affiliation / Address line 3
email@domain
Rishabh Bhardwaj1, Navonil Majumder1, Soujanya Poria1, Eduard Hovy2
1 Singapore University of Technology and Design, Singapore
2 Carnegie Mellon University, Pittsburgh, PA, USA
<EMAIL_ADDRESS>
{navonil_majumder<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Interpretability is an important aspect of the trustworthiness of a model’s
predictions. Transformer’s predictions are widely explained by the attention
weights, i.e., a probability distribution generated at its self-attention unit
(head). Current empirical studies provide shreds of evidence that attention
weights are not explanations by proving that they are not unique. A recent
study showed theoretical justifications to this observation by proving the
non-identifiability of attention weights. For a given input to a head and its
output, if the attention weights generated in it are unique, we call the
weights identifiable. In this work, we provide deeper theoretical analysis and
empirical observations on the identifiability of attention weights. Ignored in
the previous works, we find the attention weights are more identifiable than
we currently perceive by uncovering the hidden role of the key vector.
However, the weights are still prone to be non-unique attentions that make
them unfit for interpretation. To tackle this issue, we provide a variant of
the encoder layer that decouples the relationship between key and value vector
and provides identifiable weights up to the desired length of the input. We
prove the applicability of such variations by providing empirical
justifications on varied text classification tasks. The implementations are
available at https://github.com/declare-lab/identifiable-transformers.

##  1 Introduction

Widely adopted Transformer architecture Vaswani et al. (2017) has obviated the
need for sequential processing of the input that is enforced in traditional
Recurrent Neural Networks (RNN). As a result, compared to a single-layered
LSTM or RNN model, a single-layered Transformer model is computationally more
efficient, reflecting in a relatively shorter training time Vaswani et al.
(2017). This advantage encourages the training of deep Transformer-based
language models on large-scale datasets. Their learning on large corpora has
already attained state-of-the-art (SOTA) performances in many downstream
Natural Language Processing (NLP) tasks. A large number of SOTA machine
learning systems even beyond NLP Lu et al. (2019) are inspired by the building
blocks of Transformer that is multi-head self-attention Radford et al. (2018);
Devlin et al. (2018).

A model employing an attention-based mechanism generates a probability
distribution $\mathbf{a}$ = $\\{a_{1},\ldots,a_{n}\\}$ over the $n$ input
units $z$ = $\\{z_{1},\ldots,z_{n}\\}$. The idea is to perform a weighted sum
of inputs, denoted by $\sum_{i=1}^{n}a_{i}z_{i}$, to produce a more context-
involved output. The attention vector, $\mathbf{a}$, are commonly interpreted
as scores signifying the relative importance of input units. However, counter-
intuitively, it is recently observed that the weights generated in the model
do not provide meaningful explanations Jain and Wallace (2019); Wiegreffe and
Pinter (2019).

Attention weights are (structurally) identifiable if we can uniquely determine
them from the output of the attention unit Brunner et al. (2019).
Identifiability of the attention weights is critical to the model’s prediction
to be interpretable and replicable. If the weights are not unique, explanatory
insights from them might be misleading.

The self-attention transforms an input sequence of vectors $z$ =
$\\{z_{1},\ldots,z_{n}\\}$ to a contextualized output sequence $y$ =
$\\{y_{1},\ldots,y_{n}\\}$, where $y_{k}$ = $\sum_{i=1}^{n}a_{(k,i)}\;z_{i}$.
The scalar $a_{(k,i)}$ captures how much of the $i_{\text{th}}$ token
contributes to the contextualization of $k_{\text{th}}$ token. A Transformer
layer consists of multiple heads, where each head performs self-attention
computations, we break the head computations in two phases:

  * •

Phase 1: Calculation of attention weights $a_{(k,i)}$. It involves mapping
input tokens to key and query vectors. The dot product of $k_{\text{th}}$
query vector and $i_{\text{th}}$ key vector gives $a_{(k,i)}$.

  * •

Phase 2: Calculation of a contextualized representation for each token. It
involves mapping input tokens to the value vectors. The contextualized
representation for $k_{\text{th}}$ token can be computed by the weighted
average of the value vectors, where the weight of $i_{\text{th}}$ token is
$a_{(k,i)}$ computed in first phase.

The identifiability in Transformer has been recently studied by Brunner et al.
(2019) which provides theoretical claims that under mild conditions of input
length, attention weights are not unique to the head’s output. Essentially
their proof was dedicated to the analysis of the computations in the second
phase, i.e., token contextualization. However, the theoretical analysis
ignored the crucial first phase where the attention weights are generated.
Intrinsic to their analysis, the attention identifiability can be studied by
studying only the second phase of head computations. However, even if we find
another set of weights from the second phase, it depends on the first phase if
those weights can be generated as the part of key-query multiplication.

In this work, we probe the identifiability of attention weights in Transformer
from a perspective that was ignored in Brunner et al. (2019). We explore the
previously overlooked first phase of self-attention for its contribution to
the identifiability in Transformer. During our analysis of the first phase, we
uncover the critical constraint imposed by the size of the key vector111The
size of key and query vector is expected to be the same due to the subsequent
dot product operation $d_{k}$. The flow of analysis can be described as

  * •

We first show that the attention weights are identifiable for the input
sequence length $d_{s}$ no longer than the size of value vector $d_{v}$
(section 3.1) Brunner et al. (2019)222The sequence length denotes number of
tokens at input..

  * •

For the case when $d_{s}>d_{v}$, we analyse the attention weights as raw dot-
product (logits) and the $\operatorname{softmax}$ed dot-product (probability
simplex), independently. An important theoretical finding is that both
versions are prone to be unidentifiable.

  * •

In the case of attention weights as logits (section 3.2.1), we analytically
construct another set of attention weights to claim the unidentifiability. In
the case of attention weights as $\operatorname{softmax}$ed logits (section
3.2.2), we find the attention identifiability to be highly dependent on
$d_{k}$. Thus, the size of key vector plays an important role in the
identifiability of the self-attention head. The pieces of evidence suggest
that the current analysis in Brunner et al. (2019) ignored the crucial
constraints from the first phase in their analysis.

To resolve the unidentifiability problem, we propose two simple solutions
(section 4). For the regular setting of the Transformer encoder where $d_{v}$
depends on the number of attention heads and token embedding dimension, we
propose to reduce $d_{k}$. This may lead to more identifiable attention
weights. Alternatively, as a more concrete solution, we propose to set $d_{v}$
equal to token embedding dimension while adding head outputs as opposed to the
regular approach of concatenation Vaswani et al. (2017). Embedding dimension
can be tuned according to the sequence length up to which identifiability is
desired. We evaluate the performance of the proposed variants on varied text
classification tasks comprising of ten datasets (section 5).

In this paper, our goal is to provide concrete theoretical analysis,
experimental observations, and possible simple solutions to identifiability of
attention weights in Transformer. The idea behind identifiable variants of the
Transformer is—the harder it is to obtain alternative attention weights, the
likelier is they are identifiable, which is a desirable property of the
architecture. Thus, our contribution are as follows:

  * •

We provide a concrete theoretical analysis of identifiability of attention
weights which was missing in the previous work by Brunner et al. (2019).

  * •

We provide Transformer variants that are identifiable and validate them
empirically by analysing the numerical rank of the attention matrix generated
in the self-attention head of the Transformer encoder. The variants have
strong mathematical support and simple to adopt in the standard Transformer
settings.

  * •

We provide empirical evaluations on varied text classification tasks that show
higher identifiability does not compromise with the task’s performance.

##  2 Background

###  2.1 Identifiability

A general trend in machine learning research is to mathematically model the
input-output relationship from a dataset. This is carried out by
quantitatively estimating the set of model parameters that best fit the data.
The approach warrants prior (to fitting) examination of the following aspects:

  * •

The sufficiency of the informative data to the estimate model parameters,
i.e., practical identifiability. Thus, the limitation comes from the dataset
quality or quantity and may lead to ambiguous data interpretations Raue et al.
(2009).

  * •

The possibility that the structure of the model allows its parameters to be
uniquely estimated, irrespective of the quality or quantity of the available
data. This aspect is called structural identifiability. A model is said to be
structurally unidentifiable if a different set of parameters yield the same
outcome.

In this work, we focus on the structural identifiability Bellman and Åström
(1970). It is noteworthy that the goodness of the fit of a model on the data
does not dictate its structural identifiability. Similar to Brunner et al.
(2019), we focus our analysis on the identifiability of attention weights,
which are not model parameters, yet demands meaningful interpretations and are
crucial to the stability of representations learned by the model.

###  2.2 Transformer Encoder Layer

We base our analysis on the building block of Transformer, i.e., the encoder
layer (Vaswani et al., 2017). The layer has two sub-layers. First sub-layer
performs the multi-head self-attention, and second is feed-forward network.
Given a sequence of tokens $\\{x_{1},\ldots,x_{d_{s}}\\}$, an embedding layer
transforms it to a set of vector
$\\{z_{1},\ldots,z_{d_{s}}\\}\in\operatorname{\mathbb{R}}^{d_{e}}$, where
$d_{e}$ denotes token embedding dimension. To this set, we add vectors
encoding positional information of tokens
$\\{p_{1},\ldots,p_{d_{s}}\\}\in\operatorname{\mathbb{R}}^{d_{e}}$.

##### Multi-head Attention.

Input to a head of multi-head self-attention module is W
$\in\operatorname{\mathbb{R}}^{d_{s}\times d_{e}}$, i.e., a sequence of
$d_{s}$ tokens lying in a $d_{e}$-dimensional embedding space. Tokens are
projected to $d_{q}$-size query, $d_{k}$-size key, and $d_{v}$-size value
vectors using linear layers, resulting in the respective matrices - Query
$\operatorname{\textbf{Q}}\in\operatorname{\mathbb{R}}^{d_{s}\times d_{q}}$,
Key $\operatorname{\textbf{K}}\in\operatorname{\mathbb{R}}^{d_{s}\times
d_{k}}$, and Value
$\operatorname{\textbf{V}}\in\operatorname{\mathbb{R}}^{d_{s}\times d_{v}}$.
The attention weights
$\operatorname{\textbf{A}}\in\operatorname{\mathbb{R}}^{d_{s}\times d_{s}}$
can be computed by

$\operatorname{\textbf{A}}=\operatorname{softmax}\left(\frac{\operatorname{\textbf{Q}}\operatorname{\textbf{K}}^{T}}{\sqrt{d_{q}}}\right).$
(1)

The $(i,j)_{\textsuperscript{th}}$ element of $\operatorname{\textbf{A}}$
shows how much of $i_{\text{th}}$ token is influenced by $j_{\text{th}}$
token. The output of a head
$\operatorname{\textbf{H}}\in\operatorname{\mathbb{R}}^{d_{s}\times d_{e}}$ is
given by

$\operatorname{\textbf{H}}=\operatorname{\textbf{A}}\operatorname{\textbf{V}}\operatorname{\textbf{D}}=\operatorname{\textbf{A}}\operatorname{\textbf{T}},$
(2)

where $\operatorname{\textbf{D}}\in\operatorname{\mathbb{R}}^{d_{v}\times
d_{e}}$ is a linear layer and the matrix
$\operatorname{\textbf{T}}\in\operatorname{\mathbb{R}}^{d_{s}\times d_{e}}$
denotes the operation $\operatorname{\textbf{V}}\operatorname{\textbf{D}}$.
The $\operatorname{\mathbb{R}}^{d_{s}\times d_{e}}$ output of multi-head
attention can be expressed as a summation over $\operatorname{\textbf{H}}$
obtained for each head333For simplicity, we have omitted head indices.. The
$i\textsubscript{th}$ row of multi-head output matrix corresponds to the
$d_{e}$ dimensional contextualized representation of $i\textsubscript{th}$
input token. In the original work, Vaswani et al. (2017), the multi-head
operation is described as the concatenation of
$\operatorname{\textbf{A}}\operatorname{\textbf{V}}$ obtained from each head
followed by a linear transformation
$\operatorname{\textbf{D}}\in\operatorname{\mathbb{R}}^{d_{e}\times d_{e}}$.
Both the explanations are associated with the same sequence of matrix
operations as shown in fig. 1.

Figure 1: An illustration for a Transformer with two-head attention units.
Triangles depict matrix weights. The left side shows concatenation of head
outputs fed to a linear layer. The right side shows another interpretation of
the same set of operations where we consider a linear transform applied to
each head first. The transformed head outputs are then added.

In regular Transformer setting, a token vector is
$t_{i}\in\\{(z_{j}+p_{j})\\}_{i=1}^{d_{s}}$ is $d_{e}$ = $512$ dimensional,
number of heads $h$=8, size of $d_{k}$=$d_{q}$=$d_{v}$=$d_{e}/h$=64.

##### Feed-Forward Network.

This sub-layer performs the following transformations on each token
representation at the output of a head:

$\displaystyle y_{1}$
$\displaystyle=\operatorname{Linear}_{1}(\operatorname{Norm}(t_{i}+\text{head
output for }t_{i}))$ $\displaystyle y_{2}$
$\displaystyle=\operatorname{Norm}(t_{i}+\operatorname{ReLU}(\operatorname{Linear}_{2}(y_{1})))$

$\operatorname{Linear}_{1}$ and $\operatorname{Linear}_{2}$ are linear layers
with 2048 and 512 nodes, respectively. $\operatorname{Norm}$ denotes mini-
batch layer normalization.

##  3 Identifiability of Attention

The output of an attention head $\operatorname{\textbf{H}}$ is the product of
$\operatorname{\textbf{A}}$ and $\operatorname{\textbf{T}}$ (eq. 2). Formally,
we define identifiability of attention in a head:

######  Definition 3.1.

For an attention head’s output $\operatorname{\textbf{H}}$, attention weights
$\operatorname{\textbf{A}}$ are identifiable if there exists a unique solution
of
$\operatorname{\textbf{A}}\operatorname{\textbf{T}}=\operatorname{\textbf{H}}$.

The above definition can be reformulated as

######  Definition 3.2.

$\operatorname{\textbf{A}}$ is unidentifiable if there exist an
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$,
$(\operatorname{\tilde{\operatorname{\textbf{A}}}}\neq\mathbf{0})$, such that
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})$
is obtainable from phase-1 of head computations and satisfy

$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})\operatorname{\textbf{T}}=\operatorname{\textbf{A}}\operatorname{\textbf{T}}\implies\operatorname{\tilde{\operatorname{\textbf{A}}}}\operatorname{\textbf{T}}=\mathbf{0}.$
(constraint-R1)

Under this constraint, we get $\tilde{a}_{i}\operatorname{\textbf{T}}=0$ where
$\tilde{a}_{i}$ is the $i_{th}$ row of
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$. The set of vectors which
when multiplied to $\operatorname{\textbf{T}}$ gets mapped to zero describes
the left null space of $\operatorname{\textbf{T}}$ denoted by
$\operatorname{LN}(\operatorname{\textbf{T}})$. The dimension of the left null
space of $\operatorname{\textbf{T}}$ can be obtained by taking the difference
of the total number of rows ($d_{s}$) and the number of linearly independent
rows, i.e, rank of the matrix $\operatorname{\textbf{T}}$ denoted by
$\operatorname{rank}(\operatorname{\textbf{T}})$. Let $\dim(\cdot)$ denotes
the dimension of a vector space, then

$\displaystyle\operatorname{LN}(\operatorname{\textbf{T}})$
$\displaystyle=\\{\mathbf{v}\mid\mathbf{v}^{T}\operatorname{\textbf{T}}=\mathbf{0}\\}$
(3)
$\displaystyle\dim\big{(}\operatorname{LN}(\operatorname{\textbf{T}})\big{)}$
$\displaystyle=d_{s}-\operatorname{rank}(\operatorname{\textbf{T}}).$ (4)

###  3.1 “$\operatorname{\textbf{A}}$” is Identifiable for
$\mathbb{\textit{d}_{\textit{s}}\leq\textit{d}_{\textit{v}}}$

If $\dim(\operatorname{LN}(\operatorname{\textbf{T}}))=0$ then
$\operatorname{LN}(\operatorname{\textbf{T}})=\\{\mathbf{0}\\}$, it leads to
the only solution of constraint-R1 that is
$\operatorname{\tilde{\operatorname{\textbf{A}}}}=\mathbf{0}$. Therefore, the
unidentifiabilty condition does not hold. Now we will prove such a situation
exists when the number of tokens is not more than the size of value vector.

The matrix $\operatorname{\textbf{T}}$ in eq. 2 is product of $d_{s}\times
d_{v}$ value matrix $\operatorname{\textbf{V}}$ and $d_{v}\times d_{e}$
transformation $\operatorname{\textbf{D}}$. We utilize the fact that the rank
of product of two matrices $\operatorname{\textbf{P}}$ and
$\operatorname{\textbf{Q}}$ is upper bounded by the minimum of
$\operatorname{rank}(\operatorname{\textbf{P}})$ and
$\operatorname{rank}(\operatorname{\textbf{Q}})$, i.e.,
$\operatorname{rank}(\operatorname{\textbf{P}}\operatorname{\textbf{Q}})\leq\min\big{(}\operatorname{rank}(\operatorname{\textbf{P}}),\operatorname{rank}(\operatorname{\textbf{Q}})\big{)}$.
Thus, the upper bound on $\operatorname{rank}(\operatorname{\textbf{T}})$ in
eq. 4 can be determined by

$\displaystyle\operatorname{rank}(\operatorname{\textbf{T}})$
$\displaystyle\leq\min\Big{(}\operatorname{rank}(\operatorname{\textbf{V}}),\;\operatorname{rank}(\operatorname{\textbf{D}})\Big{)}$
(5) $\displaystyle\leq\min\Big{(}\min(d_{s},d_{v}),\min(d_{v},d_{e})\Big{)}$
$\displaystyle\leq\min\Big{(}d_{s},d_{v},d_{v},d_{e}\Big{)}$
$\displaystyle\leq\min\Big{(}d_{s},d_{v}\Big{)}\;\;\;\;\;\;\text{(as
$d_{e}>d_{v}$)}$ $\displaystyle=\min\Big{(}d_{s},64\Big{)}$

where the last inequality is obtained for a head in the regular Transformer
for which $d_{v}$=64.

##### Numerical rank.

To substantiate the bounds on $\operatorname{rank}(\operatorname{\textbf{T}})$
as derived above, we set up a model with a single encoder layer (section 6).
The model is trained to predict the sentiment of IMDB reviews (section 5). We
feed the review tokens to the model and store the values generated in
$\operatorname{\textbf{T}}$ of the first head. A standard technique for
calculating the rank of a matrix with floating-point values and computations
is to use singular value decomposition. The rank of the matrix will be
computed as the number of singular values larger than the predefined
threshold444The threshold value is
$\max(d_{s},d_{e})*eps*||\operatorname{\textbf{T}}||_{2}$. The $eps$ is
floating-point machine epsilon value, i.e., 1.19209e-07 in our experiments.
The fig. 2 illustrates how the rank changes with the sequence length $d_{s}$.
The numerical rank provides experimental support to the theoretical analysis.

$\operatorname{rank}(\operatorname{\textbf{T}})=\left\\{\begin{array}[]{ll}d_{s}&\mbox{if
}d_{s}\leq d_{v},\\\ d_{v}&\mbox{if }d_{s}>d_{v}.\end{array}\right.$ (6)

Thus,

$\displaystyle\dim\big{(}\operatorname{LN}(\operatorname{\textbf{T}})\big{)}$
$\displaystyle=d_{s}-\operatorname{rank}(\operatorname{\textbf{T}})$ (7)
$\displaystyle=\left\\{\begin{array}[]{ll}0&\mbox{if }d_{s}\leq d_{v},\\\
(d_{s}-d_{v})&\mbox{if }d_{s}>d_{v}.\end{array}\right.$
$\displaystyle=\max\;(d_{s}-d_{v},0)$

Figure 2: Numerical rank of $\operatorname{\textbf{T}}$ (IMDB) and dimension
of its left null space are scattered in blue and red, respectively.

With this, we infer $\operatorname{\textbf{A}}$ is identifiable if $d_{s}\leq
d_{v}$ = $64$. For the identifiability study, since we focus on a model’s
capability of learning unique attention weights, we will assume
$\operatorname{\textbf{T}}$ has the maximum obtainable rank set by its upper
bound.

###  3.2 Idenitifiability when
$\mathbf{\textit{d}_{\textit{s}}>\textit{d}_{\textit{v}}}$
(the hidden role of $\mathbf{\textit{d}_{\textit{k}}}$)

In this case, from eq. 7, we obtain a non zero value of
$\dim\big{(}\operatorname{LN}(\operatorname{\textbf{T}})\big{)}$. It allows us
to find infinite $\operatorname{\tilde{\operatorname{\textbf{A}}}}$’s
satisfying
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})\operatorname{\textbf{T}}=\operatorname{\textbf{A}}\operatorname{\textbf{T}}$.
However, constraint-R1 demands
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$ to be obtainable from the
first phase of self-attention. As a first step, we focus our analysis on the
attention matrix without applying $\operatorname{softmax}$ non-linearity,
i.e.,
$\operatorname{\textbf{A}}=\left(\frac{\operatorname{\textbf{Q}}\operatorname{\textbf{K}}^{T}}{\sqrt{d_{q}}}\right)$.
The analysis is crucial to identify constraints coming from the first phase of
self-attention in Transformer that impact identifiability. Insights from this
will help us analyse $\operatorname{softmax}$ version of
$\operatorname{\textbf{A}}$.

####  3.2.1 Attention Weights as Logits

Since the logits matrix $\operatorname{\textbf{A}}$ is obtained from the
product of $\operatorname{\textbf{Q}}$ and $\operatorname{\textbf{K}}^{T}$, we
can assert that

$\displaystyle\operatorname{rank}(\operatorname{\textbf{A}})$
$\displaystyle\leq\min\big{(}\operatorname{rank}(\operatorname{\textbf{Q}}),\operatorname{rank}(\operatorname{\textbf{K}}^{T})\big{)}$
(8) $\displaystyle\leq\min\big{(}d_{e},d_{k},d_{q},d_{e}\big{)}$
$\displaystyle=d_{k}.$

Therefore, the rank of attention matrix producible by the head in the first
phase of self-attention can at most be equal to the size of key vectors
$d_{k}$. On this basis, the head can produce only those
$\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}}$
satisfying

$\operatorname{rank}(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})\leq
d_{k}$ (constraint-R2)

######  Proposition 3.3.

There exists a non-trivial $\operatorname{\tilde{\operatorname{\textbf{A}}}}$
that satisfy
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})\operatorname{\textbf{T}}=\operatorname{\textbf{A}}\operatorname{\textbf{T}}$
and constraint-R2. Hence, $\operatorname{\textbf{A}}$ is unidentifiable.

###### Proof.

Let $a_{1},\ldots,a_{d_{s}}$ and $\tilde{a}_{1},\ldots,\tilde{a}_{d_{s}}$
denote rows of $\operatorname{\textbf{A}}$ and
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$, respectively. Without the
loss of generality, let $a_{1},\ldots,a_{d_{k}}$ be linearly independent rows.
For all $j>d_{k}$, $a_{j}$ can be represented as a linear combination
$\sum_{i=1}^{d_{k}}\lambda_{i}^{j}a_{i}$, where $\lambda_{i}^{j}$ is a scalar.
Next, we independently choose first $k$ rows of
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$ that are
$\\{\tilde{a}_{1},\ldots,\tilde{a}_{d_{k}}\\}$ from
$\operatorname{LN}(\operatorname{\textbf{T}})$. From the same set of
coefficients of linear combination $\lambda_{i}^{j}$ for
$i\in\\{1,\ldots,d_{k}\\}$ and $j\in\\{d_{k+1},\ldots,d_{s}\\}$, we can
construct $j_{\textsubscript{th}}$ row of
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$ as $\tilde{a}_{j}$ =
$\sum_{i=1}^{d_{k}}\lambda_{i}^{j}\tilde{a_{i}}$. Now, since we can construct
the $j_{\text{th}}$ row of
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})$
from the linear combination of its first $d_{k}$ rows as
$\sum_{i=1}^{d_{k}}\lambda_{i}^{j}(a_{i}+\tilde{a}_{i})$, the rank of
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})$
is not more than $d_{k}$. For a set of vectors lying in a linear space, a
vector formed by their linear combination should also lie in the same space.
Thus, the artificially constructed rows of
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$ belongs to
$\operatorname{LN}(\operatorname{\textbf{T}})$. Therefore, there exist an
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$ that establishes the
proposition which claims the unidentifiability of $\operatorname{\textbf{A}}$.
∎

####  3.2.2 Attention Weights as Softmaxed Logits

The $\operatorname{softmax}$ over attention logits generates attention weights
with each row of $\operatorname{\textbf{A}}$ (i.e., $a_{i}$’s) is constrained
to be a probability distribution. Hence, we can define constraint over
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$ as

$\displaystyle(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})$
$\displaystyle\geq\mathbf{0}$ (P1)
$\displaystyle\operatorname{\tilde{\operatorname{\textbf{A}}}}\operatorname{\textbf{T}}$
$\displaystyle=\mathbf{0}$ (P2)
$\displaystyle\operatorname{\tilde{\operatorname{\textbf{A}}}}\textbf{1}$
$\displaystyle=\mathbf{0}.$ (P3)

P1 is non-negativity constraint on
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})$
as it is supposed to be the output of $\operatorname{softmax}$; P2 denotes
$\operatorname{\tilde{\operatorname{\textbf{A}}}}\in\operatorname{LN}(\operatorname{\textbf{T}})$;
P3 can be derived from the fact
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})\textbf{1}$
= 1 $\implies$
$(\operatorname{\textbf{A}}\textbf{1}+\operatorname{\tilde{\operatorname{\textbf{A}}}}\textbf{1})$
$=$ 1 $\implies\operatorname{\tilde{\operatorname{\textbf{A}}}}\textbf{1}$ $=$
$\mathbf{0}$ as ($\operatorname{\textbf{A}}\textbf{1}$ $=$ 1). Where
$\textbf{1}\in\operatorname{\mathbb{R}}^{d_{s}}$ is the vector of ones. The
constraint in P2 and P3 can be combined and reformulated as
$\operatorname{\tilde{\operatorname{\textbf{A}}}}[\operatorname{\textbf{T}},\textbf{1}]=\mathbf{0}$.
Following the similar analysis as in eq. 7, we can obtain
$\dim\big{(}\operatorname{LN}([\operatorname{\textbf{T}},\textbf{1}])\big{)}=\max\big{(}d_{s}-(d_{v}+1),0\big{)}$.
Disregarding the extreme cases when $a_{i}$ is a one-hot distribution, Brunner
et al. (2019) proved the existence and construction of non-trivial
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$’s satisfying all the
constraints P1, P2, and P3.555For the sake of brevity, we skip the
construction method.

However, the proof by Brunner et al. (2019) missed the constraint-R2, hence
the existence of a non-trivial
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$ satisfying only the set of
constraints P1, P2 and P3 may not be a valid proposition to claim attention
weights unidentifiability. Essentially, the work largely ignored the
constraints coming from the rank of the matrix that produces
$\operatorname{\textbf{A}}$ after $\operatorname{softmax}$ 666(input to the
$\operatorname{softmax}$ is equivalent to $\operatorname{\textbf{A}}$ in
section 3.2.1). Let $\operatorname{\textbf{A}}_{l}$ denote logits
$\left(\frac{\operatorname{\textbf{Q}}\operatorname{\textbf{K}}^{T}}{\sqrt{d_{q}}}\right)$
and
$\operatorname{softmax}(\operatorname{\textbf{A}}_{l})=(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})$,
where $\operatorname{softmax}$ is operated over each row of
$\operatorname{\textbf{A}}_{l}$. We add an extra constraint on
$\operatorname{\textbf{A}}_{l}$

$\operatorname{rank}(\operatorname{\textbf{A}}_{l})\leq d_{k}.$ (P4)

The constraint P4 confirms if there exists a logit matrix
$\operatorname{\textbf{A}}_{l}$ that can generate
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})$,
given constraints P1, P2, and P3 are satisfied. The possibility of such an
$\operatorname{\textbf{A}}_{l}$ will provide sufficient evidence that
$\operatorname{\textbf{A}}$ is unidentifiable. Next, we investigate how the
existence of $\operatorname{\tilde{\operatorname{\textbf{A}}}}$ is impacted by
the size of key vector $d_{k}$ (query and key vector sizes are the same, i.e.,
$d_{q}$=$d_{k}$).

Let
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})(i,k)$
denotes $(i,k)\textsubscript{th}$ element of the matrix. We can retrieve the
set of matrices $\operatorname{\textbf{A}}_{l}$ such that
$\operatorname{softmax}(\operatorname{\textbf{A}}_{l})={\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}}}$,
where

$\operatorname{\textbf{A}}_{l}(i,k)=c_{i}+\log(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})(i,k)$
(9)

for some arbitrary $c_{i}\in\operatorname{\mathbb{R}}$; $\log$ denotes natural
logarithm. As shown in fig. 3, the column vectors of
$\operatorname{\textbf{A}}_{l}$ can be written as
$\bm{c}+\mathbf{\hat{a}_{1}},\ldots,\bm{c}+\mathbf{\hat{a}_{d_{s}}}$.

Figure 3: Column vectors $(\bm{c}+\mathbf{\hat{a}_{k}})$ of
$\operatorname{\textbf{A}}_{l}$, where $a_{(i,k)}$ represents
$\log(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})(i,k)$.

For an arbitrarily picked $\operatorname{\tilde{\operatorname{\textbf{A}}}}$
satisfying constraint P1, P2, and P3, the dimensions of affine span
$\mathcal{S}$ of $\\{\mathbf{\hat{a}_{1}},\ldots,\mathbf{\hat{a}_{d_{s}}}\\}$
could be as high as $d_{s}-1$ (fig. 4). In such cases, the best one could do
is to choose a $\mathbf{c_{a}}\in\mathcal{S}$ such that the dimension of the
linear span of
$\\{\mathbf{\hat{a}_{1}-c_{a}},\ldots,\mathbf{\hat{a}_{d_{s}}-c_{a}}\\}$,
i.e., $\operatorname{rank}(\operatorname{\textbf{A}}_{l})$ is $d_{s}-1$.
Hence, to satisfy P4, $d_{s}-1\leq d_{k}\implies d_{s}\leq d_{k}+1$. Thus, the
set of
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})$
satisfying constraint P1, P2 and P3 are not always obtainable from attention
head for $d_{s}>d_{k}$. We postulate

> Although it is easier to construct
> $\operatorname{\tilde{\operatorname{\textbf{A}}}}$ satisfying constraints
> P1, P2 and P3, it is hard to construct
> $\operatorname{\tilde{\operatorname{\textbf{A}}}}$ satisfying constraint P4
> over the rank of logit matrix $\operatorname{\textbf{A}}_{l}$. Therefore,
> $\operatorname{\textbf{A}}$ becomes more identifiable as the size of key
> vector decreases.

Figure 4: This is a simplified illustration for the case $d_{s}=3$. Affine
space (translated linear subspace) spanned by vectors $\mathbf{\hat{a}_{1}}$,
$\mathbf{\hat{a}_{2}}$ and $\mathbf{\hat{a}_{3}}$. $\mathbf{c_{a}}$ can be any
arbitrary vector in affine space. By putting $\mathbf{c}=-\mathbf{c_{a}}$, we
can obtain a linear subspace whose rank is equal to rank of the affine
subspace.

##### Experimental evidence.

We conduct an experiment to validate the minimum possible numerical rank of
$\operatorname{\textbf{A}}_{l}$ by constructing
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$. For
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$ to be obtainable from the
phase 1, the minimum possible rank of $\operatorname{\textbf{A}}_{l}$ should
not be higher than $d_{k}$. From IMDB dataset (section 5), we randomly sample
a set of reviews with token sequence length $d_{s}$ ranging from 66 to 128
777$\dim\big{(}\operatorname{LN}(\operatorname{\textbf{T}},\textbf{1})\big{)}>0$
for $d_{s}>d_{v}+1=65$. For each review, we construct 1000
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$’s satisfying constraints
P1, P2, and P3 —

First, we train a Transformer encoder-based IMDB review sentiment classifier
(section 6). We obtain an orthonormal basis for the left null space of
$[\operatorname{\textbf{T}},\textbf{1}]$ using singular value decomposition.
To form an $\operatorname{\tilde{\operatorname{\textbf{A}}}}$, we generate
$d_{s}$ random linear combinations of the basis vectors (one for each of its
row). Each set of linear combination coefficients is sampled uniformly from
$[-10,10]$. All the rows are then scaled to satisfy the constraint P1 as
mentioned in Brunner et al. (2019). Using eq. 9, we obtain a minimum rank
matrix $\operatorname{\textbf{A}}_{l}$’s by putting
$\textbf{c}=-\mathbf{\hat{a}_{1}}$. Figure 5 depicts the obtained numerical
rank of $\operatorname{\textbf{A}}_{l}$. We observed all the obtained
$\operatorname{\textbf{A}}_{l}$ from
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})$
(using eq. 9) are full-row rank matrices. However, from the first phase of
self-attention, the maximum obtainable rank of $\operatorname{\textbf{A}}_{l}$
is $d_{k}=64$. Thus, the experimentally constructed
$\operatorname{\textbf{A}}_{l}$’s do not claim unidentifiability of
$\operatorname{\textbf{A}}$ as it fails to satisfy the constraint P4, while
for Brunner et al. (2019), it falls under the solution set to prove
unidentifiability as it meets constraints P1, P2 and P3.

Figure 5: The blue curve denotes the expected rank of
$\operatorname{\textbf{A}}_{l}$’s obtained from
$(\operatorname{\textbf{A}}+\operatorname{\tilde{\operatorname{\textbf{A}}}})$,
where $\operatorname{\tilde{\operatorname{\textbf{A}}}}$ satisfies the
constraints P1, P2, and P3. The red curve denotes the maximum permissible rank
of $\operatorname{\textbf{A}}_{l}$ that is obtainable from phase 1 of the
head.

##  4 Solutions to Identifiability

Based on the Identifiability analysis in section 3, we propose basic solutions
to make Transformer’s attention weights identifiable.

##### Decoupling $\mathbf{\textit{d}_{\textit{k}}}$.

Contrary to the regular Transformer setting where $d_{k}=d_{v}$, a simple
approach is to decrease the value of $d_{k}$ that is the size of the key and
query vector. It will reduce the possible solutions of
$\operatorname{\tilde{\operatorname{\textbf{A}}}}$ by putting harder
constraints on the rank of attention logits, i.e.,
$\operatorname{\textbf{A}}_{l}$ in eq. 9. However, theoretically, $d_{k}$
decides the upper bound on dimensions of the space to which token embeddings
are projected before the dot product. Higher the upper bound, more degree of
freedom to choose the subspace dimensions as compared to the lower $d_{k}$
variants. Thus, there is a plausible trade-off when choosing between $d_{k}$
induced identifiability and the upper bound on the dimension of projected
space.

##### Head Addition.

To resolve the unidentifiability issue when sequence length exceeds the size
of value vector, we propose to keep the value vector size and token embedding
dimension to be more than (or equal to) the maximum allowed input tokens,
i.e., $d_{v}\geq d_{s\text{-max}}$. In Vaswani et al. (2017), $d_{v}$ was
bound to be equal to $d_{e}/h$, where $d_{e}$ is token embedding dimension and
$h$ is number of heads. This constraint on $d_{v}$ is because of the
concatenation of $h$ self-attention heads to produce $d_{e}$-sized output at
the first sub-layer of the encoder. Thus, to decouple $d_{v}$ from this
constraint, we keep $d_{v}=d_{e}$ and add each head’s
output.888$d_{s\text{-max}}<d_{e}$ as in the regular Transformer setting.

##  5 Classification Tasks

For the empirical analysis of our proposed solutions as mentioned in section
4, we conduct our experiments on the following varied text classification
tasks:

###  5.1 Small Scale Datasets

##### IMDB Maas et al. (2011).

The dataset for the task of sentiment classification consist of IMDB movie
reviews with their sentiment as positive or negative. Each of the train and
test sets contain 25,000 data samples equally distributed in both the
sentiment polarities.

##### TREC Voorhees and Tice (2000).

We use the 6-class version of the dataset for the task of question
classification consisting of open-domain, facet-based questions. There are
5,452 and 500 samples for training and testing, respectively.

##### SST Socher et al. (2013).

Stanford sentiment analysis dataset consist of 11,855 sentences obtained from
movie reviews. We use the 3-class version of the dataset for the task of
sentiment classification. Each review is labeled as positive, neutral, or
negative. The provided train/test/valid split is 8,544/2,210/1,101.

###  5.2 Large Scale Datasets

##### SNLI Bowman et al. (2015).

The dataset contain 549,367 samples in the training set, 9,842 samples in the
validation set, and 9,824 samples in the test set. For the task of recognizing
textual entailment, each sample consists of a premise-hypothesis sentence pair
and a label indicating whether the hypothesis entails the premise, contradicts
it, or neutral.

Please refer to Zhang et al. (2015) for more details about the following
datasets:

##### Yelp.

We use the large-scale Yelp review dataset for the task of binary sentiment
classification. There are 560,000 samples for training and 38,000 samples for
testing, equally split into positive and negative polarities.

##### DBPedia.

The Ontology dataset for topic classification consist of 14 non-overlapping
classes each with 40,000 samples for training and 5,000 samples for testing.

##### Sogou News.

The dataset for news article classification consist of 450,000 samples for
training and 60,000 for testing. Each article is labeled in one of the 5 news
categories. The dataset is perfectly balanced.

##### AG News.

The dataset for the news articles classification partitioned into four
categories. The balanced train and test set consist of 120,000 and 7,600
samples, respectively.

##### Yahoo! Answers.

The balanced dataset for 10-class topic classification contain 1,400,000
samples for training and 50,000 samples for testing.

##### Amazon Reviews.

For the task of sentiment classification, the dataset contain 3,600,000
samples for training and 400,000 samples for testing. The samples are equally
divided into positive and negative sentiment labels.

Except for the SST and SNLI, where the validation split is already provided,
we flag 30% of the train set as part of the validation set and the rest 70%
were used for model parameter learning.

##  6 Experimental Setup

##### Setting up the encoder.

We normalize the text by lower casing, removing special characters,
etc.999https://pytorch.org/text/_modules/torchtext/data/utils.html For each
task, we construct separate 1-Gram vocabulary ($U$) and initialize a trainable
randomly sampled token embedding ($U\times d_{e}$) from $\mathcal{N}(0,1)$.
Similarly, we randomly initialize a ($d_{\text{{s}-max}}\times d_{e}$)
positional embedding.

The encoder (section 2.2) takes input a sequence of token vectors
($d_{s}\times d_{e}$) with added positional vectors. The input is then
projected to key and query vector of size
$d_{k}\in\\{1,2,4,8,16,32,64,128,256\\}$. For the regular Transformer setting,
we fix the number of heads $h$ to 8 and the size of value vector
$d_{v}=d_{e}/h$ that is 64. For each token at the input, the outputs of
attention heads are concatenated to generate a $d_{e}$-sized vector. For the
identifiable variant of the Transformer encoder, $d_{v}=d_{e}=512$, this is
equal to $d_{\text{{s}-max}}$ to keep it identifiable up to the maximum
permissible number of tokens. The outputs of all the heads are then added.
Each token’s contextualized representations (added head outputs) are then
passed through the feed-forward network (section 2.2). For classification, we
use the encoder layer’s output for the first token and pass it through a
linear classification layer. In datasets with more than two classes, the
classifier output is $\operatorname{softmax}$ed. In the case of SNLI, we use
the shared encoder for both premise and hypothesis; the output of their first
tokens is then concatenated just before the final classification layer. We use
Adam optimizer, with learning rate =0.001, to minimize the cross-entropy loss
between the target and predicted label. For all the experiments, we keep the
batch size as 256 and train for 20 epochs. We report the test accuracy
obtained at the epoch with the best validation accuracy.

##### Numerical rank.

To generate the numerical rank plot on IMDB dataset as shown in fig. 2, we
train a separate Transformer encoder-based classifier. For a particular
$d_{s}$ value, we sample 100 reviews from the dataset with token length $\geq
d_{s}$ and clip each review to the maximum length $d_{s}$. The clipping will
ensure the number of tokens is $d_{s}$ before feeding it to the encoder. The
numerical rank is calculated for $\mathbf{T}$’s obtained from the first head
of the encoder.

##  7 Results and Discussion

Dataset | Version | Size of key vector $(d_{k})$
---|---|---
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128 | 256
IMDB | Con | 0.884 | 0.888 | 0.886 | 0.888 | 0.846 | 0.824 | 0.803 | 0.788 | 0.755
Add | 0.888 | 0.885 | 0.887 | 0.884 | 0.886 | 0.882 | 0.877 | 0.832 | 0.825
TREC | Con | 0.836 | 0.836 | 0.840 | 0.822 | 0.823 | 0.764 | 0.786 | 0.706 | 0.737
Add | 0.841 | 0.842 | 0.835 | 0.842 | 0.841 | 0.836 | 0.809 | 0.809 | 0.771
SST | Con | 0.643 | 0.625 | 0.627 | 0.609 | 0.603 | 0.582 | 0.574 | 0.573 | 0.554
Add | 0.599 | 0.618 | 0.628 | 0.633 | 0.628 | 0.629 | 0.592 | 0.581 | 0.586
SNLI | Con | 0.675 | 0.674 | 0.673 | 0.672 | 0.662 | 0.659 | 0.659 | 0.655 | 0.648
Add | 0.683 | 0.677 | 0.674 | 0.676 | 0.673 | 0.669 | 0.663 | 0.664 | 0.655
Yelp | Con | 0.913 | 0.911 | 0.907 | 0.898 | 0.879 | 0.862 | 0.857 | 0.849 | 0.837
Add | 0.914 | 0.915 | 0.916 | 0.914 | 0.915 | 0.916 | 0.910 | 0.909 | 0.891
DBPedia | Con | 0.979 | 0.977 | 0.977 | 0.971 | 0.966 | 0.961 | 0.957 | 0.951 | 0.949
Add | 0.979 | 0.978 | 0.979 | 0.977 | 0.978 | 0.973 | 0.970 | 0.969 | 0.964
Sogou | Con | 0.915 | 0.907 | 0.898 | 0.900 | 0.893 | 0.888 | 0.868 | 0.858 | 0.838
Add | 0.915 | 0.908 | 0.906 | 0.904 | 0.913 | 0.914 | 0.910 | 0.906 | 0.899
AG News | Con | 0.906 | 0.903 | 0.904 | 0.904 | 0.886 | 0.877 | 0.870 | 0.870 | 0.869
Add | 0.902 | 0.908 | 0.907 | 0.906 | 0.897 | 0.899 | 0.901 | 0.897 | 0.893
Yahoo | Con | 0.695 | 0.690 | 0.684 | 0.664 | 0.644 | 0.627 | 0.616 | 0.597 | 0.574
Add | 0.697 | 0.695 | 0.696 | 0.693 | 0.693 | 0.694 | 0.688 | 0.649 | 0.683
Amazon | Con | 0.924 | 0.925 | 0.923 | 0.922 | 0.900 | 0.892 | 0.887 | 0.882 | 0.873
Add | 0.925 | 0.923 | 0.925 | 0.924 | 0.924 | 0.920 | 0.907 | 0.896 | 0.889

Table 1:  The test accuracy on varied text classification tasks spread over
ten datasets. Con means the regular concatenation of heads with
$d_{v}=d_{e}/h$, Add denotes encoder variant where $d_{v}=d_{e}$ and outputs
of heads are added. In the regular Transformer encoder $Con$, the
concatenation of $d_{v}$-sized output of $h$ heads followed by $d_{e}\times
d_{e}$ linear transformation can be understood as first doing linear
$d_{v}\times d_{e}$ linear transform of each head and then addition of the
transformed output (fig. 1). In the Add variant, we first add $h$
$d_{v}$-sized head outputs followed by $d_{e}\times d_{e}$ linear
transformation.

For the identifiable variant, similar to section 3.1, we plot the numerical
rank of $\operatorname{\textbf{T}}$ with input sequence length as shown in
fig. 6. Unlike fig. 2, where
$\dim\big{(}\operatorname{LN}(\operatorname{\textbf{T}})\big{)}$ linearly
increases after $d_{s}=64$, we find the dimension is zero for a larger $d_{s}$
($\sim 380$). The zero dimensional (left) null space of
$\operatorname{\textbf{T}}$ confirms there exist no nontrivial solution to the
constraint constraint-R2, i.e.,
$\operatorname{\tilde{\operatorname{\textbf{A}}}}=\\{\mathbf{0}\\}$. Thus, the
attention weights $\operatorname{\textbf{A}}$ are identifiable for a larger
range of length of the input sequence.

Figure 6: Scatter plots in red and blue show
$\operatorname{rank}(\operatorname{\textbf{T}})$ and
$\dim\big{(}\operatorname{LN}(\operatorname{\textbf{T}})\big{)}$,
respectively, for matrices $\operatorname{\textbf{T}}$ obtained from the
second phase of attention by feeding IMDB samples to the encoder. The green
line shows the desired $\operatorname{rank}(\operatorname{\textbf{T}})$ for
which $\dim\big{(}\operatorname{LN}(\operatorname{\textbf{T}})\big{)}=0$ and
thus attention weights are identifiable.

It is important that the identifiability of attention weights should not come
at the cost of reduced performance of the model. To investigate this issue, we
compare the performance of the identifiable Transformer encoder against its
regular settings (section 6) on varied text classification tasks.

For the regular setting, as discussed in section 4 as one of the solutions,
the Transformer can be made identifiable by decreasing the size of the key
vector $d_{k}$. The rows of the Table 1 corresponding to Con denotes regular
Transformer setting with varying size of key vector. We observe the
classification accuracy at the lower $d_{k}$ is comparable or higher than
large $d_{k}$ values, thus, the enhanced identifiability does not compromise
with the model’s classification accuracy. However, we notice a general
performance decline with an increase in the size of the key vector. We
speculate that for simple classification tasks, the lower-dimensional
projection for key and query vector works well. However, as the task becomes
more involved, a higher dimension for the projected subspace could be
essential. Nonetheless, as we do not have strong theoretical findings, we
leave this observation for future work.

Another solution to identifiability is to increase $d_{v}$ to $d_{e}$ and add
the heads’ outputs. This setting corresponds to the Add rows in the Table 1.
For key vector size $d_{k}$= 1, 2, and 4, We find the identifiable
Transformer’s performance is comparable to the regular settings. For
$d_{k}\geq 8$, as a general observation, we find the performance of Add does
not drop as drastically as Con with an increase in $d_{k}$. This could be due
to the larger size of value vector leading to the more number of parameters in
Add that compensate for the significant reduction in the model’s accuracy.

On the large-scale datasets, we observe that Add performs slightly better than
Con. Intuitively, as shown in fig. 1, we can increase the size of value vector
to increase the dimension of the space on which each token is projected. A
higher dimensional subspace can contain more semantic information to perform
the specific task.

Even though the theoretical analysis shows the possibility of a full row rank
of $\operatorname{\textbf{T}}$ and identifiable attention weights, the
$\operatorname{\textbf{T}}$ obtained from a trained model might not contain
all the rows linearly independent as $d_{s}$ increases. We can explain this
from the semantic similarities between words co-occurring together Harris
(1954). The similarity is captured as the semantic relationship, such as dot
product, between vectors in a linear space. As the number of tokens in a
sentence, i.e., $d_{s}$ increases, it becomes more likely to obtain a token
vector from the linear combination of other tokens.

##  8 Conclusion

This work probed Transformer for identifiability of self-attention, i.e., the
attention weights can be uniquely identified from the head’s output. With
theoretical analysis and supporting empirical evidence, we were able to
identify the limitations of the existing study by Brunner et al. (2019). We
found the study largely ignored the constraint coming from the first phase of
self-attention in the encoder, i.e., the size of the key vector. Later, we
proved how we can utilize $d_{k}$ to make the attention weights more
identifiable. To give a more concrete solution, we propose encoder variants
that are more identifiable, theoretically as well as experimentally, for a
large range of input sequence lengths. The identifiable variants do not show
any performance drop when experiments are done on varied text classification
tasks. Future works may analyse the critical impact of identifiability on the
explainability and interpretability of the Transformer.

## Acknowledgments

This research is supported by A*STAR under its RIE 2020 Advanced Manufacturing
and Engineering programmatic grant, Award No.– A19E2b0098.

## References

  * Bellman and Åström (1970) Ror Bellman and Karl Johan Åström. 1970.  On structural identifiability.  _Mathematical biosciences_ , 7(3-4):329–339.
  * Bowman et al. (2015) Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015\.  A large annotated corpus for learning natural language inference.  _arXiv preprint arXiv:1508.05326_.
  * Brunner et al. (2019) Gino Brunner, Yang Liu, Damian Pascual, Oliver Richter, Massimiliano Ciaramita, and Roger Wattenhofer. 2019.  On identifiability in transformers.  _arXiv preprint arXiv:1908.04211_.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Harris (1954) Zellig S Harris. 1954.  Distributional structure.  _Word_ , 10(2-3):146–162.
  * Jain and Wallace (2019) Sarthak Jain and Byron C Wallace. 2019.  Attention is not explanation.  _arXiv preprint arXiv:1902.10186_.
  * Lu et al. (2019) Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019.  Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks.  In _Advances in Neural Information Processing Systems_ , pages 13–23.
  * Maas et al. (2011) Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011.  Learning word vectors for sentiment analysis.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 142–150, Portland, Oregon, USA. Association for Computational Linguistics.
  * Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018.  Improving language understanding by generative pre-training.
  * Raue et al. (2009) Andreas Raue, Clemens Kreutz, Thomas Maiwald, Julie Bachmann, Marcel Schilling, Ursula Klingmüller, and Jens Timmer. 2009.  Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood.  _Bioinformatics_ , 25(15):1923–1929.
  * Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013.  Recursive deep models for semantic compositionality over a sentiment treebank.  In _Proceedings of the 2013 conference on empirical methods in natural language processing_ , pages 1631–1642.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in neural information processing systems_ , pages 5998–6008.
  * Voorhees and Tice (2000) Ellen M. Voorhees and Dawn M. Tice. 2000.  The TREC-8 question answering track.  In _Proceedings of the Second International Conference on Language Resources and Evaluation (LREC’00)_ , Athens, Greece. European Language Resources Association (ELRA).
  * Wiegreffe and Pinter (2019) Sarah Wiegreffe and Yuval Pinter. 2019.  Attention is not not explanation.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 11–20.
  * Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  _arXiv preprint arXiv:1509.01626_.

##  Appendix A Background on Matrices

###  A.1 Span, Column space and Row space

Given a set of vectors V
$\coloneqq\\{\text{{v}}_{1},\text{{v}}_{2},\ldots,\text{{v}}_{n}\\}$, the span
of V, $\operatorname{span}(\text{{V}})$, is defined as the set obtained from
all the possible linear combination of vectors in V, i.e.,

$\operatorname{span}(\text{{V}})\coloneqq\\{\sum_{i=1}^{n}\lambda_{i}\text{{v}}_{i}\mid\lambda_{i}\in\mathbb{R},\;i\in\\{1,2,\ldots,n\\}\\}.$

The $\operatorname{span}$(V) can also be seen as the smallest vector space
that contains the set V.

Given a matrix $\operatorname{\textbf{A}}$ $\in\mathbb{R}^{m\times n}$, the
column space of $\operatorname{\textbf{A}}$,
$\operatorname{Cs}(\operatorname{\textbf{A}})$, is defined as space spanned by
its column vectors. Similarly, the row space of $\operatorname{\textbf{A}}$,
$\operatorname{Rs}(\operatorname{\textbf{A}})$, is the space spanned by the
row vectors of $\operatorname{\textbf{A}}$.
$\operatorname{Cs}(\operatorname{\textbf{A}})$ and
$\operatorname{Rs}(\operatorname{\textbf{A}})$ are the subspaces of the real
spaces $\operatorname{\mathbb{R}}^{m}$ and $\operatorname{\mathbb{R}}^{n}$,
respectively. If the row vectors of $\operatorname{\textbf{A}}$ are linearly
independent, the $\operatorname{Rs}(\operatorname{\textbf{A}})$ will span
$\operatorname{\mathbb{R}}^{m}$. A similar argument holds between
$\operatorname{Cs}(\operatorname{\textbf{A}})$ and
$\operatorname{\mathbb{R}}^{n}$.

###  A.2 Matrix Rank

The rank of a matrix $\operatorname{\textbf{P}}$ (denoted as
$\operatorname{rank}(\operatorname{\textbf{P}}))$ tells about the dimensions
of the space spanned by the row vectors or column vectors. It can also be seen
as the number of linearly independent rows or columns. The following
properties hold

$\displaystyle\operatorname{rank}\big{(}\operatorname{\textbf{P}}\big{)}$
$\displaystyle\leq\operatorname{min}\Big{(}m_{p},n_{p}\Big{)}$
$\displaystyle\operatorname{rank}\big{(}\operatorname{\textbf{P}}\operatorname{\textbf{Q}}\big{)}$
$\displaystyle\leq\operatorname{min}\Big{(}\operatorname{rank}(\operatorname{\textbf{P}}),\operatorname{rank}(\operatorname{\textbf{Q}})\Big{)}.$

Where, $\operatorname{\textbf{P}}$ and $\operatorname{\textbf{Q}}$ are
$m_{p}\times n_{p}$ and $m_{q}\times n_{q}$ dimensional matrices,
respectively.

###  A.3 Null Space

The left null space of a $m_{p}\times n_{p}$ matrix
$\operatorname{\textbf{P}}$ can be defined as the set of vectors v -

$\operatorname{LN}\big{(}\operatorname{\textbf{P}}\big{)}=\\{\text{{v}}^{T}\
\in\operatorname{\mathbb{R}}^{1\times
m_{p}}\mid\text{{v}}^{T}\operatorname{\textbf{P}}=0\\}$ (10)

If the rows of $\operatorname{\textbf{P}}$ are linearly independent
($\operatorname{\textbf{P}}$ is full-row rank) the left null space of
$\operatorname{\textbf{P}}$ is zero dimensional. The only solution to the
system of equations $\text{{v}}\operatorname{\textbf{P}}=0$ is trivial, i.e.,
v=0. The dimensions of the null space, known as nullity, of
$\operatorname{\textbf{P}}$ can be calculated as

$\dim\big{(}\operatorname{LN}(\operatorname{\textbf{P}})\big{)}=m_{p}-\operatorname{rank}(\operatorname{\textbf{P}}).$
(11)

The nullity of $\operatorname{\textbf{P}}$ sets the dimensions of the space v
lies in. In section 3, we utilize our knowledge of section A.2 and section A.3
to analyse identifiability in a Transformer.


# THz intersubband electroluminescence from n-type Ge/SiGe
quantum cascade structures

David Stark  Institute for Quantum Electronics, Department of Physics, ETH
Zürich, Switzerland  Muhammad Mirza  James Watt School of Engineering,
University of Glasgow, Glasgow G12 8LT, United Kingdom  Luca Persichetti
Michele Montanari  Dipartimento di Scienze, Universita Roma Tre, Roma 00146,
Italy  Sergej Markmann  Mattias Beck  Institute for Quantum Electronics,
Department of Physics, ETH Zürich, Switzerland  Thomas Grange  Stefan Birner
nextnano GmbH, Konrad-Zuse-Platz 8, München 81829, Germany  Michele Virgilio
Dipartimento di Fisica “E. Fermi,” Universita di Pisa, Pisa 56127, Italy
Chiara Ciano  Dipartimento di Scienze, Universita Roma Tre, Roma 00146, Italy
Michele Ortolani  Sapienza University of Rome, Department of Physics, Piazzale
Aldo Moro 2, I-00185 Rome, Italy  Cedric Corley  IHP - Leibniz-Institut für
innovative Mikroelektronik, Im Technologiepark 25, D-15236 Frankfurt (Oder)
Germany  Giovanni Capellini  Dipartimento di Scienze, Universita Roma Tre,
Roma 00146, Italy  IHP - Leibniz-Institut für innovative Mikroelektronik, Im
Technologiepark 25, D-15236 Frankfurt (Oder) Germany  Luciana Di Gaspare
Monica De Seta  Dipartimento di Scienze, Universita Roma Tre, Roma 00146,
Italy  Douglas J. Paul  James Watt School of Engineering, University of
Glasgow, Glasgow G12 8LT, United Kingdom  Jérôme Faist  Giacomo Scalari
Institute for Quantum Electronics, Department of Physics, ETH Zürich,
Switzerland<EMAIL_ADDRESS>
###### Abstract

We report electroluminescence originating from L-valley transitions in n-type
Ge/Si0.15Ge0.85 quantum cascade structures centered at 3.4 and 4.9 THz with a
line broadening of $\Delta f/f\approx 0.2$. Three strain-compensated
heterostructures, grown on a Si substrate by ultrahigh vacuum chemical vapor
deposition, have been investigated. The design is based on a single quantum
well active region employing a vertical optical transition and the observed
spectral features are well described by non-equilibrium Green’s function
calculations. The presence of two peaks highlights a suboptimal injection in
the upper state of the radiative transition. Comparison of the
electroluminescence spectra with similar GaAs/AlGaAs structure yields one
order of magnitude lower emission efficiency.

††preprint: AIP/123-QED

Since the demonstration of the semiconductor diode laser and the high
popularity of Si-based transistor technology, a laser on silicon constitutes a
long-standing goal for silicon photonics Liang and Bowers (2010). Significant
advantages for a Si based laser should result from the high yield
manufacturing processes to allow low cost at high volume but also enable low
cost photonic systems from photonic integrated circuits. The major problem to
realize a Si-based interband laser is the indirect band gap of group IV
materials, which leads to a poor radiative recombination rate. Several
solutions have been developed to achieve laser action from silicon Boyraz and
Jalali (2004); Armand Pilon  _et al._ (2019); Wan  _et al._ (2019); Seifried
_et al._ (2018); Nguyen-Van  _et al._ (2018). Employing intersubband
transitions in quantum cascade structures represent an exciting option because
such transitions are independent of the nature of the band gap.

So far the Quantum Cascade Laser (QCL)Faist  _et al._ (1994) has only been
demonstrated in polar III-V compound semiconductor materials based on
transitions between conduction band states. Intersubband electroluminescence,
however, due to valence band transitions from non-polar Si/SiGe
heterostructures have been observed for mid-infrared Dehlinger  _et al._
(2000) and far-infrared wavelengths Lynch  _et al._ (2002); Paul (2010). The
strained structure produces a complex valence band and the large effective
mass ($\sim 0.3m_{0}$Matmon  _et al._ (2010), where $m_{0}$ is the free
electron mass) results in poor gain and therefore no subsequent laser action
has been demonstrated. Theoretically n-type Ge/SiGe and Ge/GeSiSn material
configurations with lower effective mass ($\sim 0.135m_{0}$) are predicted to
be promising candidates to realize a room temperature THz QCL Driscoll and
Paiella (2006, 2007); Sun  _et al._ (2007); Valavanis  _et al._ (2011). Due to
the absence of the restrahlen band, lasing transitions above 6 THz should be
accessible. Because of the large lattice mismatch between Si and Ge, the
growth of such Ge-rich structures on Si wafers is particularly challenging
Paul (2010). Only in the last few years the Ge/SiGe heterostructures reached
the quality standard required for this kind of applicationGrange  _et al._
(2020).

While buried InGaAs/InAlAs QCLsBeck (2002), operational in continuous wave and
at room-temperature, de facto unlocked the mid-IR spectral region in different
fields Vitiello  _et al._ (2015); Galli  _et al._ (2011); Klocke  _et al._
(2018), THz QCLs remain limited to operation below room temperatures hindering
potential applications. Pulsed Peltier-cooled operation of GaAs/AlGaAs THz
QCLs was achieved in 2019 Bosco  _et al._ (2019) and recently demonstrated up
to 250 K Khalatpour  _et al._ (2020). The values of current density and
dissipated power in such devices still prevent integration into more complex
photonic structures. The high current density, high voltage drop per period
and the temperature dependence of the subband lifetimes are intrinsically
related to the polar nature of the gain material. Scattering of electrons with
LO phonons ultimately limits the population inversion in such devices.

We recently proposed a Ge/SiGe quantum cascade design where non-equilibrium
Green’s functions (NEGF) based calculations predict gain up to room
temperature Grange  _et al._ (2019). The design is based on a 4-quantum well
active region employing a bound-to-continuum transition and involves a total
of four subband states. Although such a design approach holds good promise for
the demonstration of a laser, it is not the ideal candidate to develop a new
THz quantum cascade emitter. The diagonal optical transition leads to broad
emission Blaser  _et al._ (2000) and a high voltage drop per period .

Figure 1:  (a) Conduction band diagram and Wannier-Stark states for the target
SQW design at 22 mV/period computed by NEGF. The solid black and solid grey
lines are the potentials in the L- and $\Delta_{2}$-valleys, respectively. The
orange scale shows the position and energy resolved current density at 10 K.
The integrated current density is 37 A/$\text{cm}^{2}$. The nominal period
length is 82.4 nm with a sheet doping density of $2.54\times
10^{10}\text{\,cm}^{-2}$. Starting from the injection barrier the nominal
layer sequence with thicknesses in nanometers is
4.7/19.9/3/13.9/3/11.6/3/3.8/3/3.8/3/9.7. The Ge wells are in standard font,
the $\text{Si}_{0.15}\text{Ge}_{0.85}$ barriers are in bold and the phosphorus
doped layer is underlined. (b) Reciprocal space map acquired on sample 2306
around asymmetric ($4\bar{2}\bar{2}$) Si reflections. (c) Comparison of the
measured and simulated (004) XRD rocking curve. The simulated curve is shifted
for clarity. (d) STEM image at different magnifications of a SQW
heterostructure. The SiGe barriers appear darker. (e) SEM image and schematic
diagram of the Ge/SiGe interdigitated diffraction grating. The schematic
diagram shows the cross-section of a single grating finger and its bias
configuration for the regular growth direction. If the growth direction is
reversed the polarity is switched.

To unambiguously demonstrate electroluminescence from a Ge/SiGe quantum
cascade structure, we adapted the GaAs/AlGaAs single quantum well (SQW) design
reported in Ref Scalari  _et al._ (2007). SQW active regions are not expected
to show high optical gain. Instead, the low current density together with the
moderate energy drop per period lead to reduced heating of the device. Hence,
the unwanted blackbody emission can be reduced. The narrow spectral peak of
the vertical intersubband transition should result in a clear signature in the
spectrum. In this work, similar GaAs/AlGaAs structures with the same expected
emission energy Rochat  _et al._ (1998) are used for a quantitative benchmark
comparison with the Ge/SiGe results.

The conduction band profile in the L-valley and energy resolved current
density of the target design is shown in Fig. 1(a). The main vertical
transition between states $|2\rangle$ and $|1\rangle$ is expected at an energy
of 14.5 meV at 22 mV/period with a current density of 37 A/$\text{cm}^{2}$.
The NEGF calculations are performed with the nextnano.NEGF simulation package
Grange (2014, 2015). The effective mass along the confinement direction of
$m^{*}=0.135m_{0}$ is used Gupta  _et al._ (2018); Ciano  _et al._ (2019). In
this strain-compensated design we employ tensile-strained
$\text{Si}_{0.15}\text{Ge}_{0.85}$ barriers. Such a Si-content leads to a
smaller confinement offset in the L-valley $\Delta
E_{\text{L}}=80\;\text{meV}$ compared to the
$\text{GaAs}/\text{Al}_{0.15}\text{Ga}_{0.85}\text{As}$ material system with
$\Delta E_{\Gamma}=125\;\text{meV}$ Yi  _et al._ (2010). This low Si content
separates energetically the parasitic $\Delta_{2}$-valley states from the
L-valley states to avoid potential leakage or charge trapping by the barriers.
Note that the simulation model accounts for interdiffused Ge/SiGe interfaces.
The interface properties are based on atom probe tomography measurements
Grange  _et al._ (2020). Here the interface roughness (root mean square)
$\Delta=0.11$ nm and the in-plane correlation length $\Lambda_{||}=7.0$ nm are
used.

Table 1:  Overview of the quantum cascade structures: Sample ID, nominal sheet doping density $n_{\text{2D}}$, period length $L_{\text{p}}$ measured by XRD and the number of periods $N_{\text{p}}$. Sample ID | $n_{\text{2D}}\big{(}10^{10}\text{cm}^{-2}\big{)}$ | $L_{\text{p}}\big{(}\text{nm}\big{)}$ | $N_{\text{p}}$
---|---|---|---
2306 | 5 | 85.4 | 51
2307 | 7 | 83.3 | 51
2315111reverse growth direction | 7 | 87.8 | 51
S1353222reference GaAs/$\text{Al}_{0.15}\text{Ga}_{0.85}\text{As}$ sample | 2.9 | 97.7 | 35

The strain-compensated structures were grown on $\text{Si}_{1-x}\text{Ge}_{x}$
reverse graded virtual substrate (RGVS) on Ge/Si(001) substrates by ultrahigh
vacuum chemical vapour deposition (UHV-CVD) Seta  _et al._ (2012); Grange  _et
al._ (2020); Persichetti  _et al._ (2020). The RGVS is 3.7 $\mu\text{m}$ thick
where the last 1.3 $\mu\text{m}$ is a constant composition
$\text{Si}_{0.03}\text{Ge}_{0.97}$ buffer. The quantum cascade periods are
embedded between a 400 nm bottom- and a 25 nm top-contact. Both contacts
consist of phosphorous doped $\text{Si}_{\text{0.03}}\text{Ge}_{0.97}$ with
$N_{\text{D}}\approx 2\times 10^{19}\;\text{cm}^{-3}$. The active region
consists of 51 periods of the SQW region design (see Fig. 1(a)) for a total
thickness of $4.2\;\mu$m. The main sample parameters including the GaAs/AlGaAs
reference discussed in this work are listed in Table 1. The
($4\bar{2}\bar{2}$) x-ray diffraction (XRD) reciprocal space map of sample
2306 depicted in Fig. 1(b) shows the lattice matching of the active region to
the $\text{Si}_{\text{0.03}}\text{Ge}_{0.97}$ buffer as evidenced by the
alignment of the superlattice peaks with the
$\text{Si}_{\text{0.03}}\text{Ge}_{0.97}$ signal along the vertical dotted
line. The active region period length $L_{\text{p}}$ and its reproducibility
is derived from 1D XRD rocking curves, see Fig. 1(c). The scanning
transmission electron microscopy (STEM) in Fig. 1(d) shows the quality of the
UHV-CVD growth. The threading dislocation density is estimated to be on the
order of $10^{6}\;\text{cm}^{-2}$, a state-of-the-art value for this material
system Skibitzki  _et al._ (2020). As the threading dislocations act as
acceptor-like states, the phosphorous density in the Ge/SiGe structures is
larger than the free carrier concentration targeted at the design stage in
Fig. 1(a) cla (2009). Moreover, it is not well known how many carriers per
unit length are trapped by a threading dislocation, thus the active carrier
concentration is affected by some uncertainty.

The samples were processed into deeply etched diffraction gratings with
linearly chirped periods (23 - 28 $\mu$m). The top and bottom metallizations
are interdigitated to uniformly pump the device area. To reduce the amount of
defects per device, the area of the etched mesa is kept small (5.5% of the
focal spot of the detector which is $450\times 450\;\mu\text{m}^{2}$). A SEM
image of the processed device and the schematic diagram of the cross-section
of the device are illustrated in Fig. 1(e). The silicon based samples were
etched using inductively coupled plasma (ICP) etching with
SF${}_{\text{6}}$/C${}_{\text{4}}$F${}_{\text{8}}$ chemistry Mirza  _et al._
(2012). Ohmic contacts were formed on the highly doped semiconductor material
using deposited Ni metal annealed at 330 ∘C for 30 s to form NiGe Gallacher
_et al._ (2012). The linearity of the contact resistances were evaluated with
circular transmission line test structures at 10 K (see Supplementary
material). The GaAs/AlGaAs reference structure was processed into gratings
with identical dimensions. It was etched using ICP with
SiCl${}_{\text{4}}$/N${}_{\text{2}}$ chemistry and metallized with Ge/Au/Ni/Au
serving as well as a self aligned etch mask.

Electroluminescence measurements were performed in a home-made, vacuum Fourier
transform infrared (FTIR) spectrometer equipped with a He-cooled Si bolometer
(IR Labs). The samples were mounted in a He flow cryostat. The heat sink
temperature was kept at 5 K. The setup was pre-aligned and phase calibrated
using a vertically emitting single mode THz QCL Bosco  _et al._ (2016) driven
below threshold. Current was injected following a micro-macro pulse scheme to
minimize heating effects. Generally, 627 micro-pulses of 960 ns length
(equivalent to a duty-cyle of $D=50\%$) with macro-pulse frequency of 415 Hz
are used. The interferograms were acquired in step-scan mode with a lock-in
amplifier with 2 s integration time per point and a spectral resolution of
$\Delta E=0.82$ meV (198 GHz). Multiple interferograms with the same
experimental conditions were averaged yielding typically 8-24 hours of total
integration time. The stabilities of the in-phase component, quadrature
component and the heat sink temperature were verified continuously during the
acquisition time.

The voltage-current (VI) and electroluminescence intensity-current (LI)
characteristics measured at 5 K for the Ge/SiGe samples and the GaAs/AlGaAs
reference sample are visible in Fig. 2 (VI curves as a function of the
temperature up to 290 K are reported in the Supplementary Material). The
observed Ge/SiGe VI curves are compared to the NEGF simulation for sample
2307; the measured current density exceeds the simulated one by a factor of 5,
that is consistent with theoretical lifetimes predictions, as discussed futher
below. Comparing the VI curves of the Ge/SiGe samples to the GaAs reference
measurement, the current density for the same applied electric field is one
order of magnitude larger. Additionally, the transport of the Ge/SiGe samples
does not show signs of negative differential resistance (NDR) as clearly
observed in the reference sample at a current density of 42
$\text{A}/\text{cm}^{2}$. From Fig. 2(a), it can be seen that the spectra of
the Ge/SiGe samples are acquired at low injection currents (< 200 mA or 1.8
kA/$\text{cm}^{2}$) to prevent any blackbody emission overwhelming the
intersubband signal.

Figure 2: Light-current-voltage (LIV) characteristics at $T=5\;\text{K}$ for
the Ge/SiGe samples (a) and the GaAs/AlGaAs reference sample (b). The solid
and dashed lines are the VI- and LI-curves respectively. The dots indicate the
operation points where electroluminescence spectra are acquired. The dotted
line in (a) is the VI curve computed by NEGF for sample 2307, where the
current is multiplied by 5. Note that for the reference measurement a micro-
pulse duty-cycle of $D=95\%$ was used. Figure 3: Electroluminescence spectra
of the three Ge/SiGe samples (a), (b) and (c). The dotted lines correspond to
the simulated peak positions. (c) The gray dashed line shows the theoretically
calculated electroluminescence spectrum. (d) The simulated peak positions are
compared with the measured peak positions. (e) Measured FWHMs and integrated
optical peak power as function the period length. (f) GaAs/AlGaAs reference
spectrum with a transition energy of 14.8 meV, FWHM of 0.7 meV and integrated
peak power of 2.5 pW. Note that this measurement is acquired with spectral
resolution $\Delta E=0.24$ meV and micro-pulse duty-cycle $D=95\%$.

Typical spectra measured on the three Ge/SiGe samples are shown in Fig.
3(a)-(c). Two peaks centered around 13 meV and 22 meV appear distinctly from
the spectra. We identify the first peak as the vertical transition between
states $|2\rangle$ and $|1\rangle$, see Fig. 1(a). We attribute the second
peak to a transition from a higher lying energy state $|\text{p}\rangle$ to
the upper state of the miniband $|\text{u}\rangle$, which extracts the
electrons from the SQW. In Fig. 3(c) we report as well the predicted
electroluminescence signal using NEGF. Two peaks are predicted, in fair
agreement with the measurements: the presence of the second peak highlights
the role of hot electrons due to the absence of strong inelastic scattering
(LO phonons). The measured spectra are fitted to a double Lorentzian function
and the extracted transition energies are compared to the simulated peak
positions as a function of the period length, see Fig. 3(d). The measurement
and simulations agree to within 2 meV. Note that in simulations we account for
the different period lengths obtained by XRD and the nominal doping density is
used (see Table 1). The small discrepancy of the peak positions may be due to
some uncertainty related to the material parameters of the Ge/SiGe system
(band-offset and electron effective mass), as well as to the exact knowledge
of the actual doping profile. In Fig. 3(e) the extracted FWHM and the
integrated peak power of the individual transitions are shown. The observed
FWHMs agree with optical absorption measurements reported in Ref Virgilio  _et
al._ (2014); Ciano  _et al._ (2019). Considering sample 2306 the line
broadening for both peaks is $\Delta f/f\approx 0.2$. The FWHM of the
$|2\rangle$ to $|1\rangle$ transition is 2.5 meV, which is 1 meV larger than
what is predicted by NEGF. In the context of the Kazarinov-Suris density
matrix model for transport in quantum cascade structures Kazarinov and Suris
(1972); Sirtori  _et al._ (1998), this linewidth corresponds to a dephasing
lifetime $T_{2}\simeq 0.3$ ps. It has to be mentioned that a control sample
with the doping placed homogeneously over four quantum wells of the injector
and processed in identical grating has given, as expected, a low intensity,
broad signal with no significant spectral features. Such measurements and
additional reverse bias measurements are reported in the Supplementary
Material.

The spectrum of the GaAs/AlGaAs reference sample is reported in Fig. 3(f). It
is acquired at the maximum current value before the NDR. The peak is centered
at 14.8 meV with a FWHM of 0.7 meV ($\Delta f/f\approx 0.05)$ and integrated
peak power of 2.5 pW. Only one peak appears due to proper electronic injection
into the upper state $|2\rangle$ and an accurate doping level. However, above
the NDR two peaks have been reported due to broad injection into high-energy
states Rochat  _et al._ (1998), where the corresponding spectrum resembles the
spectra of the Ge/SiGe samples. The smaller confinement offset in our
$\text{Si}_{0.15}\text{Ge}_{0.85}$ design energetically lower the parasitic
state $|\text{p}\rangle$. The weaker electron-phonon interaction in non-polar
Ge/SiGe active region leads to a larger electronic excess temperature Grange
_et al._ (2019); Bagolini  _et al._ (2020). Thus, the transport through higher
lying energy states is non negligible and leads to a second optically active
transition. Additionally, the doping level in the active region and the
contacts of our structures are not optimized. This could cause a non-
homogeneous electric field across the 51 periods of the cascade and ultimately
impair the electronic injection.

In the following the non-radiative lifetimes of the upper states $|2\rangle$
and $|p\rangle$ are estimated, considering the spectrum of sample 2315 and
using the GaAs reference to extract the collection efficiency. The
electroluminescence power can be written as

$\displaystyle
P_{\text{opt}}=\eta_{\text{inj}}\eta_{\text{coll}}\eta_{\text{rad}}\frac{h\nu}{e}N_{\text{p}}D\cdot
I,$ (1)

where $\eta_{\text{coll}}$ is the collection efficiency for the setup and the
grating, $\eta_{\text{rad}}\approx\tau_{\text{nr}}/\tau_{\text{sp}}$ the
radiative efficiency, $h\nu$ the photon energy, $D$ the duty-cycle, $I$ the
current and $e$ the absolute charge of an electron. In the case of GaAs, we
assume unitary current injection $\eta_{\text{inj}}=1$ (NDR well visible), we
use $\tau_{\text{nr}}=12.2\;\text{ps}$Scalari  _et al._ (2007) and, with the
computed spontanoeus emission lifetime $\tau_{\text{sp}}=7.4\;\mu\text{s}$ we
can extract a collection efficiency of $\eta_{\text{coll}}\approx 7\times
10^{-4}$. For the Ge/SiGe structure we assume that the total power is given by
sum of the two peak powers (see supplementary material). The injection
efficiencies into states $|2\rangle$ and $|p\rangle$ computed with NEGF
simulations are $\eta_{\text{inj,2}}=0.38$ and $\eta_{\text{inj,p}}=0.37$.
Using the computed spontaneous emission lifetimes
$\tau_{\text{sp,2}}=15.9\;\mu$s and $\tau_{\text{sp,p}}=7.1\;\mu$s, we obtain
$\tau_{\text{nr,2}}=1.9$ ps and $\tau_{\text{nr,p}}=1.3$ ps. Consequently, the
radiative efficiency is $\eta_{\text{rad}}\approx 10^{-7}$. The lifetimes
predicted by NEGF modelling are $\tau_{\text{nr,2}}^{\text{sim}}=12$ ps and
$\tau_{\text{nr,p}}^{\text{sim}}=6$ ps, roughly a factor of 5 higher. We
stress that the deduced values for the lifetimes constitute a lower limit; the
actual injection efficiency is not known due to the presence of threading
dislocations and, as discussed above, to possible inhomogeneities in the
electric field across the whole sample length. Nevertheless, a lower value of
$\tau_{\text{nr,2}}$ with respect to the NEGF simulation may also result from
the larger effective doping density adopted and the higher electronic
temperature. In fact, NEGF simulations suggest that Coulomb scattering
prevails over interface roughness in the determination of the lifetimes and of
the linewidth.

In conclusion, THz intersubband electroluminescence from n-type
Ge/$\text{Si}_{0.15}\text{Ge}_{0.85}$ quantum cascade structures has been
demonstrated from three different epitaxial layers. The spectral features
agree well with theoretical predictions based on the non-equilibrium Green’s
function simulations. The emitters have been benchmarked against a similar
GaAs/AlGaAs structure with identical device geometry. The Ge/SiGe emission
efficiency is one order of magnitude lower. This is attributed to a suboptimal
injection of the electrons into the upper state of the radiative transitions
and to a lower upper state lifetime. Future improvements will come from a more
accurate control of the doping profile, a better understanding of its
interaction with the threading dislocations and from a further reduction of
their density. Finally it would be useful to clarify whether the
$\Delta_{2}$-states confined in the barriers influence the electronics
dynamics. To achieve laser action, we will leverage on the progress made in
the material growth and structure modelling by employing higher gain
structures based on 4-quantum wells Grange  _et al._ (2019) featuring a
diagonal optical transition and embedded in double metal waveguides Gallacher
_et al._ (2020).

This project was funded by the European Union’s Horizon 2020 research and
innovation programme under Grant Agreement No. 766719 (FLASH) and European
Research Council Consolidator Grant (724344) (CHIC). The data that support the
findings of this study are available from the corresponding author upon
reasonable request. The authors would like to thank Andres Forrer, Bo Meng and
Hans Sigg for useful discussions and support.

## References

  * Liang and Bowers (2010) D. Liang and J. E. Bowers, “Recent progress in lasers on silicon,” Nature Photonics 4, 511–517 (2010).
  * Boyraz and Jalali (2004) O. Boyraz and B. Jalali, “Demonstration of a silicon Raman laser,” Optics Express 12, 5269–5273 (2004).
  * Armand Pilon  _et al._ (2019) F. T. Armand Pilon, A. Lyasota, Y.-M. Niquet, V. Reboud, V. Calvo, N. Pauc, J. Widiez, C. Bonzon, J. M. Hartmann, A. Chelnokov, J. Faist, and H. Sigg, “Lasing in strained germanium microbridges,” Nature Communications 10, 2724 (2019).
  * Wan  _et al._ (2019) Y. Wan, S. Zhang, J. C. Norman, M. J. Kennedy, W. He, S. Liu, C. Xiang, C. Shang, J.-J. He, A. C. Gossard, and J. E. Bowers, “Tunable quantum dot lasers grown directly on silicon,” Optica 6, 1394 (2019).
  * Seifried  _et al._ (2018) M. Seifried, G. Villares, Y. Baumgartner, H. Hahn, M. Halter, F. Horst, D. Caimi, C. Caër, M. Sousa, R. F. Dangel, L. Czornomaz, and B. J. Offrein, “Monolithically Integrated CMOS-Compatible III–V on Silicon Lasers,” IEEE Journal of Selected Topics in Quantum Electronics 24, 1–9 (2018).
  * Nguyen-Van  _et al._ (2018) H. Nguyen-Van, A. N. Baranov, Z. Loghmari, L. Cerutti, J.-B. Rodriguez, J. Tournet, G. Narcy, G. Boissier, G. Patriarche, M. Bahriz, E. Tournié, and R. Teissier, “Quantum cascade lasers grown on silicon,” Scientific Reports 8, 7206 (2018).
  * Faist  _et al._ (1994) J. Faist, F. Capasso, D. L. Sivco, C. Sirtori, A. L. Hutchinson, and A. Y. Cho, “Quantum Cascade Laser,” Science 264, 553–556 (1994).
  * Dehlinger  _et al._ (2000) G. Dehlinger, L. Diehl, U. Gennser, H. Sigg, J. Faist, K. Ensslin, D. Grützmacher, and E. Müller, “Intersubband Electroluminescence from Silicon-Based Quantum Cascade Structures,” Science 290, 2277–2280 (2000).
  * Lynch  _et al._ (2002) S. A. Lynch, R. Bates, D. J. Paul, D. J. Norris, A. G. Cullis, Z. Ikonic, R. W. Kelsall, P. Harrison, D. D. Arnone, and C. R. Pidgeon, “Intersubband electroluminescence from Si/SiGe cascade emitters at terahertz frequencies,” Applied Physics Letters 81, 1543–1545 (2002).
  * Paul (2010) D. J. Paul, “The progress towards terahertz quantum cascade lasers on silicon substrates,” Laser & Photonics Reviews 4, 610–632 (2010).
  * Matmon  _et al._ (2010) G. Matmon, D. J. Paul, L. Lever, M. Califano, Z. Ikonić, R. W. Kelsall, J. Zhang, D. Chrastina, G. Isella, H. von Känel, E. Müller, and A. Neels, “Si/SiGe quantum cascade superlattice designs for terahertz emission,” Journal of Applied Physics 107, 053109 (2010).
  * Driscoll and Paiella (2006) K. Driscoll and R. Paiella, “Silicon-based injection lasers using electronic intersubband transitions in the L valleys,” Applied Physics Letters 89, 191110 (2006).
  * Driscoll and Paiella (2007) K. Driscoll and R. Paiella, “Design of n-type silicon-based quantum cascade lasers for terahertz light emission,” Journal of Applied Physics 102, 093103 (2007).
  * Sun  _et al._ (2007) G. Sun, H. H. Cheng, J. Menéndez, J. B. Khurgin, and R. A. Soref, “Strain-free Ge/GeSiSn quantum cascade lasers based on L-valley intersubband transitions,” Applied Physics Letters 90, 251105 (2007).
  * Valavanis  _et al._ (2011) A. Valavanis, T. V. Dinh, L. J. M. Lever, Z. Ikonić, and R. W. Kelsall, “Material configurations for n-type silicon-based terahertz quantum cascade lasers,” Physical Review B 83, 195321 (2011).
  * Grange  _et al._ (2020) T. Grange, S. Mukherjee, G. Capellini, M. Montanari, L. Persichetti, L. Di Gaspare, S. Birner, A. Attiaoui, O. Moutanabbir, M. Virgilio, and M. De Seta, “Atomic-Scale Insights into Semiconductor Heterostructures: From Experimental Three-Dimensional Analysis of the Interface to a Generalized Theory of Interfacial Roughness Scattering,” Physical Review Applied 13, 044062 (2020).
  * Beck (2002) M. Beck, “Continuous Wave Operation of a Mid-Infrared Semiconductor Laser at Room Temperature,” Science 295, 301–305 (2002).
  * Vitiello  _et al._ (2015) M. S. Vitiello, G. Scalari, B. Williams, and P. D. Natale, “Quantum cascade lasers: 20 years of challenges,” Optics Express 23, 5167–5182 (2015).
  * Galli  _et al._ (2011) I. Galli, S. Bartalini, S. Borri, P. Cancio, D. Mazzotti, P. De Natale, and G. Giusfredi, “Molecular Gas Sensing Below Parts Per Trillion: Radiocarbon-Dioxide Optical Detection,” Physical Review Letters 107, 270802 (2011).
  * Klocke  _et al._ (2018) J. L. Klocke, M. Mangold, P. Allmendinger, A. Hugi, M. Geiser, P. Jouy, J. Faist, and T. Kottke, “Single-Shot Sub-microsecond Mid-infrared Spectroscopy on Protein Reactions with Quantum Cascade Laser Frequency Combs,” Analytical Chemistry 90, 10494–10500 (2018).
  * Bosco  _et al._ (2019) L. Bosco, M. Franckié, G. Scalari, M. Beck, A. Wacker, and J. Faist, “Thermoelectrically cooled THz quantum cascade laser operating up to 210 K,” Applied Physics Letters 115, 010601 (2019).
  * Khalatpour  _et al._ (2020) A. Khalatpour, A. K. Paulsen, C. Deimert, Z. R. Wasilewski, and Q. Hu, “High-power portable terahertz laser systems,” Nature Photonics , 1–5 (2020).
  * Grange  _et al._ (2019) T. Grange, D. Stark, G. Scalari, J. Faist, L. Persichetti, L. Di Gaspare, M. De Seta, M. Ortolani, D. J. Paul, G. Capellini, S. Birner, and M. Virgilio, “Room temperature operation of n-type Ge/SiGe terahertz quantum cascade lasers predicted by non-equilibrium Green’s functions,” Applied Physics Letters 114, 111102 (2019).
  * Blaser  _et al._ (2000) S. Blaser, M. Rochat, M. Beck, J. Faist, and U. Oesterle, “Far-infrared emission and Stark-cyclotron resonances in a quantum-cascade structure based on photon-assisted tunneling transition,” Physical Review B 61, 8369–8374 (2000).
  * Scalari  _et al._ (2007) G. Scalari, R. Terazzi, M. Giovannini, N. Hoyler, and J. Faist, “Population inversion by resonant tunneling in quantum wells,” Applied Physics Letters 91, 032103 (2007).
  * Rochat  _et al._ (1998) M. Rochat, J. Faist, M. Beck, U. Oesterle, and M. Ilegems, “Far-infrared (=88µm) electroluminescence in a quantum cascade structure,” Applied Physics Letters 73, 3724–3726 (1998).
  * Grange (2014) T. Grange, “Electron transport in quantum wire superlattices,” Physical Review B 89, 165310 (2014).
  * Grange (2015) T. Grange, “Contrasting influence of charged impurities on transport and gain in terahertz quantum cascade lasers,” Physical Review B 92, 241306 (2015).
  * Gupta  _et al._ (2018) S. Gupta, D. Nam, J. Vuckovic, and K. Saraswat, “Room temperature lasing unraveled by a strong resonance between gain and parasitic absorption in uniaxially strained germanium,” Physical Review B 97, 155127 (2018).
  * Ciano  _et al._ (2019) C. Ciano, M. Virgilio, M. Montanari, L. Persichetti, L. Di Gaspare, M. Ortolani, L. Baldassarre, M. Zoellner, O. Skibitzki, G. Scalari, J. Faist, D. Paul, M. Scuderi, G. Nicotra, T. Grange, S. Birner, G. Capellini, and M. De Seta, “Control of Electron-State Coupling in Asymmetric Ge/Si-Ge Quantum Wells,” Physical Review Applied 11, 014003 (2019).
  * Yi  _et al._ (2010) W. Yi, V. Narayanamurti, H. Lu, M. A. Scarpulla, and A. C. Gossard, “Probing semiconductor band structures and heterojunction interface properties with ballistic carrier emission: GaAs/AlxGa 1-xAs as a model system,” Physical Review B 81, 235325 (2010).
  * Seta  _et al._ (2012) M. D. Seta, G. Capellini, M. Ortolani, M. Virgilio, G. Grosso, G. Nicotra, and P. Zaumseil, “Narrow intersubband transitions in n-type Ge/SiGe multi-quantum wells: control of the terahertz absorption energy trough the temperature dependent depolarization shift,” Nanotechnology 23, 465708 (2012).
  * Persichetti  _et al._ (2020) L. Persichetti, M. Montanari, C. Ciano, L. Di Gaspare, M. Ortolani, L. Baldassarre, M. Zoellner, S. Mukherjee, O. Moutanabbir, G. Capellini, M. Virgilio, and M. De Seta, “Intersubband Transition Engineering in the Conduction Band of Asymmetric Coupled Ge/SiGe Quantum Wells,” Crystals 10, 179 (2020), number: 3 Publisher: Multidisciplinary Digital Publishing Institute.
  * Skibitzki  _et al._ (2020) O. Skibitzki, M. H. Zoellner, F. Rovaris, M. A. Schubert, Y. Yamamoto, L. Persichetti, L. Di Gaspare, M. De Seta, R. Gatti, F. Montalenti, and G. Capellini, “Reduction of threading dislocation density beyond the saturation limit by optimized reverse grading,” Physical Review Materials 4, 103403 (2020).
  * cla (2009) “Electrical and Optical Properties,” in  _Extended Defects in Germanium: Fundamental and Technological Aspects_, Springer Series in Materials Science, edited by C. Claeys and E. Simoen (Springer, Berlin, Heidelberg, 2009) pp. 65–136.
  * Mirza  _et al._ (2012) M. M. Mirza, H. Zhou, P. Velha, X. Li, K. E. Docherty, A. Samarelli, G. Ternent, and D. J. Paul, “Nanofabrication of high aspect ratio ($\sim$ 50:1) sub-10 nm silicon nanowires using inductively coupled plasma etching,” Journal of Vacuum Science & Technology B, Nanotechnology and Microelectronics: Materials, Processing, Measurement, and Phenomena 30, 06FF02 (2012).
  * Gallacher  _et al._ (2012) K. Gallacher, P. Velha, D. J. Paul, I. MacLaren, M. Myronov, and D. R. Leadley, “Ohmic contacts to n-type germanium with low specific contact resistivity,” Applied Physics Letters 100, 022113 (2012).
  * Bosco  _et al._ (2016) L. Bosco, C. Bonzon, K. Ohtani, M. Justen, M. Beck, and J. Faist, “A patch-array antenna single-mode low electrical dissipation continuous wave terahertz quantum cascade laser,” Applied Physics Letters 109, 201103 (2016).
  * Virgilio  _et al._ (2014) M. Virgilio, D. Sabbagh, M. Ortolani, L. Di Gaspare, G. Capellini, and M. De Seta, “Physical mechanisms of intersubband-absorption linewidth broadening in s -Ge/SiGe quantum wells,” Physical Review B 90, 155420 (2014).
  * Kazarinov and Suris (1972) R. Kazarinov and R. Suris, “Electric and electromagnetic properties of semiconductors with a superlattice,” Soviet Physics - Semiconductors 6, 120–134 (1972).
  * Sirtori  _et al._ (1998) C. Sirtori, F. Capasso, J. Faist, A. Hutchinson, D. Sivco, and A. Cho, “Resonant tunneling in quantum cascade lasers,” Ieee Journal Of Quantum Electronics 34, 1722–1729 (1998).
  * Bagolini  _et al._ (2020) L. Bagolini, M. Montanari, L. Persichetti, L. Di Gaspare, G. Capellini, M. Ortolani, M. De Seta, and M. Virgilio, “Disentangling elastic and inelastic scattering pathways in the intersubband electron dynamics of $n$-type Ge/SiGe quantum fountains,” Physical Review B 101, 245302 (2020), publisher: American Physical Society.
  * Gallacher  _et al._ (2020) K. Gallacher, M. Ortolani, K. Rew, C. Ciano, L. Baldassarre, M. Virgilio, G. Scalari, J. Faist, L. D. Gaspare, M. D. Seta, G. Capellini, G. Capellini, T. Grange, S. Birner, and D. J. Paul, “Design and simulation of losses in Ge/SiGe terahertz quantum cascade laser waveguides,” Optics Express 28, 4786–4800 (2020).