# Clustering Comparable Corpora of Russian and Ukrainian Academic Texts: Word
Embeddings and Semantic Fingerprints

###### Abstract

We present our experience in applying distributional semantics (neural word
embeddings) to the problem of representing and clustering documents in a
bilingual comparable corpus. Our data is a collection of Russian and Ukrainian
academic texts, for which topics are their academic fields. In order to build
language-independent semantic representations of these documents, we train
neural distributional models on monolingual corpora and learn the optimal
linear transformation of vectors from one language to another. The resulting
vectors are then used to produce ‘semantic fingerprints’ of documents, serving
as input to a clustering algorithm. The presented method is compared to
several baselines including ‘orthographic translation’ with Levenshtein edit
distance and outperforms them by a large margin. We also show that language-
independent ‘semantic fingerprints’ are superior to multi-lingual clustering
algorithms proposed in the previous work, at the same time requiring less
linguistic resources.

Keywords: word embeddings, text clustering, comparable corpora, academic
texts, cross-lingual transformations

languageresourceLanguage Resources

Clustering Comparable Corpora of Russian and Ukrainian Academic Texts: Word
Embeddings and Semantic Fingerprints

Andrey Kutuzov1, Mikhail Kopotev2, Tatyana Sviridenko3, Lyubov Ivanova3
---
University of Oslo1, University of Helsinki2, National Research University
Higher School of Economics3
Oslo1, Helsinki2, Moscow3
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

Abstract content

##  1\. Introduction

This research addresses the problem of representing the semantics of text
documents in multi-lingual comparable corpora. We present a new approach to
this problem, based on neural embeddings, and test it on the task of
clustering texts into meaningful classes depending on their topics. The
setting is unsupervised, meaning that one either does not have enough
annotated data to train a supervised classifier or does not want to be limited
with a pre-defined set of classes. There is a lot of sufficiently good
approaches to this problem in the case of mono-lingual text collections, but
the presence of multiple languages introduces complications.

When a text collection contains documents in several languages, it becomes
impractical to simply represent the documents as vectors of words occurring in
them ("bag-of-words"), as the words surface forms are different, even in
closely-related languages. Thus, one has to invent means to cross the inter-
lingual gap and bring all documents to some sort of shared representation,
without losing information about their topics or categories.

Of course, one obvious way to solve this problem is to translate all documents
into one language, and then apply any clustering algorithm. However, this
requires either buying human/machine translation services (which can be
expensive if you deal with large text collection) or training own statistical
machine translation model (which as a rule requires big parallel corpus). This
is the reason to search for other solutions.

In this paper, a novel way of reducing the problem of cross-lingual document
representation to a monolingual setting is proposed. Essentially, we train
Continuous Bag-of-Words models [Mikolov et al., 2013b] on large comparable
monolingual corpora for two languages our dataset consists of. This provides
us with vector representations of words, allowing to measure their semantic
similarity. Then, a linear transformation matrix from vectors of language A to
vectors of language B is learned, using a small bilingual dictionary as
training data. This matrix is then employed to ‘project’ word and document
representations from semantic space of language A to semantic space of
language B. It allows not only quite accurate ‘translation’ of words, but also
of document ‘semantic fingerprints’ (dense representations of document
semantics, calculated as an average of the trained distributional vectors for
all the words in document).

This approach is evaluated in a setting, where the input is a collection of
documents in several languages and some number of topics to which these
documents belong (we also have large monolingual corpora to train
distributional models on). For each document, we are given its language, but
not its topic. The task is to cluster this collection so that documents
belonging to one topic were clustered together, independent of their language.
Note that we are interested in clustering the collection as a whole, not each
language separately (which is trivial).

Our evaluation data consists of comparable corpora of Russian and Ukrainian
academic texts. On this material, we show that the ‘translated semantic
fingerprints’ method represents documents in different languages precisely
enough to allow almost exact clustering according to document topics, with
only 5% of incorrect assignments. It significantly outperforms both naive bag-
of-words baseline and the not-so-naive method of ‘orthographic translation’
based on Damerau-Levenshtein distance, even enriched with dictionary mappings.
At the same time, it does not require large parallel corpora or a ready-made
statistical machine translation model.

The rest of the paper is structured as follows. In Section 2. we describe the
foundations of our approach and the related work. Section 3. introduces the
employed corpora and the story behind them. Section 4. is dedicated to
learning the transformation matrix, and Section 5. describes our experimental
setting and evaluation results. We discuss the findings in Section 6. and
conclude in Section 7., also suggesting directions for future work.

##  2\. Related Work

Clustering multi-lingual documents has received much attention in natural
language processing. Among approaches not using some form of machine
translation, one can mention [Mathieu et al., 2004], who essentially employ a
bilingual dictionary to bring some words in the documents to a language-
independent form and then to perform clustering. In the section 5. we show
that our approach based on neural embeddings significantly outperforms their
reported results.

[Wolf et al., 2014] proposed training joint multi-lingual neural embedding
models. Theoretically, this can be used to achieve our aim of language-
independent semantic representations for documents. Unfortunately, it demands
a large word-aligned parallel corpus. This is not the case with the more
recent Trans-gram approach introduced in [Coulmance et al., 2016], also able
to learn multi-lingual models. However, it still needs sentence-aligned
corpora to train on (in the size of millions of paired sentences). Large
parallel corpora (whether word- or sentence-aligned) are often a scarce
resource, especially in the case of under-represented languages.

The approach described in this paper takes as an input only comparable
monolingual corpora and bilingual dictionaries in the size of several thousand
word pairs. Such resources are much easier to find and evaluate. We employ the
idea of learning a linear transformation matrix to map or project word
embeddings from the semantic space of one language to that of another. This
idea was first proposed in [Mikolov et al., 2013a], who applied it to lexical
translation between English, Spanish, Czech and Vietnamese. We extend it from
continuous representations of single words or collocations to ‘semantic
fingerprints’ of documents as a whole.

##  3\. Academic texts as Comparable Corpora

The Russian and Ukrainian languages are mainly spoken in Russian Federation
and the Ukraine and belong to the East-Slavic group of the Indo-European
language family. They share many common morphosyntactic features: both are SVO
languages with free word order and rich morphology, both use the Cyrillic
alphabet and share many common cognates.

Both Russia and the Ukraine have common academic tradition that makes it
easier to collect corpora, which are comparable in terms of both genre and
strictly defined academic fields. We work with such a corpus of Russian and
Ukrainian academic texts, initially collected for the purposes of cross-
lingual plagiarism detection. This data is available online through a number
of library services, but unfortunately cannot be republished due to copyright
limitations.

The Ukrainian subcorpus contains about 60 thousand extended summaries (Russian
and Ukrainian ‘автореферат’, ‘avtoreferat’) of theses submitted between 1998
and 2011. The Russian subcorpus is smaller in the number of documents (about
16 thousand, approximately the same time period), but the documents are full
texts of theses, thus the total volume of the Russian subcorpus is notably
larger: 830 million tokens versus 250 million tokens in the Ukrainian one.
Generally, the texts belong to one genre that can be defined as post-Soviet
expository academic prose, submitted for academic degree award process.

The documents were converted to plain text files from MS Word format in the
case of the Ukrainian subcorpus and mainly from OCRed PDF files in the case of
the Russian subcorpus. Because of this, the Russian documents often suffer
from OCR artifacts, such as words split with line breaks, incorrectly
recognized characters and so on. However, it does not influence the resulting
model much, as we show below.

Both Ukrainian and Russian documents come with meta data allowing to separate
them into academic fields, with economics, medicine and law being most
frequent topics for the Ukrainian data and economics, history and pedagogy
dominating the Russian data.

For evaluation, 3 topics were used, distant enough from each other and
abundantly presented in both subcorpora: economics, law and history. We
randomly selected 100 texts in each language for each topic. As an average
length of Russian texts is significantly higher (them being full theses), we
cropped them, leaving only the first 5 thousand words, to mimic the size of
the Ukrainian summaries. These 600 documents in 3 classes are used as a test
set (see Section 5. for the description of the conducted experiments).

The corpora (including test set) were PoS-tagged111We used Mystem [Segalovich,
2003] for Russian and Ugtag [Kotsyba et al., 2009] for Ukrainian.. Each word
was replaced with its lemma followed by a PoS-tag (‘диссертация_S’,
‘диссертацiя_N’). Functional parts of speech (conjunctions, pronouns,
prepositions, etc.) and numerals were removed from the texts.

##  4\. Learning to Translate: Ukrainian-to-Russian transformations

As already stated, our main proposal is using neural embedding models to
‘project’ documents in one language into the semantic space of another
language. For this, we first trained a Continuous Bag-of-Words (CBOW) and a
Continuous SkipGram model [Mikolov et al., 2013b] for each of our monolingual
subcorpora. The models were trained with identical hyperparameters: vector
size of 300 components222[Mikolov et al., 2013a] suggest to use larger vector
size for the source language model; however, we leave it for the future work.,
symmetric window of 2 words, negative sampling with 10 samples, 5 iterations
over the corpus, no down-sampling. The only language-dependent difference was
that for the Ukrainian model we ignored words with the corpus frequency less
than 10 and for the Russian model this threshold was set to 15 (as the Russian
corpus is 3 times larger). All in all, the final Ukrainian model recognizes
429 215 words and the Russian one 271 720 words. Training was performed using
CBOW and SkipGram implementation in Gensim library [Řehůřek and Sojka, 2010].

After the models were trained, we followed the path outlined in [Mikolov et
al., 2013a] to learn a linear transformation matrix from Ukrainian to Russian.
First, we extracted all noun pairs from Russian-Ukrainian bilingual dictionary
[Ganich and Oleynik, 1990], with the constraint that their frequency in our
corpora was above the already mentioned thresholds 15 and 10 for Russian and
Ukrainian words correspondingly. That made it a list of about 5 thousand pairs
of nouns being translations of each other.

For all these words, their vectors were found in the models corresponding to
the words’ languages. It provided us with a matrix of 5 thousand of
300-dimensional Ukrainian vectors and the matrix of corresponding 5 thousand
of 300-dimensional Russian vectors. This data served as a training set to
learn an optimal transformation matrix. The latter is actually a 300x301
matrix of coefficients, such that when the initial Ukrainian matrix is
multiplied by this transformation matrix, the result is maximally close to the
corresponding Russian matrix. This transformation matrix has 301 (not 300)
columns, because we add one component equal to 1 to each vector, as a bias
term.

Producing the transformation matrix is a linear regression problem: the input
is 301 components of Ukrainian vectors (including the bias term) and the
output is 300 components of Russian vectors. As we need 300 values as an
output, there are actually 300 linear regression problems and that’s why the
resulting matrix size is 300x301 (301 weights for each of 300 components).

There are two main ways to solve a linear regression problem: one can either
learn the optimal weights in an iterative way using some variant of gradient
descent, or one can solve it numerically without iteration, using normal
equation. For English and Spanish, [Mikolov et al., 2013a] used stochastic
gradient descent. However, normal equation is actually less error-prone and is
guaranteed to find the global optimum. Its only disadvantage is that it
becomes very computationally expensive when the number of features is large
(thousands and more). However, in our case the number of features is only 301,
so computational complexity is not an issue.

Thus, we use normal equation to find the optimal transformation matrix. The
algebraic solution to each of 300 normal equations (one for each vector
component $i$) is shown in the Equation 1:

$\boldsymbol{\beta}_{i}=(\textbf{X}^{\intercal}*\textbf{X})^{-1}*\textbf{X}^{\intercal}*y_{i}$
(1)

where X is the matrix of 5 thousand Ukrainian word vectors (input), $y_{i}$ is
the vector of the $i$th components of 5 thousand corresponding Russian words
(correct predictions), and $\boldsymbol{\beta}_{i}$ is our aim: the vector of
301 optimal coefficients which transform the Ukrainian vectors into the $i$th
component of the Russian vectors.

After solving such normal equations for all the 300 components $i$, we have
the 300x301 linear transformation matrix which fits the data best.

This matrix basically maps the Ukrainian vectors into the Russian ones. It is
based on the assumption that the relations between semantic concepts in
different languages are in fact very similar (students are close to teachers,
while pirates are close to corsairs, and so on). In continuous distributional
models which strive to represent these semantic spaces, mutual ‘geometrical’
relations between vectors representing particular words are also similar
across models (if they are trained on comparable corpora), but the exact
vectors for words denoting one and the same notion are different. This is
because the models themselves are stochastic and the particular values of
vectors (unlike their positions in relation to each other) depend a lot on
technical factors, including the random seed used to initialize vectors prior
to training. In order to migrate from a model A to another model B, one has to
‘rotate and scale’ A vectors in a uniform linear way. To learn the optimal
transformation matrix means to find out the exact directions of rotating and
scaling, which minimize prediction errors.

Linguistically speaking, once we learned the transformation matrix, we can
predict what a Russian vector would most probably be, given a Ukrainian one.
This essentially means we are able to ‘translate’ Ukrainian words into
Russian, by calculating the word in the Russian model with the vector closest
to the predicted one.

We had to choose between CBOW or Continuous SkipGram models to use when
learning the transformation matrix. Also, there was a question of whether to
employ regularized or standard normal equations. Regularization is an attempt
to avoid over-fitting by trying to somehow decrease the values of learned
weights. The regularized normal equation is shown in 2:

$\boldsymbol{\beta}_{i}=(\textbf{X}^{\intercal}*\textbf{X}+\lambda*L)^{-1}*\textbf{X}^{\intercal}*y_{i}$
(2)

Comparing to 1, it adds the term $\lambda*L$, where $L$ is the identity matrix
of the size equal to the number of features, with 0 at the top left cell, and
$\lambda$ is a real number used to tune the influence of regularization term
(if $\lambda=0$, there is no regularization).

To test all the possible combinations of parameters, we divided the bilingual
dictionary into 4500 noun pairs used as a training set and 500 noun pairs used
as a test set. We then learned transformation matrices on the training set
using both training algorithms (CBOW and SkipGram) and several values of
regularization $\lambda$ from 0 to 5, with a step of 0.5. The resulting
matrices were applied to the Ukrainian vectors from the test set and the
corresponding Russian ‘translations’ were calculated. The ratio of correct
‘translations’ (matches) was used as an evaluation measure. It came out that
regularization only worsened the results for both algorithms, so in the Table
1 we report the results without regularization.

For reference, we also report the accuracy of ‘quazi-translation’ via Damerau-
Levenshtein edit distance [Damerau, 1964], as a sort of a baseline. As already
stated, the two languages share many cognates, and a lot of Ukrainian words
can be orthographically transformed into their Russian translations (and vice
versa) by one or two character replacements. Thus, we extracted 50,000 most
frequent nouns from our Russian corpora; then for each Ukrainian noun in the
bilingual dictionary we found the closest Russian noun (or 5 closest nouns for
@5 metric) by edit distance and calculated how often it turned out to be the
correct translation. As the Table 1 shows, notwithstanding the orthographic
similarity of the two languages, CBOW consistently outperforms this approach
even on the test set. On the training set, its superiority is even more
obvious.

Table 1: Translation accuracy | CBOW | SkipGram |  Edit distance
---|---|---|---
| Training | Test | Training | Test |
@1 | 0.648 | 0.57 | 0.545 | 0.374 | 0.549
@5 | 0.764 | 0.658 | 0.644 | 0.486 | 0.619

As for comparison between learning algorithms for matrix translation, CBOW-
based transformation matrix is again the winner, with 57% matches on the test
set and 65% matches on the training set, beating SkipGram in both. Note that
in the context of this task, SkipGram models seem to have problems with
actually learning the optimal transformation matrix for unseen data: on the
test set they perform even worse than the edit distance approach.

CBOW is also consistently better if we consider cases when the correct word is
among 5 nearest neighbors of the predicted vector to be matches as well
(accuracy @5). This is an important metrics, because quite often the
‘translation’ is not exactly the corresponding word from the dictionary, but
still a very semantically similar one, while the dictionary translation is the
second or the third by its cosine similarity to the predicted vector. It means
that in fact the ‘semantic translation’ is successful, as the concept is
correct. For example, our algorithm translates the Ukrainian noun ‘гетьман’
hetman into Russian ‘царь’ tzar, while the correct translation ‘гетман’ is the
second nearest neighbor.

Notwithstanding the fact that the transformation matrix was trained
exclusively on nouns, it correctly ‘translates’ adjectives and verbs as well
(we did not experiment with other parts of speech though). However, it tends
to ‘substantivize’ them: for example, the Ukrainian verb ‘розробити’ to
develop is transformed into a Russian vector, which is closer to the noun
‘разработка’ development than to the corresponding verb.

Thus, at least main parts of speech seem to share a common Ukrainian-to-
Russian projection matrix, supporting the view that semantic spaces for
different languages are in comparatively simple linear relations to each
other. In the following clustering experiments we employed CBOW-based
transformation matrix and consequently CBOW models for Russian and Ukrainian.

We also applied the same transformation matrix to the document-level ‘semantic
fingerprints’. These fingerprints are simple average vectors of all words that
the document contains. Thus, if our models have vector size 300, the resulting
fingerprints are 300-dimensional vectors as well. These vectors can be
transformed with the same matrix. As we show in the Section 5., the cross-
lingual linear relations hold not only for words, but for these semantic
fingerprints as well.

##  5\. Experiment Design and Evaluation

We evaluate the cross-lingual representations described above on the task of
clustering a set of documents. Recall that our test set consists of randomly
selected 600 documents, equally divided between the topics of economics, law
and history, and Russian and Ukrainian languages. Thus, we have 100 Ukrainian
law texts, 100 Russian law texts, etc. The average length of the texts is 4000
word tokens.

We aim to find such a representation for documents which would reveal their
topical structure independent of the language. It can be tested by clustering
the whole collection in an unsupervised way into 3 clusters (in our setting,
the number of topics is a given parameter), and finding out to what extent
these clusters correspond to the topical classes: law, economics and history.
This correspondence can be calculated by mapping the resulting clusters into
topics judging by where the majority of documents belonging to this or that
topic were assigned. For example, if more than 100 history documents were
assigned to the cluster 0, we map this cluster to the history topic, etc.
Then, the ratio of incorrect assignments is calculated, as percentage from the
total number of documents. This is our primary evaluation measure. All the
clustering experiments below are performed using a well-established K-means
algorithm [Hartigan, 1975] with Euclidean distances. We intentionally employ
the most basic clustering algorithm to make the difference of the underlying
representations more visible.

The lemmatized documents were represented as bags-of-words. To reduce the
dimensionality of such representations and to filter out unimportant noise
words, some sort of feature selection is often used. We employed the most
basic variant of it: frequency threshold, where the words are ranked by their
frequencies in the whole document collection, and only top $x$ are then used
in constructing vector representations. We empirically chose $x=500$, as
several values from 100 to 1000 which we tried (with the step of 100) resulted
in worse performance, independent of the approaches tested. Note that the sets
of 500 most frequent words were selected for each topic separately, to avoid
the situation when some topics are under-represented, because words related to
them are not frequent. Then the union of these sets was used as the final
vocabulary (resulting in vectors of about 800…900 dimensions, depending on the
particular method used). Initially, binary vectors were constructed (a word is
either present in the document or not), but we also tested count vectors,
which store words’ per-document frequencies; see below.

In order to make sure that the topical division is indeed manifested in the
documents, we first clustered Ukrainian and Russian corpora separately, using
the binary bag-of-words representations described above. This gave only 4.7%
incorrect assignments for the Ukrainian texts and 34.7% incorrect assignments
for the Russian part of the test set. Thus, for Ukrainian the division is
almost perfect, while for Russian it is manifested less clearly (it seems that
economics and law are consistently mixed up), but still the overwhelming
majority of documents is clustered according to the topics. It means that the
test set does contain information to correctly cluster the documents on a
mono-lingual level, and it makes sense to try to achieve comparable (or at
least not much worse) results in the cross-lingual experiments. Note that one
can’t simply cluster the documents in Russian and in Ukrainian separately to
achieve our aim: even if the clusterings are ideal, there will be no way to
map the Russian clusters to the Ukrainian ones, or vice versa.

So, the next step was to cluster all documents together, independent of their
languages, using the techniques described in the Section 4.. The results are
shown in the Table 2.

We used two simple baseline approaches. The first one is dubbed ‘naive’: we
cluster all the texts’ bag-of-words representations as is, with no special
preprocessing (only the PoS tags are unified across languages). Transformation
from texts to bags-of-words resulted in 885-dimensional document vectors. This
baseline approach exploits the intuition that in closely-related languages
such as Russian and the Ukrainian there are many words which share both
spelling and meaning. This is true, but this fact does not help K-means to
correctly cluster the collection into topical classes: 50.17% of the documents
are assigned an incorrect cluster, much more than in any of our mono-lingual
experiments. Employing count vectors instead of binary ones lowers error rate
only down to 50%. Using tf-idf weighting [Jones, 1972] did not significantly
change the results neither for this nor for other baselines.

Looking into particular cluster assignments reveals that K-means clusters all
the Ukrainian documents into one group, and then partitions Russian texts into
two clusters roughly corresponding to history and everything else. This is
quite expected: the Ukrainian alphabet contains several frequent characters
missing in Russian (‘ґ, є, і, ї’), while the Russian-specific characters (‘ё,
ъ, ы, э’) are much rarer. Consequently, the Ukrainian documents contain a lot
of Ukrainian words specific only to them, while Russian words (or their
identically spelled Ukrainian counterparts) are used throughout the whole
collection. Anyhow, ‘naive’ approach can’t adequately represent the topical
structure of the test set.

Table 2: Clustering correspondence to document topics Method |  Incorrect assignments, %
---|---
Mono-lingual
Ukrainian | 4.7
Russian | 34.7
Cross-lingual
Naive Binary | 50.17
Naive Count | 50.00
Edit distance translation Binary | 50.50
Edit distance translation Count | 50.50
Dictionary/Edit distance Binary | 50.33
Dictionary/Edit distance Count | 49.83
Matrix translation Binary | 36.33
Matrix translation Count | 36.17
Semantic fingerprints on word types | 35.33
Semantic fingerprints on word tokens | 5.50

The second baseline employs quazi-translation of Ukrainian words into Russian
using the already described approach with Damerau-Levenshtein edit distance
[Damerau, 1964]. We replaced all words in the Ukrainian texts with the Russian
words closest to them by edit distance. Only nouns, adjectives, verbs and
abbreviations were replaced; replacements were selected only among the same
part of speech as the original word, and in case of ties, target word with the
highest frequency in Russian corpus was selected. Then the same bag-of-words
representation (now 834-dimensional) was fed to the clustering algorithm.
Though for many words ‘edit distance translation’ works quite well, it did not
help in clustering multilingual test set. Whether with binary or count
vectors, K-means still grouped all the Ukrainian documents into one cluster,
resulting in 50.5% of incorrect assignments. The possible reason is that there
are still many incorrect ‘Levenshtein translations’ resulting in target
entities which are correct Russian words, but never appear in Russian
documents from our test set. This gives K-means the ground to separate the
Ukrainian texts from all the other documents.

Then we experimented with translating Ukrainian words into Russian using the
learned transformation matrix (matrix translation). For each Ukrainian word,
we multiplied its vector in the Ukrainian model by the matrix and found the
Russian word nearest to the resulting vector. Then the Ukrainian words were
replaced with these ‘translations’ and the same bag-of-words document
representations were constructed (resulting in 845-dimensional vectors). As a
result, the K-means clustering moved substantially towards the intended
topical grouping: only 36.33% of the documents were assigned incorrect
clusters, and using count vectors made it 36.17%. In fact, the clustering
algorithm correctly separated all history documents into one cluster
independent of the language, while still mixing things up with law and
economics (as we know, they are a bit more difficult to separate even in a
monolingual setting). Thus, this document representation seems to be clearly
superior to the baseline naive or edit distance approaches. It results in the
documents grouping which is almost as efficient as mono-lingual clustering of
Russian texts, but is still not on a par with Ukrainian mono-lingual
clustering.

Note that these improvements cannot be explained by the sheer fact of
employing a bilingual dictionary. We tried to use the same dictionary
directly: that is, for the Ukrainian texts in the test set, replace all the
words with their dictionary Russian translation. The remaining out-of-
vocabulary words were ‘translated’ with the Dameral-Levenshtein distance
approach. The results are reported in the Table 2 as dictionary/edit distance
method. They are a bit better than the ones of the raw edit distance, but
still far from the performance of the matrix translation method. It means that
the algorithm itself is the cause of improvements.

Finally, the best results were received by employing the ‘semantic
fingerprint’ approach. Recall that this fingerprint is an average vector of
all words in the document. Consequently, each document is represented with a
300-dimensional vector, supposedly reflecting its ‘meaning’.

The average vector can be calculated either on vectors of word types or of
word tokens (thus taking into account individual frequencies of words in the
document). These two variants roughly correspond to binary and count variants
of the previous methods, but we intentionally dub them in another way to
emphasize that these representations are radically different from bags-of-
words. In this case we abstract away from particular words, and instead use
some generalized ‘semantic components’, hopefully similar across languages.

We computed such fingerprints for all the documents in the test set, and for
the Ukrainian documents we then multiplied the fingerprints by the
transformation matrix, thus ‘projecting’ them into Russian semantic space. The
resulting 300-dimensional representations are already numerical and can be
directly fed into a clustering algorithm, without any bag-of-words
preprocessing.

As a result, even rough semantic fingerprints calculated on word types (on
sets of words in the documents) show clustering accuracy 1% better than the
matrix translation approach. But as soon as semantic fingerprints are computed
using word tokens (lists of words), the ratio of incorrect assignments drops
drastically down to 5.5%. This result is very close to the quality of the
mono-lingual Ukrainian clustering and is much better than that of the mono-
lingual Russian clustering. It means that semantic fingerprints approach
performs almost as good in the cross-lingual setting as traditional approaches
in the mono-lingual one. Additionally, the fact that it outperformed the
Russian mono-lingual clustering might mean that using dense vector
representations for documents allowed to overcome the problems with separating
economics and law texts in Russian, which seemed intractable for the bag-of-
words approach.

To be more precise, into their respective clusters were grouped 196 of 200
economics documents (this corresponds to approximately 0.95 precision and 0.98
recall), 195 of 200 history documents (0.92 precision, 0.98 recall) and 176 of
200 law documents (0.97 precision, 0.88 recall), all independent of their
languages. Total average F1 measure is about 0.95, which significantly
outperforms the multilingual clustering performance reported in [Mathieu et
al., 2004].

Figure 1: Naive baseline clustering

Figures 1, 2 and 3 illustrate clustering mechanics for the methods described
above. We employed t-SNE dimensionality reduction technique [Van der Maaten
and Hinton, 2008] to project high-dimensional representations333300 dimensions
for semantic fingerprints, 885 and 845 for naive baseline and matrix
translation correspondingly. of the test set documents into 2-dimensional
plots. Colors reflect document language (blue for Ukrainian and white for
Russian), while marker types stand for document topic (circles for law,
squares for history and pentagons for economics). Note that these projections
inevitably lose a lot of information as compared to initial high-dimensional
data, and should be considered as only approximate visualizations.

It is clearly visible that with naive baseline representations in the Figure 1
there are almost no links between different-language documents belonging to
one topic. The dataset is clearly separated into Russian and Ukrainian
clusters, and topics can be seen inside languages, but there is hardly a way
to group documents into language-independent topical clusters. This is the
reason for K-means failing to achieve our aim with the baseline approach. On
the other hand, with matrix translation representations (Figure 2), language-
independent topics already emerge, but still with much noise. Language
boundaries are eroded, especially with economics documents.

Figure 2: Matrix translation clustering

Finally, with semantic fingerprints representations in the Figure 3 the
structure of the test set is manifested in full. There are six well-defined
clusters corresponding to topics and languages and a clear spatial structure,
which allows K-means to easily group documents into 3 larger topical clusters
without losing the ability to tell a Russian document from a Ukrainian one.
Note how the Ukrainian topical clusters seem to share a common linear relation
to the Russian ones, reminding about linear relations between different
languages’ vector spaces.

Figure 3: Semantic fingerprints clustering

Thus, we were able to correctly cluster multilingual documents according to
their topics without any proper ‘translation’ and without even considering
word spelling. This means that, first, semantic fingerprints are precise
enough to reveal topical differences between documents, and second, that this
holds even after linear transformation of such fingerprints into another
language semantic space.

##  6\. Discussion

We tested ‘transformed’ semantic representations of the documents on the
clustering task, but theoretically they can be used for any problems which
demand semantic-aware cross-lingual representations, including classification
and visualization. Also, the number of involved languages is not limited in
any way. The proposed method is relatively simple and straightforward to
implement: one needs only comparable monolingual corpora to train CBOW models
on them (using any of the available off-the-shelf toolkits) and a small
bilingual dictionary for each language pair to train linear transformation
matrices. After that, all the words and documents in the corpora can be
transformed into a unified language-independent semantic representation.

It is interesting that in our experiments semantic fingerprints’ performance
was better than direct ‘translation’ of words using the same transformation
matrix. It reveals an important advantage of such generalized representations:
they do not depend on particular words. In the case of the bag-of-words
approach, a small mistake in matrix translation can lead to replacement of a
Ukrainian word with a Russian counterpart, which is semantically similar, but
not exactly the one used to denote this concept in the Russian part of this
text collection. As a result, this word becomes useless in representing
documents cross-linguistically. On the other hand, with the semantic
fingerprints approach, an ‘approximate transformation’ is enough, as it will
still be close to the corresponding words from Russian texts in the vector
space.

This also explains the accuracy boost this approach gets from considering word
tokens instead of word types. Of course, one reason is that vectors for
frequent (arguably more topical) words become more important in determining
the final average value, but this is also the case for bag-of-words
approaches. The difference is that with the latter (including ‘matrix
translation’ method), frequencies of words from the same semantic field are
interpreted as independent features. As a result, for example, $n$ words of
frequency $z$ related to history topic will not be more important than other
random $n$ words of the same frequency. At the same time, with ‘semantic
fingerprints’, topically connected words collectively increase or decrease
expression of the corresponding semantic component or components: they are
more important in determining the resulting fingerprint, than random noise
words, even if they are frequent. This leads to better discrimination between
documents of different topics.

It is also important that ‘semantic fingerprinting’ is significantly faster
than ‘matrix translation’, as we eliminate the necessity to look for the most
similar neighbors of the predicted vector. This operation can be
computationally expensive, especially on models with large vocabulary.

Note that one can apply the described method not only to proper cross-lingual
translations, but also to problems like ‘projecting’ texts in one style or
genre into another. In fact the method is applicable to any situation, where
there are two comparable sets of texts consisting of items, for which there
theoretically exist pairwise links; one knows only a small part of these
links, but would like to compare texts independent of the corpus they belong
to. In these cases, semantic fingerprints method can be of use.

##  7\. Conclusion and Future Work

Thus, we described an approach to build language-independent semantic
representations of documents in multi-lingual comparable corpora. It was
tested on a rather small task of clustering Ukrainian and Russian academic
texts into 3 topics. However, the results seem very promising to us and we
plan to continue working on the proposed method. The models trained on our
corpora, the linear transformation matrix, the evaluation dataset we used and
Python code to work with this data are available
online444https://cloud.mail.ru/public/Eune/tN7ssqtWj.

The initial motivation behind this work was to develop a system for automatic
plagiarism detection for two closely related languages. A crucial component of
this system is a preprocessing part, which is able to cluster texts according
to their topics. We believe that this component eventually will make it
possible to compare ‘semantic fingerprints’ of the documents in order to
determine possible plagiarized texts and to perform their further analysis.

One obvious disadvantage of the proposed method is the necessity to know in
advance the desired number of clusters (topics in the text collection). We
plan to experiment with approaches to determining the optimal number of
clusters automatically. It poses serious problems in multi-lingual settings,
as the algorithms will be biased to language-based clustering, not taking into
account topical division. Thus, ways should be invented to cope with this bias
especially when the number of topics is much higher than 3 (used in this
research).

Another direction of future work is to compare our approach and bag-of-words
representations after proper machine translation. The results are not obvious:
on the one hand, MT directly casts texts into another language and that should
be a difficult baseline to beat. On the other hand, as explained in Section
6., dense document representations like semantic fingerprints can possibly be
more flexible in grasping document contents than words-based representations.

Finally, we plan to test the proposed method with other language pairs,
especially typologically distant languages. Experiments in [Mikolov et al.,
2013a] suggest that as long as the languages possess the meaningful notion of
lexical co-occurrence, genetic or typologic distances between them should not
matter. However, this is still to be tested and proved.

##  8\. Acknowledgments

We are grateful to the voluntary Dissernet community for providing us with the
data, support, and inspiration to start this work.

##  9\. Bibliographical References

## References

  * Coulmance et al., 2016 Coulmance, J., Marty, J.-M., Wenzek, G., and Benhalloum, A.  (2016).  Trans-gram, fast cross-lingual word-embeddings.  arXiv preprint arXiv:1601.02502.
  * Damerau, 1964 Damerau, F. J.  (1964).  A technique for computer detection and correction of spelling errors.  Commun. ACM, 7(3):171–176, March.
  * Ganich and Oleynik, 1990 Ganich, D. I. and Oleynik, I. S.  (1990).  Russian-Ukrainian and Ukrainian Russian Glossary.  Veselka.
  * Hartigan, 1975 Hartigan, J. A.  (1975).  Clustering algorithms.  Wiley.
  * Jones, 1972 Jones, K. S.  (1972).  A statistical interpretation of term specificity and its application in retrieval.  Journal of Documentation, 28(1):11–21.
  * Kotsyba et al., 2009 Kotsyba, N., Mykulyak, A., and Shevchenko, I. V.  (2009).  Ugtag: morphological analyzer and tagger for the ukrainian language.  In Proceedings of the international conference Practical Applications in Language and Computers (PALC 2009).
  * Mathieu et al., 2004 Mathieu, B., Besançon, R., and Fluhr, C.  (2004).  Multilingual document clusters discovery.  In Coupling approaches, coupling media and coupling languages for information retrieval, pages 116–125.
  * Mikolov et al., 2013a Mikolov, T., Le, Q., and Sutskever, I.  (2013a).  Exploiting similarities among languages for machine translation.  arXiv preprint arXiv:1309.4168.
  * Mikolov et al., 2013b Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J.  (2013b).  Distributed representations of words and phrases and their compositionality.  Advances in Neural Information Processing Systems 26, pages 3111–3119.
  * Řehůřek and Sojka, 2010 Řehůřek, R. and Sojka, P.  (2010).  Software Framework for Topic Modelling with Large Corpora.  In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45–50, Valletta, Malta, May.
  * Segalovich, 2003 Segalovich, I.  (2003).  A fast morphological algorithm with unknown word guessing induced by a dictionary for a web search engine.  In MLMTA, pages 273–280. Citeseer.
  * Van der Maaten and Hinton, 2008 Van der Maaten, L. and Hinton, G.  (2008).  Visualizing data using t-SNE.  Journal of Machine Learning Research, 9(2579-2605):85.
  * Wolf et al., 2014 Wolf, L., Hanani, Y., Bar, K., and Dershowitz, N.  (2014).  Joint word2vec networks for bilingual semantic representations.  International Journal of Computational Linguistics and Applications, 5(1):27–44.


# A note on the minimal tensor product and the C*-envelope of operator systems

Ian Koot

(February 2022)

###### Abstract

In this article, we show that the $C^{*}$-envelope of the minimal tensor
product of two operator systems is isomorphic to the minimal tensor product of
their $C^{*}$-envelopes. We do this by identifying the Šilov boundary ideal of
the minimal tensor product of two operator systems. Finally, as an application
of this result, we show that the propagation number of the minimal tensor
product of operator systems is the maximum of the propagation numbers of the
factors.

##  1 Introduction

It is a well-known result that every unital $C^{*}$-algebra is isomorphic to a
unital, self-adjoint and norm-closed subalgebra of the space of bounded
operators on some Hilbert space. Generalizing this, there is a structure
called an operator system, which are isomorphic to unital, closed and self-
adjoint subspaces of the bounded operators on some Hilbert space. These
operator systems can be abstractly characterized as vector spaces with extra
structure in the form of an involution, and a notion of when a matrix with
entries in the operator system is positive.

Any unital, closed and self-adjoint subspace of a $C^{*}$-algebra is an
operator system. Conversely, because the bounded operators on a Hilbert space
form a $C^{*}$-algebra, every operator system is isomorphic to a unital,
closed and self-adjoint subspace of a $C^{*}$-algebra. For any given operator
system, we can therefore ask the question: in which $C^{*}$-algebras can we
identify a subspace that is isomorphic to the given operator system? It turns
out that all such $C^{*}$-algebras can be projected onto a specific
$C^{*}$-algebra with the same property.

Specifically, for an operator system $E$ we can consider all maps
$i_{A}:E\hookrightarrow A$, where $A$ is a $C^{*}$-algebra, such that $i_{A}$
is an isomorphism onto its image, and without loss of generality we can assume
$A$ to be the smallest $C^{*}$-subalgebra containing $E$. We call such a map a
$C^{*}$-extension. There is a specific $C^{*}$-extension, called the
$C^{*}$-envelope of $E$, which is in some sense the ‘smallest’
$C^{*}$-extension. What’s more, one can prove that any $C^{*}$-algebra $A$
that arises in a $C^{*}$-extension contains an ideal $I$ called the Šilov
boundary ideal, such that $A/I$ is isomorphic to the $C^{*}$-envelope.

There are multiple characterizations of the Šilov boundary ideal. In this
article these different characterizations are used to clarify the relation
between the $C^{*}$-envelope and the minimal tensor product of operator
systems and $C^{*}$-algebras. Specifically, we do this by proving the
following theorem, which is the main result of this article:

######  Theorem 1.

Let $E\subseteq B(H)$ and $F\subseteq B(K)$ be operator systems, and let
$I\subseteq C^{*}(E)$ and $J\subseteq C^{*}(F)$ be their respective Šilov
boundary ideals. Then $\ker q_{I}\otimes_{\min}q_{J}$ is the Šilov boundary
ideal for $E\otimes_{\min}F\subseteq B(H\overline{\otimes}K)$. Consequently,
this means that

$C^{*}_{env}(E\otimes_{\min}F)\cong
C^{*}_{env}(E)\otimes_{\min}C^{*}_{env}(F)$

To prove this, we first formally introduces the relevant concepts such as the
$C^{*}$-envelope and the Šilov ideal in Section 2. The proof of Theorem 1 is
located in Section 3, where we identify the Šilov ideal of the tensor product
$E\otimes_{\min}F$ of two operator systems $E$ and $F$, which enables us to
prove the isomorphism.

Finally, in Section 4 we apply this identity to a property called the
propagation number of an operator system, introduced by Connes and Van
Suijlekom in [1]. This is a number associated to an operator system which,
roughly speaking, describes how close it is to being a $C^{*}$-algebra; it
turns out to behave particularly nice with respect to the tensor product, as
we will see.

##  2 Preliminaries

We use the following definition of operator systems:

######  Definition 1.

An operator system $E$ is a unital linear subspace $E\subseteq B(H)$ that is
closed in the norm-topology and for which $e^{*}\in E$ for all $e\in E$.

For the definitions and discussions of related concepts such as operator
spaces, completely bounded maps, completely isometric maps and completely
positive maps, we refer for example to [2].

Some remarks on notation: for a $C^{*}$-algebra $A$ with (closed, two-sided)
ideal $I\subseteq A$ we will denote the canonical quotient map
$A\longrightarrow A/I$ by $q_{I}$. For a linear map $\phi:E\longrightarrow F$,
we write the induced map on matrix spaces as

$\phi^{(n)}:M_{n}(E)\longrightarrow
M_{n}(F),\,\,\,(e_{ij})\mapsto(\phi(e_{ij})).$

Also, for subsets of vector spaces $S\subseteq V$ and $T\subseteq W$ we let

$S\otimes T:=\text{span}\\{s\otimes t\mid s\in S,t\in T\\}\subseteq V\otimes
W.$

For linear maps $\phi:V_{1}\longrightarrow V_{2}$ and
$\psi:W_{1}\longrightarrow W_{2}$ we let $\phi\otimes\psi:V_{1}\otimes
V_{2}\longrightarrow W_{1}\otimes W_{2}$ be the map defined by
$\phi\otimes\psi(v\otimes w)=\phi(v)\otimes\psi(w)$ on elementary tensors. If
$V_{1},V_{2},W_{1}$ and $W_{2}$ are more specifically operator spaces,
operator systems or $C^{*}$-algebras, and $\phi$ and $\psi$ bounded, we will
denote by $\phi\otimes_{\min}\psi$ the map obtained from $\phi\otimes\psi$
through continuous extension to the minimal tensor product (see Section 2.2).

###  2.1 The $C^{*}$-envelope and Šilov ideal

We start with the following definition due to Hamana [3]:

######  Definition 2.

A $C^{*}$-extension of an operator system $E$ is a pair $(i,A)$ of a
$C^{*}$-algebra $A$ and a linear map $i:E\longrightarrow A$ such that
$C^{*}(i(E))=A$, such that $i$ is a unital complete order isomorphism onto its
image. The $C^{*}$-envelope of an operator system $E$ is a $C^{*}$-extension
$(i_{E},C^{*}_{env}(E))$ with the following property:

  * •

for every $C^{*}$-extension $(i,A)$ there is a surjective *-homomorphism
$\pi:A\longrightarrow C^{*}_{env}(E)$ such that

${A}$${E}$${C^{*}_{env}(E)}$$\scriptstyle{\pi}$$\scriptstyle{i}$$\scriptstyle{i_{E}}$

commutes.

Existence and uniqueness are proven by Hamana in [3]; note that there another
definition is used, but by Corollary 4.2 in [3] and uniqueness the two
definitions are equivalent. In particular we see that for any concrete
operator system $E\subseteq B(H)$ we can construct the $C^{*}$-extension given
by $C^{*}(E)\subseteq B(H)$, from which it follows that $C^{*}_{env}(E)\cong
C^{*}(E)/\text{ker }\pi$. Hamana showed that the ideal $\ker\pi$ is in fact
the so-called Šilov boundary ideal, as defined by Arveson in [4].

######  Definition 3.

Let $E\subseteq B(H)$ be an operator system. A closed ideal $I\subseteq
C^{*}(E)$ is a boundary ideal if the quotient map $C^{*}(E)\longrightarrow
C^{*}(E)/I$ is completely isometric when restricted to $E$. The boundary ideal
that is maximal with respect to inclusion is called the Šilov boundary ideal
(or simply Šilov ideal).

There is a more concrete description of the Šilov boundary ideal, as first
described by Arveson in [4]. It makes use of the following concept:

######  Definition 4.

Let $E\subseteq B(H)$ be an operator system, and let
$\sigma:C^{*}(E)\longrightarrow B(K)$ be an irreducible representation of
$C^{*}(E)$. Then $\sigma$ is a boundary representation if for any unital
completely positive map $\hat{\sigma}:C^{*}(E)\longrightarrow B(K)$ such that
$\hat{\sigma}|_{E}=\sigma|_{E}$, we must necessarily have
$\hat{\sigma}=\sigma$. We will denote the set of all boundary representations
of $E$ by $\partial E$.

Note that $\partial E$ really depends on the specific embedding $E\subseteq
B(H)$ we choose, and not just on the abstract operator system structure.

######  Proposition 2.

Let $E\subseteq B(H)$ be an operator system and let $I\subseteq C^{*}(E)$ be
the Šilov ideal. Then

$I=\bigcap\\{\ker\sigma\mid\sigma\in\partial E\\}$

Proposition 2 had been proven in specific cases (see for example [5]), but was
proven for the general case by Davidson and Kennedy in [6].

###  2.2 The minimal tensor product

Finally, we introduce the minimal tensor product of operator systems. Given
two Hilbert spaces $H$ and $K$, we can form the algebraic tensor product
vector space $H\otimes K$. If we supply this vector space with the inner
product given by

$\langle x\otimes y,v\otimes w\rangle_{H\otimes K}:=\langle
x,v\rangle_{H}\langle y,w\rangle_{K}$

and complete the space with respect to the induced norm, we have created a new
Hilbert space, which we will denote as $H\overline{\otimes}K$. If we now have
$e\in B(H)$ and $f\in B(K)$, we can define an operator $e\otimes f\in
B(H\overline{\otimes}K)$ through

$e\otimes f:\sum_{i}x_{i}\otimes y_{i}\mapsto\sum_{i}e(x_{i})\otimes f(y_{i})$

and extending by continuity. Given two operator spaces $E\subseteq B(H)$ and
$F\subseteq B(K)$, we can then define the linear subspace

$E\otimes_{\min}F:=\overline{\text{span}}\\{e\otimes f\mid e\in E,f\in
F\\}\subseteq B(H\overline{\otimes}K),$

where the closure is taken in the operator norm. If $E$ and $F$ are operator
systems, it is also easily seen to be self-adjoint and unital by the fact that
$E$ and $F$ are self-adjoint and unital. So then $E\otimes_{\min}F$ is an
operator system. In fact, if $A$ and $B$ are $C^{*}$-algebras (so in
particular operator systems), then $A\otimes_{\min}B$ as defined above is a
$C^{*}$-algebra. The reason this construction is called the minimal tensor
product is that the norm induced by $B(H\overline{\otimes}K)$ on the algebraic
tensor product $A\otimes B$ is the smallest $C^{*}$-norm on that space.

##  3 Identifying the Šilov ideal in the tensor product

In this section, let $E\subseteq B(H)$ and $F\subseteq B(K)$ be operator
systems, and let $I\subseteq C^{*}(E)$ and $J\subseteq C^{*}(F)$ be the Šilov
boundary ideals for $E$ and $F$, respectively.

Note that $C^{*}(E\otimes_{\min}F)=C^{*}(E)\otimes_{\min}C^{*}(F)$: on the one
hand $C^{*}(E)\otimes_{\min}C^{*}(F)$ is a $C^{*}$-algebra containing
$E\otimes_{\min}F$, so the left-to-right inclusion holds; on the other hand,
we can form the set

$\\{x\in B(H)\mid x\otimes 1\in C^{*}(E\otimes_{\min}F)\\}$

which is clearly a $C^{*}$-algebra containing $E$, so it must contain
$C^{*}(E)$, and we therefore have $C^{*}(E)\otimes 1\subseteq
C^{*}(E\otimes_{\min}F)$, from which, after a similar argument for $F$, the
right-to-left inclusion follows. In the next two subsections, we will show
that the ideal

$\ker q_{I}\otimes_{\min}q_{J}\subseteq C^{*}(E\otimes_{\min}F)$

is in fact the Šilov boundary ideal for $E\otimes_{\min}F$. For this we first
show that it is a boundary ideal, and after that we show that the Šilov ideal
is contained in $\ker q_{I}\otimes_{\min}q_{J}$, making use of the
characterization through boundary representations. Since the Šilov ideal is
the maximal boundary ideal, we have that any boundary ideal that contains it
must be the Šilov ideal itself.

###  3.1 Boundary ideal

In order to prove that the qoutient by $\ker q_{I}\otimes_{\min}q_{J}$ is
completely isometric on $E\otimes_{\min}F$, we prove that
$q_{I}\otimes_{\min}q_{J}$ is completely isometric on $E\otimes_{\min}F$. This
is sufficient because of the following lemma:

######  Lemma 3.

Let $\pi:A\longrightarrow B$ be a *-homomorphism between unital
$C^{*}$-algebras, and let $E\subseteq A$ be an operator sytem. Then $\pi|_{E}$
is completely isometric if and only if $q_{\ker\pi}|_{E}$ is completely
isometric.

###### Proof.

Note that we have an *-isomorphism $\phi:A/\ker\pi\longrightarrow\pi(B)$, with
the property that $\phi\circ q_{\ker\pi}=\pi$. Since *-homomorphisms, the
restriction of a completely isometric map to an operator system and the
composition of complete isometric maps are all completely isometric, the lemma
follows. ∎

What remains is to prove that the tensor product of two completely isometric
maps is again completely isometric. For this, we use the argumentation from
[7, Sec. 1.5.1], where the proof of Lemma 5 is given as a sketch. Given
operator spaces $E$ and $F$, we can give the space of completely bounded maps
between them, which we call $CB(E,F)$, an (abstract) operator space structure
through the identification

$M_{n}(CB(E,F))\cong CB(E,M_{n}(F)).$

We then define the dual operator space as $E^{*}:=CB(E,\mathbb{C})$. We can
use this operator space to describe the minimal tensor product as

$E\otimes_{\min}F\cong CB(F^{*},E).$

A discussion of these identifications can for example be found in [8], [9] and
[2].

######  Lemma 4.

Let $\phi:E_{1}\longrightarrow E_{2}$ be a completely isometric map. Then its
pushforward $\phi_{*}:CB(F,E_{1})\longrightarrow CB(F,E_{2})$ given by
$f\mapsto\phi\circ f$ is completely isometric.

###### Proof.

Note that with the induced map

$(\phi_{*})^{(n)}:M_{n}(CB(F,E_{1}))\longrightarrow M_{n}(CB(F,E_{2}))$

we can construct the commuting diagram

${{M_{n}(CB(F,E_{1}))}}$${{M_{n}(CB(F,E_{2}))}}$${{CB(F,M_{n}(E_{1}))}}$${{CB(F,M_{n}(E_{2}))}}$$\scriptstyle{(\phi_{*})^{(n)}}$$\scriptstyle{\sim}$$\scriptstyle{(\phi^{(n)})_{*}}$$\scriptstyle{\sim}$

We note that

$\|(\phi^{(n)})_{*}(f)\|_{cb}=\|\phi^{(n)}\circ
f\|_{cb}=\sup\\{\|(\phi^{(n)}\circ f)^{(m)}\|_{m}\mid 1\leq m\\}$

and

$\|(\phi^{(n)}\circ f)^{(m)}\|_{m}=\sup\\{\|\phi^{(nm)}(f^{(m)}(x))\|_{m}\mid
x\in M_{m}(F),\|x\|_{m}\leq 1\\}.$

Because $\phi$ is completely isometric, $\phi^{(nm)}$ is isometric, and so
$\|(\phi^{(n)}\circ f)^{(m)}\|_{m}=\|f^{(m)}\|_{m}$. So $(\phi^{(n)})_{*}$ is
isometric, and $(\phi_{*})^{(n)}$ is then the composition of isometric maps,
making it isometric. So $\phi_{*}$ is completely isometric. ∎

######  Lemma 5.

If $\phi:E_{1}\longrightarrow E_{2}$ and $\psi:F_{1}\longrightarrow F_{2}$ are
complete isometries between operator spaces, then
$\phi\otimes_{\min}\psi:E_{1}\otimes_{\min}F_{1}\longrightarrow
E_{2}\otimes_{\min}F_{2}$ is a complete isometry.

###### Proof.

First, note that $\phi\otimes_{\min}\psi=\phi\otimes_{\min}I_{F_{2}}\circ
I_{E_{1}}\otimes_{\min}\psi$, where $I_{E_{1}}$ and $I_{F_{2}}$ is the
identity on $E_{1}$ and $F_{2}$, respectively. It therefore suffices to show
that $\phi\otimes_{\min}I_{F_{2}}$ is a complete isometry.

So let $\phi:E_{1}\longrightarrow E_{2}$ be a complete isometry, and $F$
another operator space. Then because $\phi$ is completely isometric, by Lemma
4 it induces the completely isometric pushforward

$\phi_{*}:CB(F^{*},E_{1})\longrightarrow CB(F^{*},E_{2}),f\mapsto\phi\circ f.$

We therefore have the commuting diagram

${{CB(F^{*},E_{1})}}$${{CB(F^{*},E_{2})}}$${E_{1}\otimes_{\min}F}$${E_{2}\otimes_{\min}F}$$\scriptstyle{\phi_{*}}$$\scriptstyle{i_{1}}$$\scriptstyle{\phi\otimes
I_{F}}$$\scriptstyle{i_{2}}$

Since $\phi_{*}\circ i_{1}$ is completely isometric as the composition of
completely isometric maps, we have that $i_{2}\circ\phi\otimes I_{F}$ is also
completely isometric. So $\phi\otimes I_{F}$ is completely isometric. ∎

######  Corollary 6.

Let $E\subseteq B(H)$ and $F\subseteq B(K)$ be operator systems, and $I$ and
$J$ their respective Šilov boundary ideals. Then $\ker
q_{I}\otimes_{\min}q_{J}$ is a boundary ideal for $E\otimes_{\min}F\subseteq
B(H\overline{\otimes}K)$.

###### Proof.

By Lemma 3 we need to prove that the map
$q_{I}\otimes_{\min}q_{J}|_{E\otimes_{\min}F}$ is completely isometric. Since
$I$ and $J$ are by assumption boundary ideals, we have that $q_{I}|_{E}$ and
$q_{J}|_{F}$ are completely isometric. So by Lemma 5 we have that

$q_{I}\otimes_{\min}q_{J}|_{E\otimes_{\min}F}=q_{I}|_{E}\otimes_{\min}q_{J}|_{F}$

is completely isometric. ∎

###  3.2 Boundary representations

We start this section with a crucial property of boundary representations.

######  Lemma 7.

Let $E\subseteq B(H)$ and $F\subseteq B(K)$ be operator systems. If
$\sigma_{1}\in\partial E$ and $\sigma_{2}\in\partial F$, then
$\sigma_{1}\otimes_{\min}\sigma_{2}\in\partial(E\otimes_{\min}F)$.

The proof is a combination of [10, Lemma 3], where it is proved that
$\sigma_{1}\otimes_{\min}\sigma_{2}$ has a unique completely positive
extension from $E\otimes_{\min}F$, and for example [11, Proposition IV.4.13],
where it is proved that $\sigma_{1}\otimes_{\min}\sigma_{2}$ is irreducible.

Next, we have the main lemma for this section. The proof is adapted from [12,
Lemma 2.2], with the primary difference being that the proof below references
the Kirchberg Slice Lemma [13, Lemma I.4.1.9] for the fact that an ideal in
the minimal tensor product contains a simple tensor.

######  Lemma 8.

Let $A$ and $B$ be $C^{*}$-algebras, and let $\mathcal{K}$ and $\mathcal{L}$
be families of ideals in $A$ and $B$, respectively. Now define

$I=\bigcap_{K\in\mathcal{K}}K\,\,\,\textup{and}\,\,\,J=\bigcap_{L\in\mathcal{L}}L$

Then

$\ker q_{I}\otimes_{\min}q_{J}=\bigcap\\{\ker
q_{K}\otimes_{\min}q_{L}\mid(K,L)\in\mathcal{K}\times\mathcal{L}\\}$

###### Proof.

For readability, we will write $\otimes_{\min}$ simply as $\otimes$, and we
will write $M=\bigcap\\{\ker q_{K}\otimes
q_{L}\mid(K,L)\in\mathcal{K}\times\mathcal{L}\\}$.

  * •

First, we prove the inclusion $\ker q_{I}\otimes q_{J}\subseteq M$. For all
$K\in\mathcal{K}$ and $L\in\mathcal{L}$, we have that $I\subseteq K$ and
$J\subseteq L$. So we can apply the isomorphisms
$\rho:(A/I)/(K/I)\xrightarrow{\sim}A/K$ and
$\sigma:(B/J)/(L/J)\xrightarrow{\sim}B/L$ to see that

$q_{K}\otimes q_{L}=(\rho\otimes\sigma)(q_{K/I}\otimes q_{L/J})(q_{I}\otimes
q_{J})$

so $\ker q_{I}\otimes q_{J}\subseteq\ker q_{K}\otimes q_{L}$, proving the
inclusion.

  * •

Second, for the inclusion $\ker q_{I}\otimes q_{J}\supseteq M,$ we define the
seminorm

$N(x)=\sup\\{\|q_{K/I}\otimes
q_{L/J}(x)\|\mid(K,L)\in\mathcal{K}\times\mathcal{L}\\}.$

on $A/I\otimes B/J$. It is easily seen that $N(xy)\leq N(x)N(y)$ and
$N(x^{*}x)=N(x)^{2}$. Note that $N(x)=0$ if and only if $x\in\ker
q_{K/I}\otimes q_{L/J}$ for all $(K,L)\in\mathcal{K}\times\mathcal{L}$. In
particular we have that $\ker N$ is the intersection of ideals, and so it is
an ideal itself.

Note that if $[x]\otimes[y]\in A/I\otimes B/J$ is nonzero, then there are $K$
and $L$ such that $x\not\in K$ and $y\not\in L$. But

$0\neq q_{K}\otimes q_{L}(x\otimes y)=(\rho\otimes\sigma)(q_{K/I}\otimes
q_{L/J})(q_{I}\otimes q_{J})(x\otimes y)$ $=(\rho\otimes\sigma)(q_{K/I}\otimes
q_{L/J})([x]\otimes[y])$

so

$N([x]\otimes[y])\geq\|(q_{K/I}\otimes
q_{L/J})([x]\otimes[y])\|=\|q_{K}\otimes q_{L}(x\otimes y)\|>0.$

Specifically, $\ker N$ does not contain any simple tensors. But by the
Kirchberg Slice Lemma [13, Lemma I.4.1.9] every nontrivial ideal contains a
nontrivial simple tensor. So this means that $\ker N$ is trivial, and so $N$
is actually a $C^{*}$-norm. But those are unique, so $N(x)=\|x\|$ on
$A/I\otimes B/J$.

Finally, as we remarked above, we have

$q_{K}\otimes q_{L}(x)=(\rho\otimes\sigma)(q_{K/I}\otimes
q_{L/J})(q_{I}\otimes q_{J})(x).$

So if $x\in M$, then the left-hand side is zero for all
$(K,L)\in\mathcal{K}\times\mathcal{L}$, and $(q_{I}\otimes q_{J})(x)\in\ker
q_{K/I}\otimes q_{L/J}$ for all $(K,L)\in\mathcal{K}\times\mathcal{L}$ because
$\rho\otimes\sigma$ is injective. We therefore have that

$\|q_{I}\otimes q_{J}(x)\|=N(q_{I}\otimes q_{J}(x))=0,$

so $x\in\ker q_{I}\otimes q_{J}$, so that indeed $M\subseteq\ker q_{I}\otimes
q_{J}$.

In conclusion, we have that $M=\ker q_{I}\otimes q_{J}$, proving the lemma. ∎

######  Corollary 9.

Let $E\subseteq B(H)$ and $F\subseteq B(K)$ be operator systems, and let $I$
and $J$ be their respective Šilov boundary ideals. Then

$\ker
q_{I}\otimes_{\min}q_{J}=\bigcap\\{\ker\sigma_{1}\otimes_{\min}\sigma_{2}\mid(\sigma_{1},\sigma_{2})\in\partial
E\times\partial F\\}$

Specifically, this means that $\ker q_{I}\otimes_{min}q_{J}$ contains the
Šilov boundary ideal of $E\otimes_{\min}F$.

###### Proof.

Using Proposition 2 we see that

$I=\bigcap_{\sigma_{1}\in\partial
E}\ker\sigma_{1},\,\,\,J=\bigcap_{\sigma_{2}\in\partial F}\ker\sigma_{2}.$

According to Lemma 8, we therefore have that

$\ker q_{I}\otimes_{\min}q_{J}=\bigcap\\{\ker
q_{\ker\sigma_{1}}\otimes_{\min}q_{\ker\sigma_{2}}\mid(\sigma_{1},\sigma_{2})\in\partial
E\times\partial F\\}.$

However, because for $i=1,2$ there exists a *-isomorphism $\phi_{i}$ such that
$\phi_{i}\circ\sigma_{i}=q_{\ker\sigma_{i}}$, we have that

$\ker
q_{\ker\sigma_{1}}\otimes_{\min}q_{\ker\sigma_{2}}=\ker\sigma_{1}\otimes_{\min}\sigma_{2}$

for all $\sigma_{1}\in\partial E$ and $\sigma_{2}\in\partial F$. So we can
indeed conclude that

$\ker
q_{I}\otimes_{\min}q_{J}=\bigcap\\{\ker\sigma_{1}\otimes_{\min}\sigma_{2}\mid(\sigma_{1},\sigma_{2})\in\partial
E\times\partial F\\}.$

Moreover, since $(\sigma_{1},\sigma_{2})\in\partial E\times\partial F$ implies
that $\sigma_{1}\otimes_{\min}\sigma_{2}\in\partial(E\otimes_{\min}F)$ by
Lemma 7, we have that

$\bigcap\\{\ker\sigma_{1}\otimes_{\min}\sigma_{2}\mid(\sigma_{1},\sigma_{2})\in\partial
E\times\partial
F\\}\supseteq\bigcap\\{\ker\sigma\mid\sigma\in\partial(E\otimes_{\min}F)\\}.$

So indeed, $\ker q_{I}\otimes_{\min}q_{J}$ contains the Šilov boundary ideal.
∎

We now arrive at the proof of Theorem 1, where we prove that $\ker
q_{I}\otimes_{\min}q_{J}$ satisfies the properties of the Šilov ideal. Since
we can use the Šilov ideal to construct the $C^{*}$-envelope, this immediately
leads to a result about $C^{*}$-envelopes.

###### Proof of Theorem 1.

By Corollary 9 we know that $\ker q_{I}\otimes_{\min}q_{J}$ contains the Šilov
boundary ideal for $E\otimes_{\min}F$. But by Corollary 6 we know that it is a
boundary ideal itself. Since the Šilov boundary ideal is the maximal boundary
ideal, we must have that $\ker q_{I}\otimes_{\min}q_{J}$ is the Šilov boundary
ideal for $E\otimes_{\min}F\subseteq B(H\overline{\otimes}K)$.

In particular this means that

$C^{*}_{env}(E\otimes_{\min}F)\cong C^{*}(E\otimes_{\min}F)/\ker
q_{I}\otimes_{\min}q_{J}$ $=C^{*}(E)\otimes_{\min}C^{*}(F)/\ker
q_{I}\otimes_{\min}q_{J}.$

Because $q_{I}:C^{*}(E)\longrightarrow C^{*}(E)/I\cong C^{*}_{env}(E)$ and
$q_{J}:C^{*}(F)\longrightarrow C^{*}(F)/I\cong C^{*}_{env}(F)$, both being
surjective *-homomorphisms, we have that

$q_{I}\otimes_{\min}q_{J}:C^{*}(E)\otimes_{\min}C^{*}(F)\longrightarrow
C^{*}_{env}(E)\otimes_{\min}C^{*}_{env}(F)$

is a surjective *-homomorphism, so

$C^{*}(E)\otimes_{\min}C^{*}(F)/\ker q_{I}\otimes_{\min}q_{J}\cong
C^{*}_{env}(E)\otimes_{\min}C^{*}_{env}(F),$

proving the theorem. ∎

##  4 Application: the propagation number

In [1], Connes and Van Suijlekom define a property of operator systems called
the propagation number. The definition of this property is based on the
$C^{*}$-envelope. Using Theorem 1 we can fairly directly determine the
behaviour of this property under the tensor product.

######  Definition 5.

For $E\subseteq C^{*}(E)$ a concrete operator system, let

$E^{\circ n}:=\overline{\textup{span}}\\{e_{1}\cdot\ldots\cdot e_{n}\mid
e_{i}\in E\textup{ for }i=1,\ldots,n\\}\subseteq C^{*}(E).$

Let $(i_{E},C^{*}_{env}(E))$ be the $C^{*}$-envelope of $E$. Then the
propagation number is defined as

$\textup{prop}(E):=\inf\\{n\in\mathbb{N}\mid(i_{E}(E))^{\circ
n}=C^{*}_{env}(E)\\}.$

######  Proposition 10.

Let $E\subseteq A$ and $F\subseteq B$ be concrete operator systems. Then

$E^{\circ n}\otimes_{\min}F^{\circ n}=(E\otimes_{\min}F)^{\circ n}\subseteq
A\otimes_{\min}B.$

###### Proof.

Because

$\left(\sum_{i}e_{1i}\otimes f_{1i}\right)\left(\sum_{j}e_{2j}\otimes
f_{2j}\right)=\sum_{i,j}(e_{1i}e_{2j})\otimes(f_{1i}f_{2j})$

we have that

$\\{x_{1}x_{2}\cdots x_{n}\mid x_{i}\in E\otimes F\\}\subseteq E^{\circ
n}\otimes F^{\circ n}\subseteq E^{\circ n}\otimes_{\min}F^{\circ n}.$

Each element in $(E\otimes_{\min}F)^{\circ n}$ is of the form
$y_{1}y_{2}\cdots y_{n}$ for $y_{i}\in E\otimes_{\min}F$, while each $y_{i}$
is the limit of some sequence $(x_{ij})_{j=1}^{\infty}$ in $E\otimes F$. But
by continuity of multiplication, this means that $x_{1j}x_{2j}\cdots
x_{nj}\rightarrow y_{1}y_{2}\cdots y_{n}$ as $j\rightarrow\infty$ and so

$(E\otimes_{\min}F)^{\circ n}\subseteq\overline{\\{x_{1}x_{2}\cdots x_{n}\mid
x_{i}\in E\otimes F\\}}\subseteq E^{\circ n}\otimes_{\min}F^{\circ n}.$

Conversely, an element of the form $\sum_{i}(e_{i1}e_{i2}\ldots
e_{in})\otimes(f_{i1}f_{i2}\ldots f_{in})$ can be seen to lie in the linear
span of elements of the form $(e_{i1}\otimes f_{i1})(e_{2i}\otimes
f_{2i})\ldots(e_{in}\otimes f_{in})\in(E\otimes_{\min}F)^{\circ n}$. So
$E^{\circ n}\otimes F^{\circ n}\subseteq(E\otimes_{\min}F)^{\circ n}$, and by
the fact that $(E\otimes_{\min}F)^{\circ n}$ is closed, we have that $E^{\circ
n}\otimes_{\min}F^{\circ n}\subseteq(E\otimes_{\min}F)^{\circ n}$. ∎

######  Lemma 11.

Let $A$ and $B$ be $C^{*}$-algebras, and let $S\subsetneq A$ be a proper
closed subspace. Then $S\otimes_{min}B$ is a proper subspace of
$A\otimes_{\min}B$.

###### Proof.

Take $x\in A\setminus S$, and let $f_{x}\in A^{*}$ with $f_{x}(x)=1$ and
$f_{x}(S)=\\{0\\}$ (such an element exists by a standard separation result,
e.g. [14, Corollary IV.3.15]). Take $b\in B$ nonzero, and $f_{b}\in B^{*}$
such that $f_{b}(b)=1$. Then define the continuous functional

$g:=f_{x}\otimes_{\min}f_{b}:A\otimes_{\min}B\longrightarrow\mathbb{C}\otimes\mathbb{C}\cong\mathbb{C}.$

Clearly, $g(x\otimes b)=1$, and $g(S\otimes B)=\\{0\\}$. So by continuity we
have $g(S\otimes_{\min}B)=\\{0\\}$, and we must therefore have that $x\otimes
b\not\in S\otimes_{\min}B$. ∎

######  Theorem 12.

Let $E$ and $F$ be operator systems. Then

$\textup{prop}(E\otimes_{\min}F)=\max\\{\textup{prop}(E),\textup{prop}(F)\\}.$

###### Proof.

We identify $E\otimes_{\min}F$ with its image in its $C^{*}$-envelope, which
by Theorem 1 is $C^{*}_{env}(E)\otimes_{\min}C^{*}_{env}(F)$. Let
$M=\max\\{\textup{prop}(E),\textup{prop}(F)\\}$. Then

$(E\otimes_{\min}F)^{\circ M}=E^{\circ M}\otimes_{\min}F^{\circ
M}=C^{*}_{env}(E)\otimes_{\min}C^{*}_{env}(F),$

so $\textup{prop}(E\otimes_{\min}F)\leq M$. Conversely, if we have that
$m:=\textup{prop}(E\otimes_{\min}F)<M$, then either $E^{(m)}\subsetneq
C^{*}_{env}(E)$ or $F^{(m)}\subsetneq C^{*}_{env}(F)$. So by Lemma 11 this
means that

$(E\otimes_{\min}F)^{\circ m}=E^{\circ m}\otimes_{\min}F^{\circ m}\neq
C^{*}_{env}(E)\otimes_{\min}C^{*}_{env}(F),$

but this is a contradiction because $m=\textup{prop}(E\otimes_{\min}F)$. So

$M=\max\\{\textup{prop}(E),\textup{prop}(F)\\}=\textup{prop}(E\otimes_{\min}F)$

as claimed. ∎

######  Remark.

Interestingly, while in this article we only work with unital operator
systems, the equivalent result of Theorem 12 for non-unital operator systems
would directly prove invariance of the propagation number under stable
equivalence. This was already proven by a different method in [1]. Also
interesting to mention in this regard are the recent results by Eleftherakis,
Kakariadis and Todorov [15], where they discuss Morita equivalence for
operator systems, and show that two operator systems are equivalent in this
way if and only if they are stably equivalent. So this implies that the
propagation number is also an invariant under this type of Morita equivalence.

### Acknowledgements

I would like to sincerely thank Walter van Suijlekom, for the feedback in
regards to this article, as well as for his support and guidance during the
writing of the Master’s thesis from which this article is adapted.

## References

  * [1] A. Connes and W. D. van Suijlekom, “Spectral Truncations in Noncommutative Geometry and Operator Systems,” Commun. Math. Phys., vol. 383, pp. 2021–2067, May 2021.
  * [2] V. Paulsen, Completely Bounded Maps and Operator Algebras.  Cambridge Studies in Advanced Mathematics, Cambridge University Press, 2003.
  * [3] M. Hamana, “Injective envelopes of operator systems,” Publications of the Research Institute for Mathematical Sciences, vol. 15, no. 3, pp. 773–785, 1979.
  * [4] W. B. Arveson, “Subalgebras of $C^{*}$-algebras,” Acta Mathematica, vol. 123, pp. 141 – 224, 1969.
  * [5] W. Arveson, “The noncommutative Choquet boundary,” Journal of the American Mathematical Society, vol. 21, pp. 1065–1084, Oct. 2008.
  * [6] K. R. Davidson and M. Kennedy, “The Choquet boundary of an operator system,” Duke Mathematical Journal, vol. 164, no. 15, pp. 2989 – 3004, 2015.
  * [7] D. P. Blecher, C. Le Merdy, et al., Operator algebras and their modules: an operator space approach.  No. 30 in Londen Mathematical Society monographs: new series, Clarendon Press, 2004.
  * [8] E. Effros and Z. Ruan, Operator Spaces.  London Mathematical Society monographs, Clarendon Press, 2000.
  * [9] G. Pisier, Introduction to Operator Space Theory.  London Mathematical Society Lecture Note Series, Cambridge University Press, 2003.
  * [10] A. Hopenwasser, “Boundary Representations and Tensor Products of $C^{\ast}$-Algebras,” Proceedings of the American Mathematical Society, vol. 71, no. 1, pp. 95–98, 1978.
  * [11] M. Takesaki, Theory of Operator Algebras I.  Encyclopaedia of Mathematical Sciences, Springer Berlin Heidelberg, 2001\.
  * [12] A. J. Lazar, “The space of ideals in the minimal tensor product of c*-algebras,” Mathematical Proceedings of the Cambridge Philosophical Society, vol. 148, no. 2, p. 243–252, 2010.
  * [13] M. Rørdam and E. Størmer, Classification of Nuclear $C^{*}$-Algebras. Entropy in Operator Algebras.  Encyclopaedia of Mathematical Sciences, Springer Berlin Heidelberg, 2001\.
  * [14] J. Conway, A Course in Functional Analysis.  Graduate Texts in Mathematics, Springer New York, 1994.
  * [15] G. K. Eleftherakis, E. T. A. Kakariadis, and I. G. Todorov, “Morita equivalence for operator systems,” 2021.