# San-BERT: Extractive Summarization for Sanskrit Documents using BERT and its
variants

Kartik Bhatnagar
Department of Mathematics and Computer Science
Sri Sathya Sai Institute of Higher Learning
Prashanti Nilayam, Andra Pradesh, IN 515134
<EMAIL_ADDRESS>
Sampath Lonka
Department of Mathematics and Computer Science
Sri Sathya Sai Institute of Higher Learning
Prashanti Nilayam, Andra Pradesh, IN 515134
<EMAIL_ADDRESS>
Jammi Kunal
Department of Mathematics and Computer Science
Sri Sathya Sai Institute of Higher Learning
Prashanti Nilayam, Andra Pradesh, IN 515134
<EMAIL_ADDRESS>
Mahabala Rao M G
Department of Language and Literature
Sri Sathya Sai Institute of Higher Learning
Prashanti Nilayam, Andra Pradesh, IN 515134
<EMAIL_ADDRESS>

###### Abstract

In this work, we develop language models for the Sanskrit language, namely
Bidirectional Encoder Representations from Transformers (BERT) and its
variants: A Lite BERT (ALBERT), and Robustly Optimized BERT (RoBERTa) using
Devanagari Sanskrit text corpus. Then we extracted the features for the given
text from these models. We applied the dimensional reduction and clustering
techniques on the features to generate an extractive summary for a given
Sanskrit document. Along with the extractive text summarization techniques, we
have also created and released a Sanskrit Devanagari text corpus publicly.

_K_ eywords Sanskrit Documents $\cdot$ Extractive Summarization $\cdot$ Word
Embedding $\cdot$ Clustering $\cdot$ ROUGE Score $\cdot$ BERT Score

##  1 Introduction

Sanskrit is an ancient and incredibly important language that has had an
incredible impact on Indian literature. After all, it is the source of a vast
body of scientific and literary works covering astronomy, mathematics,
medicine, philosophy, technology, and so much more. Language was used as the
main form of communication for scholarly discourses across India up until
quite recently. As a result, Sanskrit literature has kept up a steady
production of work throughout its two-thousand-year history [1]. Much of this
literature has not yet been explored by modern humans. There is a wealth of
knowledge stored in Sanskrit texts and ancient manuscripts that the public is
unaware of. When information is abundant everywhere, people rarely have time
to read a complete document in today’s society. Having a summary of the
content is preferable to avoid the tedious process of reading the entire
document. An excellent summary can obtain crucial information and key points
from the entire document, saving the reader time. Today, various summarization
strategies are accessible, ranging from rule-based, statistical-based, graph-
based, and current state-of-the-art deep learning-based techniques. Many
Natural-Language-based jobs are improving due to recent advancements in the
field of Deep Learning, and summarization is one of them. There are primarily
two types of summarization techniques: abstractive and extractive. In
abstractive text summarization, we summarize the crucial information by
paraphrasing all of the relevant statements in the document. In extractive
text summarization, on the other hand, we extract the important sentences from
a document and make it a summary. The extractive summary condenses all
significant sentences that address the documents underlying theme and presents
them in the form of a summary. Although people normally produce summaries in
abstract ways, extractive summarization algorithms have received greater
attention in recent studies [2]. Extractive summarizing systems perform better
than abstractive summarizing systems in many cases [3] based on available
evaluation metrics. We have the following findings in this work:

  1. (a)

Developed a digitalized Devanagari Sanskrit corpus and made the data available
to the public111https://www.kaggle.com/datasets/kartikbhatnagar18/sanskrit-
text-corpus.

  2. (b)

We have trained a set of language models for the Sanskrit language, including
BERT, ALBERT and RoBERTa. There are different word-embedding techniques
discussed for Sanskrit in [4] using transliteration of Sanskrit text corpus.
As far as we know, ours is the first attempt to develop BERT and its variants
using the Devanagari script for Sanskrit.

  3. (c)

We developed an extractive summary through the extraction of features from
language models. Following that, we evaluated the generated summary utilizing
ROUGE scores [5] and BERT scores [6].

Most of these results are part of the first author’s Master Dissertation [7].

##  2 Previous Works

Luhn H.P [8] initially presented a concept behind text summarization in 1958,
based on word frequency. A further refinement of his ideas was proposed in
[9], which suggested that sentence position and word order should be
considered when assessing the importance of sentences to form an effective
summary. Since the 1970s, researchers have been developing rule-based
approaches for summarization, but the problem they faced was that the rules
were not generic; for different domain documents, they had to make different
rules for summarization algorithms. In the 1990s, several graph-based
techniques emerged, but they struggled to account for linguistic features. Due
to the furtherance in computer linguistics, modern machine learning, and deep
learning, text summarization has made immense strides in recent decades.

There are different techniques developed for text summarization both
abstractive and extractive using machine learning models. As these models are
data-driven, supervised [10],[11],[12], unsupervised [13], [3], [14], [15] and
Reinforcement Learning techniques [16], [17] developed to generate the
extractive summary. The supervised approach to summarizing requires a labeled
data set. For Sanskrit, it can often be hard to find labeled data suitable for
training a model. In this situation, an unsupervised approach is extremely
advantageous.

To our knowledge, Barve et al. [18] were the first researchers to generate
extractive summaries from Sanskrit text. They applied various unsupervised
query-based summarization approaches, namely, Vector Space Models (VSM),
Graph-based methods using PageRank, and term-frequency and inverse-sentence
frequency (tf-isf). Using these approaches extracted the sentences which are
more relevant to the given query and ranked all the extracted sentences in
descending order. The top-ranked sentences together form an extractive
summary. Then, the generated summaries are evaluated by human experts in
Sanskrit, and the results are shown in [18].

Note that the results shown in both works [18], [19] are limited to a single
page or small paragraphs.

##  3 Language Models for Sanskrit

Bidirectional Encoder Representations from Transformers [20] (BERT) [21],
models have achieved state-of-the-art performance for many natural language
processing (NLP) tasks. To use BERT models for various NLP tasks in Sanskrit,
we will need to train the appropriate BERT family model(s). As such, a data-
driven approach is taken and it is critical to have large corpora to support
these models during training and testing. In this section, we discuss the
training process required to effectively implement various BERT models.

###  3.1 Data

We scraped Sanskrit data in Devanagari script from different resources
222https://sa.wikipedia.org/wiki/ 333https://oscar-
corpus.com/post/oscar-v22-01/ 444https://www.sanskritworld.in/
555SanskritDocuments.org 666http://sanskrit.jnu.ac.in/currentSanskritProse/
777https://sanskrit.uohyd.ac.in/Corpus/ we used Python packages Beautiful Soup
4.9.0 and requests 2.27.1 to extract the data from HTML pages. In addition, we
remove extra spaces, extra punctuation marks, website links, and unwanted
characters other than the Devanagari script from the scraped data using
functions from the Regex Python library. After cleaning the document, the
entire text present is split on the basis of the delimiter purnavirama
(’$|$’), resulting in creating the list of sentences. After cleaning, we have
800 MB of Sanskrit Text Corpus (STC) for our training purpose and it is
available for public
888https://www.kaggle.com/datasets/kartikbhatnagar18/sanskrit-text-corpus. We
have a total of 30M tokens in the STC.

###  3.2 Training Language Models

The architecture and configurations of the models BERT, ALBERT, and RoBERTa
are taken from Hugging Face999https://huggingface.com/models.

####  3.2.1 San-BERT

Bidirectional Encoder-Representation from Transformer (BERT) [21] is a
transformer-based natural language processing model that uses surrounding text
to help computers understand what words imply in context. Unlike other
standard language models, which can only read sequential data in one
direction, the BERT model can read data in both directions, from left to right
and vice versa, resulting in improved learning. There are majorly two tasks on
which BERT Model is prepared, one is Masked-Language Model (MLM), and the
other is Next-Sentence Prediction (NSP). In the MLM task, throughout the text
data, few words are masked, and BERT tries to predict those words, and in the
NSP task, BERT tries to find out whether the preceding line is logical or
correct. These two tasks help the model learn the inner parameters, which are
used to perform downstream tasks. For fine-tuning BERT for our corpus, we use
only the MLM task. For the training BERT, since the NLP community avoids BERT
models to train on NSP task, as it is an easy task for the model to in
comparison to the MLM task, and it does help the model to learn better, so
following the same line we trained our BERT model in MLM task. The first task
during the training of the BERT model is preparing the vocabulary; generally,
BERT models have around 30k tokens in their vocabulary, which are sufficient
enough to represent any text data for training. To create our Vocabulary for
Sanskrit text we used BERT Word Piece Tokenizer, the tokenizers have the
capability to deal with the problem of Out Of Vocabulary Word (OOV) Problem.
The tokenizer keeps the whole word in the vocabulary, or in some cases a whole
word is divided into different parts; in the latter case, the broken word is
prefixed with a double hash($\\#\\#$), symbolizing it as part of the whole
word. So, if in case a larger word comes to the model which is not directly
represented in the vocabulary, then in this case, to deal with the problem of
out-of-vocabulary (OOV), the tokenizer will simply break the words into
smaller pieces until it matches its sub-word kept prefixed with double
hash($\\#\\#$).The use of creating the vocabulary is that we can represent a
text (line or paragraph) to its equivalent token id, as the computer would not
accept a word directly; rather it would be suitable to represent a word in its
equivalent mathematical form. The input taken by a BERT model for training is
text data in its equivalent token ids (input ids), attention masks, and
special token masks. For our task, we have trained the BERT model on our data
set with MLM task, on Lenovo Legion RTX 3060 6 GB Nvidia Graphic card, CPU
processor AMB 5800 H, SSD 256 GB for 18 hours.

####  3.2.2 San-ALBERT

BERT models have revolutionized the field of NLP with their outstanding
results; the disadvantage of BERT models is their giant architecture size. For
a BERT model, we need to train about 340M parameters, and it is a huge task to
do it on a personal computer. In order to reduce the computational parameters
and the training time, we will be using another variant of the BERT model,
that is, a Lite BERT (ALBERT) [22]. Using the novel techniques ALBERT model
reduces around 90 percent of the parameters in comparison to the BERT model
and it still can maintain its performance and to scale up to the level of
BERT. The tokenizer used for training the vocabulary is Sentence-Piece
Trainer. With the help of a Sentence-Piece trainer, around 32k tokens were
produced. These tokens were constituents of words in their full form, subwords
prefixed with a double hash ($\\#\\#$), and some special tokens such as [CLS],
[SEP], [MASK]. After making the vocabulary, we imported the ALBERTTokenizer
function to tokenize throughout the corpus, which will be given for training.
The same Masked Language Model (MLM) training task was used to train the
ALBERT model, around 15% of the words were masked for the training. We trained
our ALBERT model on the Google Colaboratory, Tesla K-80 GPU with driver
version 460.32.03 and CUDA version 11.2 for about 9 hours. Around 80 percent
of the text was in the training file and around 20 percent of the text data
was in the validation file.

####  3.2.3 San-RoBERTa

Robustly optimized BERT (RoBERTa) [23] is another variant of the BERT model;
it was developed by Meta AI researchers; they thoroughly examined the BERT
architecture, which was developed by Google, and found the various
optimizations to be done in the architecture to perform high quality related
to natural languages. In RoBERTa, there has been an improvement about the
hyperparameter and the learning rate is also comparatively higher, which helps
the model to learn faster. The Next Sentence Prediction (NSP) task is removed
in the RoBERTa Model and the Masked Language Modeling (MLM) task is improved.
With such advantages being offered by the RoBERTa Model, we also trained a
RoBERTa model on our corpus. The tokenizer that we used to make the vocabulary
is the Byte Level tokenizer namely _Bpet RoBERTa_ tokenizer. We made a
vocabulary of around 30K tokens and send the whole corpus through the
tokenizer. The RoBERTa tokenizer is different from other tokenizers, it uses
byte level tokenizer which is also used in GPT-2 [24] architectures, and the
schemas used in pertaining techniques are different, which helps the model to
give better representation for training. The special tokens used by the
tokenizer are: $<s>,\ <pad>,\ </s>,\ <unk>,\ <mask>.$

From the Hugging
Face101010https://huggingface.co/docs/transformers/model$\\_$doc/roberta, we
imported RobertaTokenizer, RobertaConfig, and RobertaForMaskedLM for our
training purpose. We used PyTorch to represent labels, mask tokens, input ids,
and mask arrays in the form of tensors, which are then fed to the input layer
of the RoBERTa Model. In the input layer we adjusted the max length of the
tensor to 514, the hidden layer size is 768, and the number of hidden layers
is set to 6. The learning rate we used for training is $1e-4.$ We trained this
model on an Nvidia Tesla P100 GPU and trained the model for around 18 hours.

###  3.3 Embedding Representation

The purpose of training the BERT, ALBERT, and RoBERTa models was to get a
better-contextualized representation embedding of any given Sanskrit word.
Contextualized embedding generates a separate representation for each sense of
meaning depending on the context words surrounding [4]. These embeddings will
be the representation of the words present in the document, for which we have
to obtain the extractive summary.

To demonstrate the performance of our San-BERT model, we ran embedding for six
sentences taken from the analysis of [4]. After applying dimension reduction
of output to three dimensions, the results are shown in fig.1. In the first
three examples, the Word $chakr$ means the $Potter$ and has the same word
embedding, and in the following three sentences the word $chakr$ has a
different meaning and so the embedding is different.

Figure 1: Vector representation of the word $chakr$ in a 3-dimensional space
using Contextualized word Embedding

For any given document we will be getting the embeddings in two stages, the
first is Word Embedding and the other is Sentence Embedding.

##  4 Extractive Text Summarization Techniques

In this section, we explore several extractive summarization techniques to
generate the summary from Sanskrit documents. We discussed two approaches to
get the extractive summary for given Sanskrit document(s): statistical-based
and neural-based. The details of each technique are discussed below:

###  4.1 Statistical Based Technique

The Term Frequency - Inverse Document Frequency (TF-IDF) is a statistical
technique, which helps to find the significance of a given word in the entire
document. TF-IDF calculates the score of a given word based on its repetitive
occurrences in a document and also holds into account the word’s occurrence in
the other documents. TF-IDF will help to capture the importance of words
throughout the document by calculating the word occurrence in a particular
sentence (TF) and also by calculating the number of sentences constituting
that word (IDF). Together, TF and IDF, we calculate the TF-IDF score for each
word $w$ as follows:

$TF-IDF(w)=TF(w)*IDF(w)$

We use these scores for each word in a given sentence $S,$ then we compute the
sentence score _Sentence TF-IDF Score_ as follows:

$Sentence\ TF-IDF\ Score={\sum_{w\in S}TF-IDF(w)\over|S|}$

We sort the sentences based on sentence TF-IDF Scores, then select the first
$k$ sentences. We generate the extractive summary by reordering the top $k$
sentences. The procedure is shown in Figure 2.

Figure 2: Pipeline for Extractive text Summary using TF-IDF Scores

###  4.2 Neural Based Techniques

In this subsection, we discuss how to generate an interactive summary using
the BERT family of models.

####  4.2.1 Overall Workflow

For generating the extractive summary from the Sanskrit document(s), we use
the preprocessed document(s). The list of sentences from the documents is
passed through the model, where we will obtain the word embedding from the
trained BERT models, and sentence embedding is obtained by averaging all the
word embedding present in a sentence. Then with the help of the K-means
clustering technique, we will make the clusters of sentence embedding vectors,
then we will figure out the number of optimal clusters formed by the
embeddings through the elbow method. After obtaining the number of clusters
formed, the next task is to find out the central embedding in those clusters.
In the next step, we will find the distance of each sentence embedding with
the respective cluster’s central embedding via a cosine distance measure.
After having all the distances of sentences, we will sort the sentences
according to the distances, the sentences having less distance from the
central sentence will have a higher rank than the sentence far away from the
center of the cluster. After ranking the sentences, the next step will be to
select the top $k$ sentences to be displayed in summary. The next step after
selecting the top - k-sentences is to reorder them back in chronological
order. These selected important sentences will be the extractive summary of
the whole document(s).

Figure 3: Pipeline for Extractive text Summary using variants of BERT Models

####  4.2.2 Token Embeddings

In the BERT model, ALBERT model and RoBERTa model we have 12 encoders, 1
encoder and 6 encoders layers, respectively. The encoder layer is followed by
the output layer, this layer gives out the embedding of each token present in
the input. The number of embeddings coming out of the model is equal to the
number of tokens produced by the tokenizer for a given piece of input text.

####  4.2.3 Sentence Embedding

For a given document(s) our input to the BERT models will be in the form of
sentences. These sentences will be obtained from the preprocessing and data
cleaning stage. Then for each sentence passed to a BERT model we will be
getting different embeddings for each token present in the sentence. In order
to get the Sentence embedding, we will average out all the token embeddings
obtained.

###  4.3 Dimension Reduction

The Embeddings of a single sentence we are obtaining from the BERT models are
of very high dimension (768), and in a document(s) we will have numerous
sentences. There are various disadvantages to using data in higher dimensions,
the foremost of which are high data storage, and high data computation. The
distances between the closest and farthest data points can become equidistant
in high dimensions, limiting the accuracy of some distance-based analytic
techniques. To deal with the problems given by the higher-dimensional data,
the best option is to reduce the data dimension. Dimensionality reduction aids
in the resolution of these above-mentioned challenges while striving to
preserve the majority of the important data. The more features there are in
the data it will be difficult to visualize and then operate on the training
set. Most of these traits are frequently linked and thus redundant. In this
case, dimensionality reduction techniques are useful. The dimension reduction
we are going to use is PCA ( Principal-Component Analysis), which discovers
the relationships between the data. PCA in the high-dimensional data finds the
highest-variance directions and then projects them onto a new subspace with
fewer dimensions in comparison to the original data. Thus PCA will be helpful
to us to reduce the Sentence embedding dimensions and work efficiently with
the sentence embedding in the lower dimension space.

###  4.4 Clustering

A whole document(s) will not be a constituent of the same topic or theme,
within a document there will be different subtopics and different themes
present in it. So to capture capturing all the themes and subtopics within a
document(s), in an Extractive Summary, we will be using the Clustering
technique.

Figure 4: Different Clusters Formed by the Sentences Embedded within a
Document

After obtaining the sentence embeddings of all the sentences present
throughout the document(s), we will find the optimal number of clusters formed
by those embeddings with the help of the Elbow method. Based on our
experiments on average we are getting around the optimal 3 to 4 clusters for a
document. After finding out the optimal number of clusters, we will use the
K-means clustering technique and assign sentences to its cluster. Sentences
present in the same cluster represent the same subtopic or theme of the
document. Thus considering this idea will give the extractive summary
consisting of the main points of each sub-topic and theme.

###  4.5 Sentence Ranking

In the next step after assigning each sentence to its cluster, we will be
finding out the distance of each embedding from the center of its cluster. We
used various distance measures like Manhattan distance, Euclidean Distance,
Cosine similarity measure, and cosine distance for finding out the distances.
It turned out that cosine similarity and cosine distance measures were giving
more accurate results. So, based on the cosine similarity and cosine distances
we will rank the sentences. The sentence embedding whose cosine distance from
the center of its cluster is minimal will have a higher rank and those
sentence embeddings, with a cosine distance is more will have a poor rank.
Following this procedure, we will have all the sentences sorted based on their
ranks.

###  4.6 Extractive Summary

After obtaining the sentences according to their ranks, the next task is to
select the top $k$ sentences that are going to be part of the final extractive
summary. We made our function dynamic, users can select the number of lines
they want in their Extractive Summary. It is generally preferred to have 1/5th
of the total lines as a summary, as it can cover all the aspects of the
document(s) in a condensed form. After selecting the top sentences, the next
process is to reorder them back in the chronological order of their
occurrence. Our neural-based models are capable of generating a summarization
of any Sanskrit documents in Devanagari script and condensing essential,
relevant information into one concise summary.

##  5 Results

In this section, we evaluate our models and present the results. We evaluated
our models in both cases: smaller documents (10 - 15 sentences) and longer
documents (A couple of pages).

###  5.1 Evaluation Metrics: Rouge Scores

ROUGE is a standard evaluation measure for summarization tasks, and it stands
for Recall-Oriented Understudy for Gisting Evaluation. This metric was
initially introduced in the paper [5] and considers both Precision and Recall
between predicted and reference summaries. It includes three scores: ROUGE-1,
ROUGE-2, and ROUGE-L.

  * •

Recall (R): Recall calculates the total number of n-grams present in the
predicted summary and the reference and then it gets divided by the total
number of n-grams present in the reference summary.

$R=\frac{Count_{match}(gram_{n})}{Count(ref-gram_{n})}$

  * •

Precision (P): It is calculated in almost the exact same way like Recall, but
rather than dividing by the reference n-gram count, we divide by the predicted
n-gram count.

$P=\frac{Count_{match}(gram_{n})}{Count(cand-gram_{n})}$

  * •

F1-Score(F): That gives us a reliable measure of our model performance that
relies not only on the model capturing as many words as possible (recall) but
doing so without outputting irrelevant words (precision).

$F1-Score(F)=2*\frac{precision\ *\ recall}{precision\ +\ recall}$

  * •

Rouge1, Rouge2, Rouge L Scores: Rouge-1 matches the number of Uni grams with
our machine generated summary and with the referenced summary. And in the case
of Rouge-2 it matches the bi-grams. Rouge-L is used to measure the longest
sequences of sharing tokens between our reference summary and the machine
generated summary.

###  5.2 BERT Score

BERT Score [6] is an evaluation metric for text generation tasks. We evaluate
our candid-reference summaries by computing BERT Score. For each token $x$ in
reference summary, the BERT embedding is denoted as $\textbf{x}.$ Similarly,
for each token $\hat{x}$ in candid summary, the BERT embedding is denoted as
$\hat{\textbf{x}}.$ For given sentence, assume that that
$\textbf{x}=\langle\textbf{x}_{1},\ldots,\textbf{x}_{m}\rangle$ and
$\hat{\textbf{x}}=\langle\hat{\textbf{x}}_{1},\ldots,\hat{\textbf{x}}_{\ell}\rangle$
are BERT embedding of reference $x$ and candid $\hat{x}$ summaries
respectively. Recall, precision, and F1 scores for $x$ and $\hat{x}$ are
defined as follows:

$R_{BERT}={1\over|x|}\sum_{x_{i}\in
x}max_{\hat{x}_{j}\in\hat{x}}\textbf{x}_{i}^{T}\hat{\textbf{x}}_{j},\
P_{BERT}={1\over|\hat{x}|}\sum_{\hat{x}_{j}\in\hat{x}}max_{x_{i}\in
x}\textbf{x}_{i}^{T}\hat{\textbf{x}}_{j},$ $F_{BERT}=2{P_{BERT}.R_{BERT}\over
P_{BERT}+R_{BERT}}$

###  5.3 Results

The following values are observed in our experiments for smaller documents:

  * •

Rouge-1 and Rouge-2 Scores:

| Rouge-1 Score | Rouge-2 Score
---|---|---
Model | recall | precision | f1-score | recall | precision | f1-score
Tf-Idf | 0.49 | 0.64 | 0.55 | 0.46 | 0.57 | 0.51
BERT (Vanila) | 0.51 | 0.58 | 0.54 | 0.46 | 0.52 | 0.49
San-BERT | 0.61 | 0.58 | 0.59 | 0.54 | 0.52 | 0.53
San-ALBERT | 0.38 | 0.71 | 0.69 |  0.666 | 0.64 | 0.65
San-RoBERTa | 0.52 | 0.52 | 0.51 | 0.46 | 0.46 | 0.46

  * •

Rouge-L and BERT Scores:

Model | Rouge-L Score | BERT Score
---|---|---
| recall | precision | f1-score | recall | precision | f1-score
Tf-Idf | 0.48 | 0.62 | 0.54 | 0.892 | 0.921 | 0.906
BERT (Vanila) | 0.48 | 0.56 | 0.52 | 0.904 | 0.913 | 0.908
San-BERT | 0.60 | 0.56 | 0.57 | 0.921 | 0.914 | 0.917
San-ALBERT | 0.69 | 0.70 | 0.68 | 0.937 | 0.942 | 0.939
San-RoBERTa | 0.50 | 0.51 | 0.49 | 0.899 | 0.907 | 0.902

The following values are observed in our experiments for longer documents:

  * •

Rouge-1 and Rouge-2 Scores:

| Rouge-1 Score | Rouge-2 Score
---|---|---
Model | recall | precision | f1-score | recall | precision | f1-score
Tf-Idf | 0.46 | 0.59 | 0.52 | 0.40 | 0.47 | 0.43
Bert (Vanila) | 0.59 | 0.65 | 0.62 | 0.54 | 0.56 | 0.55
San-BERT | 0.72 | 0.72 | 0.71 | 0.66 | 0.64 | 0.64
San-ALBERT | 0.75 | 0.77 | 0.76 | 0.70 | 0.70 | 0.70
San-RoBERTa | 0.50 | 0.58 | 0.54 | 0.46 | 0.49 | 0.47

  * •

Rouge-L and BERT Scores:

| Rouge-L Score | BERT Score
---|---|---
Model | recall | precision | f1-score | recall | precision | f1-score
Tf-Idf | 0.41 | 0.52 | 0.46 | 0.893 | 0.924 | 0.907
Bert (Vanila) | 0.57 | 0.63 | 0.60 | 0.932 | 0.933 | 0.933
San-BERT | 0.704 | 0.707 | 0.70 | 0.941 | 0.929 | 0.935
San-ALBERT | 0.701 | 0.713 | 0.707 | 0.949 | 0.962 | 0.955
San-RoBERTa | 0.50 | 0.58 | 0.54 | 0.909 | 0.927 | 0.918

##  6 Conclusions and Future Works

Our extractive text summarization models performed admirably. The results
obtained using the San-ALBERT and San-BERT models are superior to those
obtained using the San-RoBERTa model and the TF-IDF method. These results may
differ depending on the documents and their related summaries; the referenced
summary we utilized for evaluation was human-generated. Compared to other
extraction processes used in the past, the results are generally favorable. In
future work [25], the authors develop an abstract summarization model for
Sanskrit documents using the San-BERT and Point Generator models.

##  7 Acknowledgements

We dedicate this work to Bhagawan Sri Sathya Sai Baba.

## References

  * [1] Kulkarni Amba and Das Monali.  Discourse analysis of sanskrit texts.  In Proceedings of the Workshop on Advances in Discourse Analysis and its Computational Aspects (ADACA), pages 1–16. COLING 2012, 2012.
  * [2] Mehdi Allahyari, Seyedamin Pouriyeh, Mehdi Assefi, Saeid Safaei Elizabeth D. Trippe, Juan B. Gutierrez, and Krys Kochut.  Text summarization techniques: A brief survey.  International Journal of Advanced Computer Science and Applications, 8(10):397–405, 2017.
  * [3] Günes Erkan and Dragomir R. Radev.  Lexrank: Graph-based lexical centrality as salience in text summarization.  J. Artificial Intelligence Research, 22(1):457–479, dec 2004.
  * [4] Jivnesh Sandhan, Om Adideva, Digumarthi Komal, Laxmidhar Behera, and Pawan Goyal.  Evaluating Neural Word Embeddings for Sanskrit.  arXiv e-prints, April 2021.
  * [5] Chin-Yew Lin.  ROUGE: A package for automatic evaluation of summaries.  In Text Summarization Branches Out, pages 74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics.
  * [6] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi.  Bertscore: Evaluating text generation with BERT.  CoRR, abs/1904.09675, 2019.
  * [7] Karthik Bhatnagar.  Neural extractive summarization for sanskrit documents.  Master’s Dissertation, 2022.
  * [8] H. P. Luhn.  The automatic creation of literature abstracts.  IBM J. Res. Dev., 2(2):159–165, April 1958.
  * [9] H. P. Edmundson.  New methods in automatic extracting.  J. ACM, 16:264–285, 1969.
  * [10] Sarita Mandal, Priya Achary, Shubhada Phalke, KVK Poorvaja, and Madhuri Kulkarni.  Extractive text summarization using supervised learning and natural language processing.  In 2021 International Conference on Intelligent Technologies (CONIT), pages 1–7, 2021.
  * [11] Ramesh Nallapati, Feifei Zhai, and Bowen Zhou.  Summarunner: A recurrent neural network based sequence model for extractive summarization of documents.  CoRR, 2016.
  * [12] Yang Liu.  Fine-tune bert for extractive summarization.  ArXiv, 1903.10318, 2019.
  * [13] Pengjie Ren, Zhumin Chen, Zhaochun Ren, Furu Wei, Jun Ma, and Maarten de Rijke.  Leveraging contextual sentence relations for extractive summarization using a neural attention model.  In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’17, page 95–104, New York, NY, USA, 2017. Association for Computing Machinery.
  * [14] N. Moratanch and Chitrakala Gopalan.  A survey on extractive text summarization.  pages 1–6, 01 2017.
  * [15] Madhurima Dutta, Ajit Kumar Das, Chirantana Mallick, Apurba Sarkar, and Asit K. Das.  A graph based approach on extractive summarization.  In Ajith Abraham, Paramartha Dutta, Jyotsna Kumar Mandal, Abhishek Bhattacharya, and Soumi Dutta, editors, Emerging Technologies in Data Mining and Information Security, pages 179–187, Singapore, 2019. Springer Singapore.
  * [16] Shashi Narayan, Shay B. Cohen, and Mirella Lapata.  Ranking sentences for extractive summarization with reinforcement learning.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1747–1759, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
  * [17] Kaichun Yao, Libo Zhang, Tiejian Luo, and Yanjun Wu.  Deep reinforcement learning for extractive document summarization.  Neurocomputing, 284:52–62, 2018.
  * [18] S. Barve, S. Desai, and R. Sardinha.  Query-based extractive text summarization for sanskrit.  In Proceedings of the 4th International Conference on Frontiers in Intelligent Computing: Theory and Applications (FICTA), volume 404 of Advances in Intelligent Systems and Computing, page 559–568, New Delhi, INDIA, 2015. Springer.
  * [19] Shalini Tomar, Sandeep Rana, and Baldivya Mitra.  Sanskrit language text summarization using modified page rank algorithm.  Journal of Xian University of Architecture and Technology, XIII(6):707–713, 2021.
  * [20] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  CoRR, abs/1706.03762, 2017.
  * [21] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  page 4171–4186, Minneapolis, Minnesota, 6 2019.
  * [22] Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut.  ALBERT: A lite BERT for self-supervised learning of language representations.  CoRR, abs/1909.11942, 2019.
  * [23] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized BERT pretraining approach.  CoRR, abs/1907.11692, 2019.
  * [24] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever.  Language models are unsupervised multitask learners.  2019\.
  * [25] Jammi Kunal and Sampath Lonka.  Abstract Summarization for Sanskrit Documents using Pointer Generated Networks.  preprint, 2023.


# A Graph Analysis of the Impact of COVID-19 on Emergency Housing Shelter
Access Patterns

Geoffrey Messier
University of Calgary
2500 University Dr. NW, Calgary, AB, Canada, T2N 1N4
<EMAIL_ADDRESS>
###### Abstract

This paper investigates how COVID-19 disrupted emergency housing shelter
access patterns in Calgary, Canada and what aspects of these changes persist
to the present day. This analysis will utilize aggregated shelter access data
for over 40,000 individuals from seven major urban shelters dating from 2018
to the present. A graph theoretic approach will be used to examine the
journeys of individuals between shelters before, during and after the COVID-19
lockdown period. This approach treats shelters as nodes in a graph and a
person’s transition between shelter as an arrow or edge between nodes. This
perspective is used to create both timeline and network diagrams that
visualize shelter use and the flow of people between shelters. Statistical
results are also presented that illustrate the differences between the cohorts
of people who only used shelter pre/post-lockdown, people who stayed in
shelter during lockdown and people who used shelter for the first time during
lockdown. The results demonstrate not only how a complex system of care
responded to the pandemic but also the characteristics of the people most
likely to continue to rely on that system during an emergency.

_Keywords_ emergency shelter, housing first, vulnerability indicators, chronic
homelessness, COVID-19 pandemic

##  1 Introduction

Understanding how COVID-19 changed the movement of people through systems of
care is important to gain a retrospective appreciation of the pandemic and to
help plan for future similar events. Now that many of the pandemic related
restrictions have been lifted, it is equally important to understand which
trends established during COVID-19 endure. This later perspective is most
urgent since it informs whether a system of care needs to adapt to more
effectively address the long term effects of the pandemic.

During the initial onset of the COVID-19 pandemic, housing and homelessness
systems of care (HHSCs) across the world transformed in an effort to adopt
measures to limit the spread of the virus [15, 12, 16, 6]. This transformation
was particularly urgent in the congregate living settings of emergency housing
shelters. The pandemic precipitated the opening of temporary additional
spaces, an increased emphasis on housing placements to facilitate exits from
shelter and a range of in-shelter measures designed to reduce disease spread.

The contribution of this paper is to illuminate how the pandemic and its
associated impacts on shelter operations affected the flow of people through
an HHSC in a major North American city. This analysis will focus specifically
on the emergency shelter component of that HHSC. This is important since
shelters typically have the lowest barrier to entry within an HHSC and often
serve the people with the most complex physical and mental health challenges.
Shelters are also congregate living environments which make them particularly
susceptible to viral spread [12].

The data used in this paper captures the flow of 43,263 people between the top
seven busiest emergency shelters in Calgary, Canada from March 1, 2018 to May
1, 2023. Critically, this data provides not just shelter access information
during the height of the pandemic but also for the two years leading up to the
start of the pandemic and for approximately two years after most COVID-19
restrictions were eased. This will demonstrate not only how the Calgary HHSC
changed in response to the pandemic but also which of these changes continue
to persist.

There have been only a limited number studies on how people flow between
shelters due to the often fragmented nature of HHSC shelter data [7, 9].
Beyond these publications, shelter access patterns have mostly been analyzed
by applying clustering techniques to stay and episode data for shelter use
[11, 10, 1] which does not shed light on movement between shelters. The work
in [9] is particularly relevant since it was also done for the Calgary HHSC in
response to the COVID-19 pandemic. However, while COVID-19 motivated [9], the
study was conducted entirely using pre-pandemic data. It is also limited to an
investigation of how often a person chooses to move to a different shelter but
does not provide information on which specific shelter has been chosen. This
is an important omission since understanding which paths between shelters see
the most traffic is a critical part of efficiently allocating resources to
support shelter users during an emergency or otherwise.

This omission can be addressed by creating a picture that shows not only when
someone leaves a particular shelter but also where they have chosen to go.
This is not straightforward since the journey of each person through an HHSC
is unique and often involves interacting with many services in different
orders and for different lengths of time. However, this paper will demonstrate
that this complexity can be managed by representing the HHSC as a directed
graph. Graph theory and visualization is a vast field [2] that has found
utility in better understanding many diverse phenomena including the transit
system, the Internet, social networks [8, 13].

In the following, Section 2 will describe the data set used to study HHSC
shelter use before, during and after the height of the COVID-19 lockdown
restrictions. It also describes how this data is pre-processed to create a
directed graph that represents the flow of people through the HHSC shelter
system. This graph perspective is used to create timeline diagrams, cohort
statistics and network diagram visualizations in Section 3. Section 4 will
discuss the implications of these results and concluding remarks are made in
Section 5.

##  2 Methods

###  2.1 The Data Set

This study is conducted using daily emergency housing shelter access data
collected and aggregated by the Calgary Homeless Foundation (CHF) between
March 1, 2018 and May 1, 2023. Records from multiple shelters for the same
individual are linked and anonymized by the CHF before being released to the
researchers. Each data record consists of an anonymized individual identifier,
shelter access date, shelter name and duration of the shelter stay. The
protocol governing the anonymization, secure storage and analysis of this
secondary data set was approved by the University of Calgary Conjoint Ethics
Review Board (REB-19-0095). The data set contains 43,263 people with records
of accessing an HHSC consisting of four adult shelters, two family shelters
and one seniors’ shelter.

To properly illustrate the effect of the pandemic, it is necessary to divide
the data timeline into three periods. The term “lockdown” will be used to
refer to the period where shelter activity noticeably reduced due to COVID-19
restrictions. The pre-lockdown period stretches from the start of the data on
March 1, 2018 to March 17, 2020, the lockdown period is from March 18, 2020 to
July 1, 2021 and the post-lockdown period is from July 2, 2021 to May 1, 2023.
March 17, 2020 was the declaration of the first COVID-19 related state of
emergency in Alberta [5] and corresponds approximately to the introduction of
widespread COVID-19 restrictions in Canada around the world. July 1, 2021
corresponds to reaching Stage 3 of Alberta’s “Open for Summer” plan [5].

Note that it is not the intent of this analysis to declare COVID-19 “over” on
July 1, 2021. Clearly, both the spread of COVID-19, some pandemic related
restrictions and the long term medical and societal effects of the pandemic
continued to persist after this date. July 1, 2021 is selected since it
approximately marks a noticeable increase in shelter use activity, as will be
illustrated in Section 3.

To more clearly understand the impact of COVID-19 on emergency shelter users,
it is useful to divide the emergency shelter population into cohorts based on
how their first and last days of shelter use co-incide with the COVID-19
lockdown periods. These cohorts, the number of people in each cohort and the
cohort inclusion criteria are shown in Table 1. A 30 day exclusion window
relative to the start and end dates in the data set were used to calculate the
dates in Table 1 in order to reduce the number of people included in the
cohorts who have records of shelter access that are left or right censored by
the start or end of the data set.

Cohort Name | Size |  Criteria
---|---|---
Before Lockdown (Before) | 13654/43263 (31.6%) |  Started after 31/03/2018. Ended before 17/03/2020
Stayed after Lockdown (Stayed) | 7986/43263 (18.5%) |  Started between 31/03/2018 and 17/03/2020. Ended between 31/03/2020 and 01/04/2023
Started during Lockdown (During) | 3874/43263 (9.0%) |  Started between 17/03/2020 and 01/06/2021. Ended before 01/04/2023
After Lockdown (After) | 9672/43263 (22.4%) |  Started after 01/06/2021. Ended before 01/04/2023
Table 1: HHSC population cohorts.

###  2.2 Representing Shelter Access using Graphs

A graph consists of a series of nodes connected by edges [2]. A directed graph
associates a direction of travel from one node to the next. Edges in a
directed graph are typically represented using arrows. A directed graph is
created from the HHSC shelter data described in Section 2.1 where each
emergency shelter is a node. A directed edge is created between two nodes if
one or more shelter users uses the first shelter and then makes a transition
to use the second shelter.

In addition to shelter nodes, the HHSC directed graph will also contain
gateway nodes that represent entries to and exits from the HHSC. The Entry
gateway node marks the point where a person first enters the HHSC and the Exit
node marks the point where a person makes a final exit from the HHSC and no
longer appears in the data set. The Gap gateway node represents when a person
disappears from the data for a period of 30 days or longer but reappears at a
later time to continue making use of the HHSC. Interactions with a housing
placement or other support program would also be recorded as a Gap if the
person was not also actively using shelter at the same time. A Multiple
gateway node is also created to capture when a person interacts with multiple
shelters within a 24 hour period. Creating a composite node is primarily to
simplify the graph visualization since people can access multiple shelters in
a large number of unique combinations. Representing each combination of
shelter access by a unique node would cause an unacceptable amount of clutter
in the graph visualization.

When visualizing a graph, it is useful to adjust the size of nodes and edges
to convey information on relative utilization. For example, a busier shelter
can be represented using a bigger dot and a commonly used transition between
shelters as a thicker arrow. This utilization information can be determined
simply by counting the total number of stays in a shelter and the total number
of times a person makes a transition from one shelter to another.

Gaps in a person’s record of shelter access are common and judging when a gap
is large enough to represent a significant departure from an HHSC is
subjective. However, this study will adopt a threshold of 30 days which is
commonly used to define different episodes of homelessness [11, 10, 1]. This
means that if there is a gap of 30 days or less between a person accessing
first Shelter A and then Shelter B, that will be recorded as a direct
transition from Shelter A to Shelter B. If the gap is longer than 30 days, it
would be recorded as a transition from Shelter A to the Gap node and then a
second transition from the Gap node to Shelter B.

For example, consider an observation window where two people interact with an
HHSC consisting of Shelter A and Shelter B. The data shows that the first
person used Shelter A for a single day. This is recorded as one transition
from Entry to Shelter A, one interaction with Shelter A and one transition
from Shelter A to Exit. The data shows the second person first using Shelter A
for three consecutive days and then using Shelter B 31 days later for a single
day. This is recorded as a transition from Entry to Shelter A, three
interactions with Shelter A, a transition from Shelter A to Gap to account for
the 31 day absence, a transition from Gap to Shelter B, a single interaction
with Shelter B and then a transition from Shelter B to Exit. The directed
graph showing HHSC utilization for these two people is shown in Figure 1.

Figure 1: Example HHSC graph visualization.

##  3 Results

###  3.1 Shelter Stay and Transition Timelines

Figure 2 shows the average number of times people interact with shelters in
the Calgary HHSC from March 1, 2018 to May 1, 2023. The figure also shows the
number of interactions for each of the cohorts in Table 1. Note that average
number of interactions per day will be higher than the more commonly reported
number of unique individuals using the shelter system per day [4] since Figure
2 can include the same person accessing multiple shelters and/or day and night
sleep services in the same day. The figure also shows that shelter use by the
Before cohort seems to taper in anticipation of lockdown. However, this is an
artifact of how the cohort is created. The cohorts in Table 1 are selected to
include entire records of shelter access. Since only a minority of the Before
cohort will have their records finish immediately before the start of
lockdown, a tapering effect is observed. Figure 3 shows the average number of
shelter transitions per day where a transition is calculated as described in
Section 2.2.

Clearly, the total number of times shelter is accessed and the total number of
shelter transitions reduce dramatically at the start of lockdown. Both remain
low until mid 2021. However, since the purpose of this paper is to study how
people move through a system of care, it is worth examining whether the
reduction in transitions reflects a fundamental change in how shelters are
used or if it is primarily an artifact of simply having fewer people in the
system. This can be established using Figure 4 which shows the number of
transitions per day divided by the number of shelter interactions per day.
This figure can be interpreted as the percentage of shelter interactions that
involve a change to a different shelter.

Figure 2: Shelter interaction timeline.

Figure 3: Shelter transition timeline.

Figure 4: Proportion of shelter interactions that involve transition to a
different shelter.

The total number of transitions in Figure 3 for the pre-lockdown era matches
the results in [9] almost exactly. While the shelter interaction values in
Figure 2 cannot be directly compared to the census style results in [9], the
authors of that paper do demonstrate that the overall percentage of people
choosing to switch shelters on any particular day is quite low. This is
consistent with Figure 4.

###  3.2 Cohort Shelter Access Statistics

Clearly, the cohorts of people shown in Figures 2 and 3 utilized the Calgary
HHSC very differently before, during and after lockdown. It is important to
take a closer look at the individuals in each of these groups to better
understand why. Table 2 shows shelter access statistics for the cohorts in
Table 1 during the pre-lockdown, lockdown and post-lockdown periods. The table
shows total number stays, the number of days between a person’s first and last
day using the HHSC (their tenure), their shelter use percentage (number of
stays divided by tenure), number of unique shelters accessed and number of
shelter to shelter transitions. Shelter use percentage is included since it
has been shown to be a good metric to differentiate between long term
steady/chronic and long term sporadic/episodic shelter access patterns [14].

While some cohorts are present during multiple eras, the statistics shown for
each cohort are calculated only using the portion of a person’s record that
falls within an era. For example, pre-lockdown statistics for the Stay cohort
during lockdown were calculated using only the portion of their HHSC access
patterns that fell within the date range of the pre-lockdown period.

| Pre-Lockdown | Lockdown | Post-Lockdown
---|---|---|---
| Cohorts | Cohorts | Cohorts
| Before | Stay | During | Stay | During | Stay | After
Period Duration (days) | 687 | 687 | 381 | 381 | 639 | 639 | 639
Median Tenure (days) | 11 | 123 | 8 | 25 | 75 | 49 | 10
Mean Tenure (days) | 87.6 | 209.7 | 37.6 | 79.4 | 161.5 | 124.2 | 61.8
95th Pctl Tenure (days) | 448 | 612 | 192 | 335 | 552 | 473 | 317
Median Stays | 4 | 19 | 4 | 9 | 11 | 13 | 4
Mean Stays | 22.5 | 46.5 | 17.2 | 27.9 | 36.1 | 33.4 | 21.4
95th Pctl Stays | 99 | 175 | 78 | 124 | 161 | 122 | 95
Median Use Percent | 6.7% | 15.6% | 10.0% | 16.7% | 10.0% | 16.8% | 10.0%
Mean Use Percent | 24.3% | 27.2% | 28.5% | 30.6% | 25.1% | 30.5% | 26.9%
95th Pctl Use Percent | 100% | 100% | 100% | 100% | 100% | 100% | 100%
Median Unique Shelters | 1 | 1 | 1 | 1 | 1 | 1 | 1
Mean Unique Shelters | 1.1 | 1.2 | 1.1 | 1.2 | 1.3 | 1.2 | 1.1
95th Pctl Unique Shelters | 2 | 2 | 2 | 2 | 3 | 2 | 2
Median Transitions | 0 | 0 | 0 | 0 | 0 | 0 | 0
Mean Transitions | 0.4 | 1.5 | 0.3 | 0.8 | 1.5 | 0.8 | 0.4
95th Pctl Transitions | 2 | 8 | 2 | 4 | 7 | 4 | 2
Table 2: Shelter access statistics.

###  3.3 Graph Visualization

Figure 5 shows a visualization of the directed graph representation of the
Calgary HHSC described in Section 2.2 for all shelter users in the pre-
lockdown period. Rather than displaying an absolute count of shelter
interactions and transitions, as shown in Figure 1, the interaction and
transition counts were divided by the duration of the pre-lockdown era (687
days) to arrive at an average number of stays and transitions per day. Edge
thickness is proportional to the number of transitions per day except that
edges representing fewer than one transition/day are unlabeled, shown in light
grey and have a fixed width.

Figure 5: Calgary HHSC shelter utilization before lockdown.

Figures 6 and 7 demonstrate shelter use and shelter transition frequencies
during the lockdown and post-lockdown eras respectively. Both shelter
interaction and transition frequencies are expressed as percentages relative
to the pre-lockdown era. For example, in Figure 6, the average number of
people using the Adult Shelter 2 shelter per day was 54.1% of the average
number per day using that shelter during the pre-lockdown period.

Figure 6: Calgary HHSC shelter utilization during lockdown.

Figure 7: Calgary HHSC shelter utilization after lockdown.

##  4 Discussion

The timeline diagrams in Section 3.1 demonstrate that the drop in shelter use
due to lockdown is remarkably well defined. After approximately July 1, 2021,
use of shelter climbs quickly to levels comparable to the pre-lockdown period,
even though winter 2021 saw the arrival of the Omicron COVID-19 wave and
periodic pandemic restrictions on gathering, workplaces and schools persisted
until early 2022. This return to shelter may have been encouraged by the
uptake of the COVID-19 vaccine during the summer of 2021 and an increased
comfort with using personal protective equipment (PPE) within shelter [3].

While Figures 3 and 4 indicate that transitions between shelters did decrease
during lockdown, Figure 4 indicates that the transitions per shelter
interaction remained relatively constant after an initial dip. Comparing
Figures 2 and 3 also shows that the Stay cohort was responsible for a higher
proportion of shelter transitions than the During cohort. This suggests that
longer term shelter users with established patterns of moving between shelter
will continue those patterns even during emergency. However, consistent with
[9], the most important message from Figure 4 is that only a small percentage
of shelter interactions overall involve a change to a different shelter. This
is important since high traffic between shelters during a pandemic would be a
mechanism to quickly spread infection from one shelter to the next.

Figure 2 also shows that the Stay cohort represents a group of people who were
making heavy use of shelter before lockdown and continued to rely heavily on
the shelter system after the pandemic began. This is supported by Table 2
which shows the Stay cohort using shelter much more frequently than the cohort
who exited the system before lockdown or the cohort who used shelter for the
first time during lockdown. It is intuitive that a group of people making
heavy use of shelter will have few options other than to continue relying on
shelter even during an emergency that would make congregate living much more
hazardous. However, Figure 2 and Table 2 are a reminder than any planning for
a future pandemic response should anticipate continued shelter use by those
who are currently interacting with shelter on a consistent basis.

Table 2 demonstrates that it is very uncommon for a person to interact with a
large number of shelters. Even the 95th percentile of all the cohorts and
periods shown in Table 2 interact with 2 or 3 unique shelters. The median
number of transitions is 0 and only the cohorts with well established records
of shelter interactions have a high number of transitions in the 95th
percentile. However, the number of unique shelters statistics suggest that
even these higher number of transitions are most likely switching back and
forth between the same two shelters.

In Table 2, it is important to note that the cohort in the post-lockdown era
most likely to make shelter transitions are those who started using shelter
for the first time during lockdown. Accessing shelter for the first time
during a pandemic would be a daunting decision. A person making this choice
would very likely be facing multiple challenges that would have left them with
few other options. Now in the post-lockdown period, Table 2 suggests that this
group displays a longer term, sporadic shelter access pattern with a
relatively higher number of shelter transitions. This is unusual and may
identify a unique cohort of people who warrant special attention both by
researchers and support agencies within the HHSC.

While Sections 3.1 and 3.2 provide information on aggregate shelter
interaction and what people experience on the individual level, the
visualizations presented in Section 3.3 demonstrate the highly complex and
interconnected nature of the Calgary HHSC when we visualize the unique sources
and destinations associated with shelter transitions. Figure 5 shows that the
traffic into and out of each shelter is proportional to shelter size (ie.
bigger shelters see more traffic). There is also considerable traffic with the
Gap node due to the episodic nature of how many people interact with emergency
shelters. Interestingly, the traffic into/out of the Multiple shelter node
(indicating interactions with multiple shelters in one day) is approximately
comparable to the sum of the smaller arrows representing direct transition
between shelters. This means that a sizable proportion of inter-shelter
interactions occur in very close proximity.

The network visualization in Figure 5 also provides insight into the nature of
the people accessing particular shelters. For example, the traffic between the
Gap node and Adult Shelter 3 is comparable the traffic between Gap and the
Adult Shelter 1 even though the Adult Shelter 1 is considerably larger. This
suggests that the people using Adult Shelter 3 are more episodic in how they
access shelter which is important information when designing programs for
Adult Shelter 3 users.

Figure 6 shows the overall expected decrease in shelter interactions and
transitions for the lockdown period. In many cases, the a shelter’s transition
traffic and the number of interactions with that shelter decrease by
approximately the same amount. For example, Adult Shelter 1 shelter
interactions drop to 61.6% pre-lockdown levels and the traffic from Entry to
Adult Shelter 1 and Adult Shelter 1 to exit decrease to 60.8% and 54.6%,
respectively. However, there are also some interesting variations. Many
transitions directly between shelters and to/from the Multiple node decrease
by larger margins than the shelter interactions. This suggests a tendency to
“shelter-in-place” during lockdown.

Figure 7 shows post-lockdown shelter use approaching but still below pre-
lockdown utilization. This is mainly because the post-lockdown era, as defined
in this paper, does include approximately one year of shelter operations where
COVID-19 restrictions would have had at least some effect on shelter and
program capacity. The picture is also not uniform. For example, Adult Shelter
3 traffic is at or slightly above pre-lockdown levels. Figure 7 also shows a
higher rate of direct transition to system exit from the top three busiest
shelters: 100.4%, 136.8% and 167.7% for the Adult Shelter 1, Adult Shelter 2
and Adult Shelter 3, respectively. While there are many reasons for leaving an
HHSC, this increase in system exits does co-incide with shelters continuing to
make transitioning people from shelter to housing a priority in the post-
pandemic era.

##  5 Conclusion

The pandemic had a profound impact on the operation of HHSCs and the people
they serve. Understanding this impact in retrospect provides valuable
information on how an HHSC can be better prepared to support people during
future pandemics or other emergencies.

The visualizations presented in Section 3.3 reveal an HHSC consisting of
agencies that are richly connected by the journeys of the people they serve.
Almost every possible transition between agencies was observed. Figure 4 also
indicates that the proportion of people choosing to move between shelters
remains surprisingly robust over the pandemic timeline. At the same time, this
same figure shows that the overall percentage of shelter interactions that
involve a transition to a different shelter remain quite small. This means
that monitoring or attempting to discourage transitions between shelter in
future pandemics may not be the best use of resources.

A higher impact activity would be to better understand the people who continue
to rely on emergency shelters during future pandemics or other emergencies.
Not surprisingly, Table 2 shows that these people tend to be the longer term
or chronic shelter users. These people require different supports than short
term or episodic shelter users and their voices should be included in any
future HHSC pandemic response plans.

Finally, characterizing an HHSC as a directed graph is a powerful tool for
understanding the complex and highly connected way people access the agencies
making up the HHSC. Pandemic or not, system flow visualizations like those
presented in Section 3.3 reveal how agencies are connected by the people they
serve. This perspective should be a powerful motivator for the operators of
those agencies to continue to collaborate and communicate along the lines of
how people flow into and out of their programs. It also encourages us to look
at an HHSC as single system of care since that is how many people choose to
engage with it.

##  6 Acknowledgments

The author would like to gratefully acknowledge the support of Making the
Shift, the Calgary Homeless Foundation and the Government of Alberta. This
study is based in part on data provided by Alberta Seniors, Community and
Social Services. The interpretation and conclusions contained herein are those
of the researchers and do not necessarily represent the views of the
Government of Alberta. Neither the Government of Alberta nor Alberta Seniors,
Community and Social Services express any opinion related to this study.

## References

  * [1] Tim Aubry, Susan Farrell, Stephen W. Hwang and Melissa Calhoun  “Identifying the Patterns of Emergency Shelter Stays of Single Individuals in Canadian Cities of Different Sizes”  In _Housing Studies_ 28.6 Routledge, 2013, pp. 910–927  DOI: 10.1080/02673037.2013.773585
  * [2] Béla Bollobás  “Modern Graph Theory” 184, Graduate Texts in Mathematics  New York, NY: Springer, 1998  DOI: 10.1007/978-1-4612-0619-4
  * [3] Calgary Herald  “Calgary’s Homeless Shelters Prepare to Face Second Winter amid COVID-19 Pandemic”  In _calgaryherald_ , 2021  URL: https://calgaryherald.com/news/local-news/calgarys-homeless-shelters-prepare-to-face-second-winter-amid-covid-19-pandemic
  * [4] Calgary Homeless Foundation  “Administrative Count Findings: Calgary Data Report”, 2022  URL: https://www.calgaryhomeless.com/wp-content/uploads/2022/01/PiT-Admin-Data-Report%5C_Calgary%5C_Final.pdf
  * [5] Canadian Institute for Health Information  “Canadian COVID-19 Intervention Timeline — CIHI”, 2022  URL: https://www.cihi.ca/en/canadian-covid-19-intervention-timeline
  * [6] Julia Corey et al.  “A Scoping Review of the Health Impact of the COVID-19 Pandemic on Persons Experiencing Homelessness in North America and Europe”  In _International Journal of Environmental Research and Public Health_ 19.6 Multidisciplinary Digital Publishing Institute, 2022, pp. 3219  DOI: 10.3390/ijerph19063219
  * [7] Dennis P. Culhane et al.  “Public Shelter Admission Rates in Philadelphia and New York City: The Implications of Turnover for Sheltered Population Counts”  In _Housing Policy Debate_ 5.2 Routledge, 1994, pp. 107–140  DOI: 10.1080/10511482.1994.9521155
  * [8] Sybil Derrible and Christopher Kennedy  “Applications of Graph Theory and Network Science to Transit Network Design”  In _Transport Reviews_ 31.4 Routledge, 2011, pp. 495–519  DOI: 10.1080/01441647.2010.543709
  * [9] Ali Jadidzadeh and Ron Kneebone  “Homeless Shelter Flows in Calgary and the Potential Impact of COVID-19”  In _Canadian Public Policy_ 46.S2 University of Toronto Press, 2020, pp. S160–S165  DOI: 10.3138/cpp.2020-059
  * [10] Ronald D. Kneebone, Meaghan Bell, Nicole Jackson and Ali Jadidzadeh  “Who Are the Homeless? Numbers, Trends and Characteristics of Those Without Homes in Calgary”  In _The School of Public Policy Publications (SPPP)_ 8 Calgary, Canada: University of Calgary, The School of Public Policy, 2015  DOI: 10.11575/sppp.v8i0.42510
  * [11] Randall Kuhn and Dennis P. Culhane  “Applying Cluster Analysis to Test a Typology of Homelessness by Pattern of Shelter Utilization: Results from the Analysis of Administrative Data”  In _American Journal of Community Psychology_ 26.2, 1998, pp. 207–232  DOI: 10.1023/A:1022176402357
  * [12] Justine Levesque et al.  “COVID-19 Prevalence and Infection Control Measures at Homeless Shelters and Hostels in High-Income Countries: A Scoping Review”  In _Systematic Reviews_ 11.1, 2022, pp. 223  DOI: 10.1186/s13643-022-02089-x
  * [13] Abdul Majeed and Ibtisam Rauf  “Graph Theory: A Comprehensive Survey about Graph Theory Applications in Computer Science and Social Networks”  In _Inventions_ 5.1 Multidisciplinary Digital Publishing Institute, 2020, pp. 10  DOI: 10.3390/inventions5010010
  * [14] Geoffrey G. Messier  “A Simpler Method for Understanding Emergency Shelter Access Patterns”  arXiv, 2023  DOI: 10.48550/arXiv.2210.13619
  * [15] Abe Oudshoorn et al.  “A Rapid Review of Practices to Support People Experiencing Homelessness during COVID-19”  In _Housing, Care and Support_ 24.3/4 Emerald Publishing Limited, 2021, pp. 105–122  DOI: 10.1108/HCS-11-2020-0018
  * [16] Cameron Parsell, Andrew Clarke and Ella Kuskoff  “Understanding Responses to Homelessness during COVID-19: An Examination of Australia”  In _Housing Studies_ 38.1 Routledge, 2023, pp. 8–21  DOI: 10.1080/02673037.2020.1829564