# Term Expansion and FinBERT fine-tuning for Hypernym and Synonym Ranking of
Financial Terms

Ankush Chopra111Contact Author Equal Contribution Sohom Ghosh†Fidelity
Investments, AI CoE, Bengaluru, India
{ankush01729<EMAIL_ADDRESS>
###### Abstract

Hypernym and synonym matching are one of the mainstream Natural Language
Processing (NLP) tasks. In this paper, we present systems that attempt to
solve this problem. We designed these systems to participate in the FinSim-3,
a shared task of FinNLP workshop at IJCAI-2021. The shared task is focused on
solving this problem for the financial domain. We experimented with various
transformer based pre-trained embeddings by fine-tuning these for either
classification or phrase similarity tasks. We also augmented the provided
dataset with abbreviations derived from prospectus provided by the organizers
and definitions of the financial terms from DBpedia Auer et al. (2007),
Investopedia, and the Financial Industry Business Ontology (FIBO). Our best
performing system uses both FinBERT Araci (2019) and data augmentation from
the afore-mentioned sources. We observed that term expansion using data
augmentation in conjunction with semantic similarity is beneficial for this
task and could be useful for the other tasks that deal with short phrases. Our
best performing model (Accuracy: 0.917, Rank: 1.156) was developed by fine-
tuning SentenceBERT Reimers et al. (2019) (with FinBERT at the backend) over
an extended labelled set created using the hierarchy of labels present in
FIBO.

##  1 Introduction

Ontologies are rich sources of information that provide deep information about
the underlying concepts and entities. This information is described for a
specific domain. It contains the clearly defined relationships, and it is
organized in a defined structure mostly as a hierarchy. These properties make
ontologies a great source for getting a deeper understanding of the
relationship and properties of resources of the domain in consideration.

Public knowledge graphs and ontologies like DBpedia and Yago have been shown
to work on various applications like the ones described in Kobilarov et al.
(2009) and Hahm et al. (2014). This has motivated and paved ways for the
creation of domain focused ontologies like
FIBO222https://spec.edmcouncil.org/fibo/.

Effective techniques that enable identifying lexical similarity between the
terms or concepts increase the effectiveness of the ontologies. These methods
not only help in building new ontologies faster or augment the existing ones,
but also it helps in the effective querying and searching of concepts.

FinSim Maarouf et al. (2020); Mansar et al. (2021) competitions are being held
to promote the development of effective similarity measures. In the third
edition of the competition
FinSim-3333https://sites.google.com/nlg.csie.ntu.edu.tw/finnlp2021/shared-
task-finsim (accessed on 8th July 2021) (being held in conjunction with the
30th International Joint Conference on Artificial Intelligence (IJCAI-21)),
the participants are challenged to develop methods and systems to rank
hypernym and synonyms to financial terms by mapping them to one of the 17
high-level financial concepts present in FIBO.

In this paper, we present the systems developed by our team Lipi for hypernym
and synonym ranking. We experimented with basic featurization methods like TF-
IDF and advanced methods like pre-trained embedding models. Our top 3 systems
use pre-trained FinBERT Araci (2019) embedding model that was fine-tuned on
the data specific to financial domain . We also augmented the training data by
utilizing the knowledge from DBpedia, Investopedia, FIBO and text corpus of
prospectus shared with us. We describe the works related to our solution in
the next section. Section 3 contains the formal problem statement, followed by
data description in section 4. We describe our top three systems in section 5.
Section 6 contains the details of the experimentation that we performed and
the results obtained from some of them. We draw our conclusions in section 7
while giving a glimpse of things that we would like to try in the future.

##  2 Related Works

Hypernym-hyponym extraction and learning text similarity using semantic
representations have been very challenging areas of research for the NLP
community. SemEval-2018 Task 9 Camacho-Collados et al. (2018) was such an
instance. Team CRIM Bernier-Colborne and Barrière (2018) performed the best in
this shared task. They combined a supervised word embedding based approach
with an unsupervised pattern discovery based approach. The FinSim shared tasks
Maarouf et al. (2020); Mansar et al. (2021) deal with adopting these
challenges specific to the Financial Domain. Team IIT-K Keswani et al. (2020)
won FinSim-1 using a combination of context-free static embedding Word2Vec
Mikolov et al. (2013) and contextualized dynamic embedding BERT Devlin et al.
(2019). Anand et al. Anand et al. (2020) from the team FINSIM20 explored the
use of cosine similarity between terms and labels encoded using Universal
Sentence Encoder Cer et al. (2018). They also tried to extract hypernyms
automatically using graph based approaches. Team PolyU-CBS Chersoni and Huang
(2021) won FinSim-2 shared task using Logistic Regression trained over word
embedding and probabilities derived from BERT Devlin et al. (2019) model. They
also experimented with GPT-2 Radford et al. (2019). Team L3i-LBPAM Nguyen et
al. (2021) comprising Nguyen et al. performed better than the baseline by
using Sentence BERT Reimers et al. (2019) to calculate cosine similarity
between terms and hypernyms. Saini (2020); Pei and Zhang (2021) and Jurgens
and Pilehvar (2016) discussed various techniques to enrich the data which was
available for training. In this edition of FinSim, the number of training
samples and labels (financial concepts) were more than the previous two
editions.

##  3 Problem Statement

Given a set F consisting of n tuples of financial terms and their
hypernyms/top-level concepts/labels i.e.
$F=\\{(t\textsubscript{1},h\textsubscript{1}),(t\textsubscript{2},h\textsubscript{2}),...(t\textsubscript{n},h\textsubscript{n})\\}$
where hi represents the hypernym corresponding to the ith term ti and
$h\textsubscript{i}\epsilon$ set of labels mentioned in Table 1. For every
unseen financial term, our task is to generate a ranked list $\hat{y_{i}}$
consisting of these 17 hypernyms in order of decreasing semantic similarity.

Evaluation Metrics The expected output is a raked list of predicted labels for
every scored instance. The proposed systems are evaluated based on Accuracy
and Mean Rank metrices as per the shared task rules. Evaluation script was
provided by organizers, where accuracy and mean rank were defined as:
$Accuracy=\frac{1}{n}\sum_{i=1}^{n}I(y_{i}=\hat{y_{i}}[1])$
$MeanRank=\frac{1}{n}\sum_{i=1}^{n}(\hat{y_{i}}.index(y_{i}))$
where $\hat{y_{i}}$ is the ranked list (with index starting from 1) of
predicted labels corresponding to the expected label $y_{i}$. I is an identity
matrix.

##  4 Data

###  4.1 Data Description

The training dataset shared for this task has a total of 1050 single and
multi-word terms tagged to 17 different classes/labels out of which 1040 term-
label pairs are unique. More than 91% of the terms have 6 words or less and
the longest term has 22 words. There were 10 duplicate entries, and 3 terms
were assigned 2 different labels. Along with this, a corpus of prospectuses in
English that had 211 documents was provided. Some of the terms mentioned in
the training data were present in the corpus. Table 1 shows the distribution
of these labels in the training set.

Label | Count
---|---
Equity Index | 280
Regulatory Agency | 205
Credit Index | 125
Central Securities Depository | 107
Debt pricing and yields | 58
Bonds | 55
Swap | 36
Stock Corporation | 25
Option | 24
Funds | 22
Future | 19
Credit Events | 18
MMIs | 17
Stocks | 17
Parametric schedules | 15
Forward | 9
Securities restrictions | 8
Total | 1040
Table 1: Label distribution in the training set

###  4.2 Data Augmentation

Since the majority of the terms had only a few tokens, we decided to expand
the terms wherever possible using various sources. This approach had also been
adopted by Saini (2020) and Pei and Zhang (2021) while participating in
FinSim-1 and FinSim-2 respectively.

Acronym expansion: As mentioned by Keswani et al. Keswani et al. (2020), the
presence of acronyms created a major issue in maintaining consistency. We used
the abbreviation extractor available in spaCy444https://spacy.io/Honnibal et
al. (2020) package on the corpus of the prospectus to extract all the acronyms
and their expansions. Upon manual inspection of a sample output, we identified
that not all the extracted items were valid acronyms and their expansions. We
cleaned the extracted list by dropping the records where:

  * •

expansion had equal or less length than the acronym.

  * •

expansion had parenthesis

  * •

extracted acronym was a valid English word such as ”fund” or ”Germany”.

  * •

the expansion had less than or equal to 5 characters.

We managed to extract 635 acronyms from the prospectus corpus after applying
the above exclusions. We used this data to expand the matching terms in the
given train set and test sets.

Definitions from DBpedia: We used the DBpedia search
API555https://lookup.dbpedia.org/api/search to extract the description of the
terms present in the train and test sets. We present such an example in Figure
1. In addition to the description, the label was also retained from the result
payload to identify the right description for the input terms. We tried token
overlap-based similarity of input terms with both matching labels and
descriptions. We decided to use the label to term match for description
matching after going through a randomly drawn sample. We cleaned both input
terms and labels from DBpedia results by converting them to lower case,
replacing punctuations by space, removing repetitive spaces, and singularizing
the text. We calculated the token overlap ratios for cleaned term and DBpedia
labels using these formulas: $Ratio1=length(s_{1}\cap s_{2})/length(s_{1})$,
$Ratio2=length(s_{2})/length(s_{1})$ where s1 and s2 represents sets of
tokenized cleaned terms and tokenized cleaned DBpedia labels respectively. We
empirically decided to use all the instances with $Ratio1=1$ and
$Ratio2<=1.25$ for matching a DBpedia label (and hence description) to the
input term.

Figure 1: Sample output from DBpedia search API

Definitions from Investopedia and FIBO: Inspired by Saini (2020), we obtained
definitions of the terms present in Investopedia’s data
dictionary666https://www.investopedia.com/financial-term-dictionary-4769738 by
crawling it. We downloaded a glossary of financial terms from the website of
FIBO. We cleaned all the terms from the train and test set and also the terms
present in Investopedia’s data dictionary using the steps described in the
above DBpedia section. We then assigned the Investopedia or FIBO definition to
the terms from the train and test sets where cleaned terms from train and test
data matched to cleaned Investopedia terms perfectly.

The test set which was provided to us had 326 terms. We augmented the original
train and test set with the records where we could either find definition or
expansion using the above sources. The train set size increased to 1836
records and the test set size increased to 607 after the data augmentation. We
present an example of data augmentation for the term “callable bond” in Table
2. Table 3 states the number of instances we used from each of the sources to
augment the data we had.

Expanded Term/Term Definition | Label | Source
---|---|---
Callable bond | Bonds |  | original and
---
acronym expansion
| bond that includes a stipulation allowing the issuer
---
the right to repurchase and retire the bond at the call price after the call
protection period
Bonds | FIBO
| A callable bond (also called redeemable bond) is a type of bond (debt
security) that allows
---
the issuer of the bond to retain the privilege of redeeming the bond at some
point before
the bond reaches its date of maturity.
Bonds | DBpedia
Table 2: Result of Data Augmentation of the term ”Callable bond” Data Source | Count
---|---
Original modelling data | 1040
DBpedia | 257
FIBO | 236
Investopedia | 85
Acronym expansion | 218
Table 3: Details of various data sources

##  5 System Description

We tried to solve this problem as the term classification and term similarity
problems. Two of our 3 submissions are modelled as the term classification
problem, whereas the third system is designed to be a phrase/sentence
similarity problem between terms (or expanded terms from the augmented
dataset) and the definitions of 17 class labels that were extracted from FIBO
/ Internet. All the systems rely on semantic similarity and use FinBERT model
to generate the term or token embedding representations. We divided the given
data into training and validation sets having 832 and 208 terms respectively.

###  5.1 System - 1 (S1)

This is the simplest of our proposed systems, where we did not use the
augmented dataset and used only the original set that was shared by
organizers. We loaded FinBERT pre-trained model and fine-tuned it by trying to
classify the representation of [CLS] token into one of the 17 labels mentioned
previously. Since the original data did not have longer terms, we kept the
maximum length to 32, and train and validation batch sizes of 64. We used Adam
optimizer with a learning rate of 0.00002. We ran the model for 40 epochs and
picked the model saved after 18th epoch based on the performance on the
validation set. Finally, we ranked the predictions based on the predicted
probability of each class.

###  5.2 System - 2 (S2)

This system is similar to System-1 with the only difference that data being
the augmented set and not the original dataset. Since the augmented dataset
had the descriptions of the terms, the inputs were considerably longer. Hence,
we increased the maximum length to 256 while keeping all the other hyper-
parameters the same. After, training the model for 40 epochs we selected the
model saved after the 17th epoch as the best model based on validation set
performance.

###  5.3 System -3 (S3)

We explored the FIBO ontology to understand the hierarchy Stepišnik Perdih et
al. (2021) of the 17 labels as depicted in Figure 2. We used the augmented
data described in section 4.2 to create a labelled dataset having similarity
scores. For every term definition (T) to label definition (L) mapping which
existed in the extended training set, we assigned a similarity score of 1.0 to
the (T,L) pair and picked up 10 training instances randomly ensuring none of
their label definition was same as L. For each of the label definitions (LL)
present in this sample, we extracted its root node and first child node. We
did the same for the original label definition (L). Then, we compared these
nodes. If the root node and first child node of L were different from that of
LL then we assigned a similarity score of 0 to the (T, LL) pair. If the root
nodes were the same, we assigned a similarity score of ’k’ when the first
child nodes differed and a similarity score of ’2k’ when they were the same
(where $0<k<1$). We empirically figured out that k=0.4 works the best. As
expected, the number of instances with a similarity score equal to 0 increased
substantially. We under-sampled such instances and the new training set had
30% instances with similarity score 1.0, 12% instances with similarity score
’k’, 28% instances with similarity score ’2k’ and 30% instances with
similarity score 0. After that, we fine-tuned a FinBERT Araci (2019) model
using Sentence BERT Reimers et al. (2019) framework with this newly generated
labelled data for 25 epochs with a batch size of 20. Our objective was to
minimize the multiple negatives ranking loss and online contrastive loss. We
used a margin of 0.5 and cosine distance as a distance metric while training
this model. Finally, we converted all of the 17 labels’ definitions and term
definitions from the validation set to vectors using this fine-tuned model.
For every such term definition, we performed a semantic search over the label
vectors and ranked them in decreasing order of cosine similarity.
System 2 and 3 take advantage of term expansion during both model training and
scoring phases, which causes certain observations to appear more than once
(reference: Table 3). We derive the final prediction by averaging the output
probabilities for all the 17 classes for all the occurrences of the term.

Figure 2: Label Hierarchy from FIBO. Bold (leaf nodes) denotes the labels.

##  6 Experimentation and Results

We had 1040 observations after removing the duplicates. We did an 80:20 split
to create a training and validation set from this. We augmented the given
modelling set by incorporating definitions from DBpedia, FIBO and
Investopedia. We used the list of acronyms extracted from the prospectus
corpus to create a copy with acronym expansion. This helped us to increase the
original data to 1836 records (mentioned in Table 1). It should be noted that
we could not find the expansions for all the terms given in the modelling set.
Train and validation set sizes for the original modelling set and expanded
data were (832 & 208) and (1470 & 366) respectively.

We established a baseline by running the scripts provided by the organizers.
Then, we considered original modelling data and fine-tuned base BERT-cased
model Devlin et al. (2019) to predict the class label by taking the
representation of [CLS] token while passing it through few layers of a feed-
forward network. This performed better than baseline. We then tried the same
BERT-base model on the expanded dataset, which gave us further performance
improvement. Since the only major change between these runs was the data, the
improvement can be attributed to the expanded data.

We experimented with a few of the other pre-trained models that are available
on the Huggingface model repository Wolf et al. (2020). We observed clear
improvement when we used the FinBERT model which was trained on data specific
to the financial domain. The model performance successively increased when we
used a combination of data expansion with FinBERT. Furthermore, we tried to
fine-tune FinBERT using Sentence Transformers Reimers et al. (2019) to capture
semantic textual similarity. For this, we used several combinations of term
and term definitions with label and label definitions.

All the hyperparameters for the final 3 models have been already mentioned in
the system description. After rigorous experimentation, these hyperparameters
were selected empirically based on validation set performance. The results are
presented in Table 4. Since the number of submissions was restricted to 3 for
each team, we do not have the performance numbers of the BERT models in the
test set. Analysing the results we see that SentenceBERT trained with FinBERT
at the backed as mentioned in section-5.3 performed the best.

|  | Validation set | Test set
---|---|---|---
Model | Data | Rank | Acc. | Rank | Acc.
Base-1 | Org. | 2.158 | 0.498 | 1.941 | 0.564
Base-2 | Org. | 1.201 | 0.876 | 1.75 | 0.669
BERT | Org. | 1.177 | 0.899 | - | -
BERT | Ext. | 1.153 | 0.928 | - | -
FinBERT(S1) | Org. | 1.117 | 0.928 | 1.257 | 0.886
FinBERT(S2) | Ext. | 1.110 | 0.942 | 1.220 | 0.895
SBERT(S3) | Ext. | 1.086 | 0.947 | 1.156 | 0.917
Table 4: Results on validation and test set. Org. represents original and Ext.
represents extended. Base refers to baseline.

##  7 Conclusion and Future Works

In this work, we attempted to solve the hypernym and synonym discovery hosted
at FinSim-3. This challenge aimed to enable the better use of ontologies like
FIBO using hypernyms and synonyms, and we used these ontologies themselves to
develop our systems which perform significantly better than the provided
baseline systems. This proves the present use of these ontologies. The
presented solution is recursive in a sense as it uses knowledge from
ontologies to further increase the effectiveness and use of the same. Apart
from data augmentation, our solution relies upon semantic similarity learnt
from pre-trained embedding models that were learnt on the relevant domain. We
observed the clear benefits of domain specific pretraining during the
experimentation.

In future, we would like to explore Knowledge Graphs (as described in Portisch
et al. (2021)) to further improve the improve performance of the models. We
also want to explore other variants of FinBERT Araci (2019) and fine-tune them
using the Masked Language Modeling technique (as mentioned by the winner of
FinSim-2 Chersoni and Huang (2021)) and Next Sentence Prediction objective.
Moreover, this research can be extended by extracting sentences present in the
prospectus (similar to Goel et al. (2021)) to create more positive and
negative samples.

## References

  * Anand et al. [2020] Vivek Anand, Yash Agrawal, Aarti Pol, and Vasudeva Varma.  FINSIM20 at the FinSim task: Making sense of text in financial domain.  In Proceedings of the Second Workshop on Financial Technology and Natural Language Processing, pages 104–107, Kyoto, Japan, 5 January 2020\.
  * Araci [2019] Dogu Araci.  Finbert: Financial sentiment analysis with pre-trained language models, 2019.
  * Auer et al. [2007] Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives.  Dbpedia: A nucleus for a web of open data, 2007.
  * Bernier-Colborne and Barrière [2018] Gabriel Bernier-Colborne and Caroline Barrière.  CRIM at SemEval-2018 task 9: A hybrid approach to hypernym discovery.  In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 725–731, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
  * Camacho-Collados et al. [2018] Jose Camacho-Collados, Claudio Delli Bovi, Luis Espinosa-Anke, Sergio Oramas, Tommaso Pasini, Enrico Santus, Vered Shwartz, Roberto Navigli, and Horacio Saggion.  SemEval-2018 task 9: Hypernym discovery.  In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 712–724, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
  * Cer et al. [2018] Daniel Cer, Yinfei Yang, Sheng yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil.  Universal sentence encoder, 2018.
  * Chersoni and Huang [2021] Emmanuele Chersoni and Chu-Ren Huang.  PolyU-CBS at the FinSim-2 Task: Combining Distributional, String-Based and Transformers-Based Features for Hypernymy Detection in the Financial Domain, page 316–319.  Association for Computing Machinery, New York, NY, USA, 2021.
  * Devlin et al. [2019] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
  * Goel et al. [2021] Tushar Goel, Vipul Chauhan, Ishan Verma, Tirthankar Dasgupta, and Lipika Dey.  TCS WITM 2021 @FinSim-2: Transformer Based Models for Automatic Classification of Financial Terms, page 311–315.  Association for Computing Machinery, New York, NY, USA, 2021.
  * Hahm et al. [2014] Younggyun Hahm, Jungyeul Park, Kyungtae Lim, Youngsik Kim, Dosam Hwang, and Key-Sun Choi.  Named entity corpus construction using wikipedia and dbpedia ontology.  In LREC, pages 2565–2569, 2014.
  * Honnibal et al. [2020] Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd.  spaCy: Industrial-strength Natural Language Processing in Python, 2020\.
  * Jurgens and Pilehvar [2016] David Jurgens and Mohammad Taher Pilehvar.  SemEval-2016 task 14: Semantic taxonomy enrichment.  In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 1092–1102, San Diego, California, June 2016\. Association for Computational Linguistics.
  * Keswani et al. [2020] Vishal Keswani, Sakshi Singh, and Ashutosh Modi.  IITK at the FinSim task: Hypernym detection in financial domain via context-free and contextualized word embeddings.  In Proceedings of the Second Workshop on Financial Technology and Natural Language Processing, pages 87–92, Kyoto, Japan, 5 January 2020.
  * Kobilarov et al. [2009] Georgi Kobilarov, Tom Scott, Yves Raimond, Silver Oliver, Chris Sizemore, Michael Smethurst, Christian Bizer, and Robert Lee.  Media meets semantic web – how the bbc uses dbpedia and linked data to make connections.  In Lora Aroyo, Paolo Traverso, Fabio Ciravegna, Philipp Cimiano, Tom Heath, Eero Hyvönen, Riichiro Mizoguchi, Eyal Oren, Marta Sabou, and Elena Simperl, editors, The Semantic Web: Research and Applications, pages 723–737, Berlin, Heidelberg, 2009. Springer Berlin Heidelberg.
  * Maarouf et al. [2020] Ismail El Maarouf, Youness Mansar, Virginie Mouilleron, and Dialekti Valsamou-Stanislawski.  The FinSim 2020 shared task: Learning semantic representations for the financial domain.  In Proceedings of the Second Workshop on Financial Technology and Natural Language Processing, pages 81–86, Kyoto, Japan, 5 January 2020.
  * Mansar et al. [2021] Youness Mansar, Juyeon Kang, and Ismail El Maarouf.  The FinSim-2 2021 Shared Task: Learning Semantic Similarities for the Financial Domain, page 288–292.  Association for Computing Machinery, New York, NY, USA, 2021.
  * Mikolov et al. [2013] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean.  Efficient estimation of word representations in vector space, 2013.
  * Nguyen et al. [2021] Nhu Khoa Nguyen, Emanuela Boros, Gael Lejeune, Antoine Doucet, and Thierry Delahaut.  L3i LBPAM at the FinSim-2 Task: Learning Financial Semantic Similarities with Siamese Transformers, page 302–306.  Association for Computing Machinery, New York, NY, USA, 2021.
  * Pei and Zhang [2021] Yulong Pei and Qian Zhang.  Goat at the finsim-2 task: Learning word representations of financial data with customized corpus.  In Companion Proceedings of the Web Conference 2021, WWW ’21, page 307–310, New York, NY, USA, 2021. Association for Computing Machinery.
  * Portisch et al. [2021] Jan Portisch, Michael Hladik, and Heiko Paulheim.  FinMatcher at FinSim-2: Hypernym Detection in the Financial Services Domain Using Knowledge Graphs, page 293–297.  Association for Computing Machinery, New York, NY, USA, 2021.
  * Radford et al. [2019] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever.  Language models are unsupervised multitask learners, 2019.
  * Reimers et al. [2019] Nils Reimers, Iryna Gurevych, Nils Reimers, Iryna Gurevych, Nandan Thakur, Nils Reimers, Johannes Daxenberger, and Iryna Gurevych.  Sentence-bert: Sentence embeddings using siamese bert-networks.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2019\.
  * Saini [2020] Anuj Saini.  Anuj at the FinSim task: Anuj@FINSIM¡VLearning semantic representation of financial domain with investopedia.  In Proceedings of the Second Workshop on Financial Technology and Natural Language Processing, pages 93–97, Kyoto, Japan, 5 January 2020.
  * Stepišnik Perdih et al. [2021] Timen Stepišnik Perdih, Senja Pollak, and Blaž Škrlj.  JSI at the FinSim-2 Task: Ontology-Augmented Financial Concept Classification, page 298–301.  Association for Computing Machinery, New York, NY, USA, 2021.
  * Wolf et al. [2020] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.  Huggingface’s transformers: State-of-the-art natural language processing, 2020.


# Random sequential adsorption of aligned regular polygons and rounded
squares: Transition in the kinetics of packing growth

Michał Cieśla<EMAIL_ADDRESS>Piotr Kubala
<EMAIL_ADDRESS>Institute of Theoretical Physics, Jagiellonian
University, 30-348 Kraków, Łojasiewicza 11, Poland  Aref Abbasi Moud
<EMAIL_ADDRESS>Department of Chemical and Biological Engineering, The
University of British Columbia, Vancouver, British Columbia V6T 1Z3, Canada

###### Abstract

We study  two-dimensional random sequential adsorption  (RSA) of  flat
polygons and rounded squares  aligned in parallel to find a transition in the
asymptotic behavior of the kinetics of packing growth. Differences in the
kinetics for RSA of disks and parallel squares were confirmed in previous
analytical and numerical reports. Here, by analyzing the two classes of shapes
in question we  can precisely control the shape of packed figures and thus
localize the transition. Additionally, we study how the asymptotic properties
of the kinetics depend on the packing size. We also provide accurate
estimations of saturated packing fractions. The microstructural properties of
generated packings are analyzed in terms of the density autocorrelation
function.

##  I Introduction

Random sequential adsorption (RSA) is a numerical protocol used for generating
random packings [1, 2, 3]. According to it, the shapes are placed randomly one
after another, however, the placing occurs only if the next shape does not
overlap any of the previously added shapes. After placing, the position and
orientation of each figure  remain unchanged. The procedure continues until
the packing is saturated – there is no place for any other shape. In contrast
to the so-called random close packings (RCP) where the neighboring particles
typically are in contact [4], here the packing is rather loose and the mean
packing fraction is significantly smaller.

Although the history of RSA begins in 1939 when Flory used  the random process
described above to study the structure of a linear polymer to which some
groups of molecules can be attached at random places [5], the real interest in
RSA began in 1980 when Feder noticed that the structure of such two-
dimensional random packings resembles monolayers produced in irreversible
adsorption experiments [6]. The similarities were so  substantial that
saturated packing fractions of disks on a flat surface were determined using
adsorption experiments [7]. On the other hand, the numerical generation of
large, strictly saturated packings was ineffective because when the packing is
almost saturated, the probability that a randomly placed and oriented object
will not intersect with any previously added one is tiny. Thus, the number of
such attempts has to be very large to place the next figure. Although for some
specific shapes, there exist methods overcoming this problem e.g. [8, 9, 10],
the properties of saturated state are  still often estimated using the
kinetics of packing growth  computed for almost saturated packings. For a
majority of shapes, the asymptotic kinetics is given by Feder’s law

$\theta-\theta(t)\sim t^{-\frac{1}{d}},$ (1)

when the packing is close to a saturated state. Here, $\theta$ is the
saturated packing fraction, and $\theta(t)$ is the packing fraction after $t$
tries of adding a shape to the packing. Parameter $d$ depends on shape and
packing  dimensionality. For example, for  $k$-dimensional (hyper)spheres
packed in the (hyper)space of the same dimensionality, $d=k$, while for
anisotropic, randomly oriented  two-dimensional shapes placed on the  two-
dimensional flat surface $d=3$ [11, 12, 9]. On the other hand, for parallel
squares or rectangles

$\theta-\theta(t)\sim\frac{\log t}{t}.$ (2)

Both these relations were confirmed analytically [13, 14] and numerically e.g.
[11, 15, 12, 16, 9]

Here, we want to study the transition between these two regimes. We tried to
achieve this in two ways. The first one is to generate  two-dimensional random
packings composed of  flat regular polygons aligned in parallel. For  the RSA
of squares, as noted above, the kinetics of packing growth is governed by (2).
When the number of regular polygon sides grows, its shape approaches the disk
for which kinetics is given by (1) with parameter $d=2$. A similar study was
recently presented in [17], but the author focused on saturated packing
fractions while the presented results on the RSA kinetics of  squares did not
agree with the analytical law (2). The second way is to generate packings
built of aligned squares with rounded corners. By increasing the radius of
this rounding the shape approaches the disk, thus the transition in the
packing’s growth kinetics should be visible. The second method seems to be
superior to the first one because the radius can be changed continuously,
while the number of regular polygon’s sides is a discrete value and the disk
is approached only in the limit of  an infinite number of sides.

##  II Numerical details

RSA protocol consists of iterations of the following steps:

  * •

select the position of a virtual polygon randomly with the probability
uniformly distributed over the packing;

  * •

check if the virtual particle does not overlap with any polygon inside the
packing;

  * •

if it does not, add it to the packing, otherwise, remove and abandon it.

To generate strictly saturated  packings according to RSA protocol we traced
the regions where subsequent particles can be added. This idea was used for
the first time by Akeda et al. in the case of packings built of parallel
squares [18] and by Wang for disks [8]. The method is based on the division of
the packing into small regions called voxels, and each voxel is tested if
there is  a possibility to place there  the center of the next shape without
overlapping existing polygons. If not, such a voxel is removed  from the list
of existing voxels. Thus, the random sampling of the position of the virtual
shape is limited only to  the voxels that are on the list, which speeds up the
packing generation. The voxels can be divided into smaller ones to better
estimate the region when placing is possible. The simulation ends where there
are no voxels left, thus, the packing is saturated. A variant of this method
for polygons was invented by Zhang [19] and improved further in [20] and the
details about the voxel removal criterion can be found there. Although in its
original version, this method was designed for the generation of saturated
packings built of arbitrarily oriented polygons, its restriction to a single
orientation is straightforward.

It should be mentioned that when the sampling of the virtual shape position
covers only existing voxels occupying a fraction of the whole packing surface
area $S$, one iteration corresponds to $S/S_{v}$ iterations in the original
RSA protocol, where $S_{v}$ is the total surface area of these voxels.
Additionally, to compare the results obtained for different sizes of packings,
the number of iterations is expressed in the so-called dimensionless time
units where one unit contains $S/S_{p}$ iterations. Here, $S_{p}$ is the
surface area of a single polygon. Throughout this study, the number of
iterations shall be expressed in these units and denoted as $t$.

We studied saturated RSA packings built of regular polygons of the number of
sides ranging from $3$ to $1000$. To estimate the kinetics and other
properties, we generated $100$ independent random packings for each type of
polygon. The figures were placed on the square of the surface area $S=10^{6}$,
while the surface area of a single polygon was normalized to $S_{p}=1$. To
minimize finite-size effects, periodic boundary conditions were used [10]. The
number of iterations needed to form saturated packing differs significantly
between independent packings, as  it in general  is distributed according to a
heavy tail probability distribution function [21]. Therefore, to estimate the
asymptotic value of the parameter $d$ in the power law (1) we restricted to
the data from the range $[t_{\text{min}}/100,t_{\text{min}}]$, where
$t_{\text{min}}$ is the dimensionless time when the first packing becomes
saturated. Such an approach guarantees sufficient statistics and also analyzes
kinetics close to saturation.  It is worth noting that
$\log[\theta(2t)-\theta(t)]$ exhibits the same asymptotic scaling as (1) when
plotted against $\log(t)$ [22, 23], which gives another way to determine the
exponent $d$.

Similarly we studied RSA packings of rounded squares. The shape is
parameterized by one additional parameter $r$ which corresponds to the circle
radius at each corner of the square – see Fig. 1.

Figure 1: Parametrization of a rounded square. The circumscribed circle of the
square has a unit radius and circumscribed circle of the rounded square has a
radius  of $1+r$.

Parameter $r$ can vary from $0$ (square) to infinity (disk), but here it was
restricted to $r\in[0,1]$. The surface area of the shape in Fig. 1 is
$S_{r}=2+4r\sqrt{2}+\pi r^{2}$, and the linear size of the rounded square was
always rescaled to obtain $S_{r}=1$. This  parameterization was used in Ref.
[24], where one can also find a detailed description of the method for
generating saturated RSA packing built of rounded polygons.

##  III Results

Example saturated packings built of polygons aligned in parallel are shown in
Fig. 2.

Figure 2: Example saturated packings built of equilateral triangles, squares,
pentagons, and rounded squares for $r=0.2$ aligned in parallel. The packing
size is $S=400$ and the periodic boundary conditions are used.

Note that the square with rounded corners characterized by $r=0.2$ is visually
indistinguishable from a  normal square.

###  III.1 Kinetics of packing growth

The kinetics is presented in Figs. 3 and 4. Fig. 3 presents the data according
to (2).

Figure 3: The dependence of the packing fraction near saturation on $\ln(t)/t$
for packing built of oriented regular polygons of a different number of sides.
Straight lines correspond to the kinetics governed by (2).

The only case where a straight line is observed corresponds to packings built
of squares ($n=4$). In analogy, we can analyze the data according to (1). The
dependence of the fitted value of the parameter $d$ from (1) on the number of
regular polygon sides $n$ is presented in Fig. 4.

Figure 4: The dependence of the fitted value of the parameter $d$ from (1) on
the number of sides $n$ of the polygon. The dashed horizontal line corresponds
to $d=2$ and the dashed vertical line denotes $n=4$. Inset shows the kinetics
of packings growth for several different regular polygons.

We observe that only in the case of packings built of squares the parameter
$d$ describing the kinetics of packing growth significantly differs from 2.
For all other regular polygons the kinetics seems to be governed by the power
law with $d=2$ – the same as for spheres, which is consistent with the
argument that $d$ corresponds to the number of  particles’ degrees of freedom
[25, 26]. Here, for all shapes, even for squares, there are only two degrees
of freedom corresponding to the position of the center of a two-dimensional
shape. For squares, as derived by Swendsen, the kinetics does not follow the
power-law (1) but the one described by (2) [14]. However, for time scales
$t<t_{\text{min}}$ appearing in our study, it is hard to distinguish between
$\log t/t$ and $t^{-\alpha}$ with $\alpha$ slightly smaller than $1$ ($d$
slightly larger than $1$). Thus,  although the RSA kinetics for parallel
squares is governed by (2) – see Fig. 3, we can successfully fit the power law
to it – see Fig. 4, with the fitted value of parameter $d$ slightly larger
than $1$.

The above results show the uniqueness of the square shape. This is the only
regular polygon that leads to dissimilar kinetics of RSA packing growth. As
noted before, to study this phenomenon carefully we also analyzed packings
built of squares with rounded corners. The results are presented in Fig. 5.

Figure 5: The dependence of  the fitted value of the parameter $d$ from (1) on
the parameter $r$ describing rounded squares. The dashed line corresponds to
$d=2$.

Here, we observe the transition of $d$ for $r$ between $r=0.02$ and $r=0.12$
from $d=1.2$ to $d=2.0$, respectively. Note, that it is very hard to visually
distinguish a square from a rounded square even with not insignificant
rounding $r=0.2$ – see Fig. 2. It implies that even tiny changes in shape can
significantly influence the RSA kinetics. Similar effects were studied
analytically by Baule for two-dimensional shapes placed on a one-dimensional
line [22] and then supported by numerical simulations [27]. There, the
kinetics depended on the analytical nature of the contact function, which is
defined as the separation distance at which two particles are in contact.
Interestingly, fine details of the contact function are revealed in numerical
simulations only for some percentage of packings, and the value of this
percentage depends on the packing size [27]. Therefore, the sharpness and
place of the observed transition depend on the packing size. Regardless, it
does not explain the uniqueness of the square shape in comparison with other
regular polygons.

The last thing related to the kinetics of packing growth that we want to
explain is the recent result obtained in [17], where the kinetics of RSA of
squares is described by the power-law (1) with $d\approx 2$. The packing sizes
under consideration in the study mentioned are significantly smaller than the
ones we used. Additionally, the author worked with non-saturated
configurations. As those two differences may be the source of the discrepancy,
we analyzed how the exponent in a power law fit depends on the packing size
and the dimensionless time at which we calculate it. The results are presented
in Fig. 6.

Figure 6: The dependence of the parameter $d$ from (1) on the packing size
$S$. Inset shows the dependence of the parameter $d$ on the dimensionless time
$t$ at which the packing generation was stopped for packings of size
$S=10^{7}$.

The plots clearly  show that the fitted value of the parameter $d$ is larger
for both, small and non-saturated packings, which explains the results from
the former manuscript. Interestingly,  although the saturated packing
fractions can be determined quite accurately using relatively small packings
[10], the study of the kinetics of packing growth requires a few orders of
magnitude larger packing sizes. This observation agrees with the results for
the kinetics of packing growth for several different figures placed on a one-
dimensional line [27]. It is however important to recall that the value of $d$
for squares will always depend on $t$, regardless of how large it is, because
the true asymptotic  behavior is not described by the power law (1), but (2).

Having obtained saturated packing of squares of different sizes we are able to
use another way to determine the kinetics of packing growth. It was
analytically shown that for disks the median of dimensionless time at which
the last shape is added to the packing $M[t_{\text{sat}}]$ scales with a
packing size $S$ as

$M[t_{\text{sat}}]\sim S^{d},$ (3)

where $d$ is the same parameter as in (1) [21]. It seems that similar relation
is also valid for a packing built of oriented squares – see Fig. 7.

Figure 7: The dependence of the median of saturation time $t_{\text{sat}}$ on
the packing size $S$. Dots are the data determined numerically using $100$
independently generated packing and solid line is a fit
$M[t_{\text{sat}}]=1.4351\cdot S^{1.147}$.

Moreover, the  fitted value of the exponent  $1.147\pm 0.016$ is close to the
parameter $d$ determined from (1) for $S=10^{7}$, but here the value is size
independent.

In the next sections, we study other basic characteristics of random packings
to see if they also reflect the variability of the kinetics of packing growth.

###  III.2 Mean saturated packing fraction

The mean density of saturated packing is a basic property  of interest. Here,
because we generated strictly saturated configurations, we only need to
average  the obtained  densities without any extrapolation. Because the
surface area of a single  figure is always normalized to $1$ the density
equals the number of deposited shapes divided by  the packing area $S=10^{6}$.
The results obtained are shown in Fig. 8.

Figure 8: The dependence of  the packing fraction on the number of regular
polygon sides. The dashed line corresponds to  $\theta=0.547$ approximating
the RSA packing fraction of disks [3, 6, 18]. The inset shows the dependence
of the packing fraction of the rounded square on parameter $r$. The dashed
lines highlight two limits:  $\theta=0.547$ for the RSA of disks and
$\theta=0.562$ for the RSA of squares. In both plots, dots are the values
determined from generated packings. The error bars are smaller than the dot
size and thus they are omitted. The thin solid lines connecting dots are to
guide the eye.

For regular polygons, we see oscillations of packing fractions for even and
odd numbers of polygon sides. This effect was already observed in previous
studies [19, 17]. The values  presented here are, in general, in agreement
with these results – note that in Ref. [19] RSA of unoriented polygons was
studied. The packing fraction for rounded squares shows the transition between
two limits – the upper one for aligned squares and the lower one for disks.
However, this transition occurs  for larger r – it starts at $r=0.1$ and
approaches the packing fraction of disks near $r=1$, while the kinetics for
rounded squares is indistinguishable from the one for disks at $r\approx 0.1$.
It  shows that the behavior of packing fractions  weakly correlated with the
kinetics of packings  for the systems in question. For convenience, the
presented data has been collected in Tab. 1.

$n$ | $\theta$ | $d$
---|---|---
$3$ |  $0.366410\pm 0.000016$ |  $1.926\pm 0.025$
$4$ | $0.5620219\pm 0.0000072$ | $1.100\pm 0.010$
$5$ | $0.489682\pm 0.000016$ | $2.017\pm 0.035$
$6$ | $0.549713\pm 0.000016$ | $1.957\pm 0.035$
$7$ | $0.518584\pm 0.000019$ | $2.025\pm 0.032$
$8$ | $0.547189\pm 0.000017$ | $2.003\pm 0.025$
$9$ | $0.530072\pm 0.000018$ | $1.980\pm 0.026$
$10$ | $0.547463\pm 0.000019$ | $1.960\pm 0.033$
$14$ | $0.547073\pm 0.000018$ | $1.987\pm 0.031$
$20$ | $0.547049\pm 0.000018$ | $1.960\pm 0.038$
$35$ | $0.545938\pm 0.000018$ | $2.013\pm 0.027$
$50$ | $0.547039\pm 0.000018$ | $2.000\pm 0.027$
$70$ | $0.547037\pm 0.000018$ | $1.965\pm 0.028$
$140$ | $0.547037\pm 0.000018$ | $2.028\pm 0.036$
$200$ | $0.547037\pm 0.000018$ | $1.992\pm 0.031$
$350$ | $0.547035\pm 0.000018$ | $2.039\pm 0.030$
$500$ | $0.547035\pm 0.000018$ | $1.988\pm 0.026$
$700$ | $0.547036\pm 0.000018$ | $1.993\pm 0.032$
$1000$ | $0.547035\pm 0.000018$ | $1.931\pm 0.036$
$r$ | $\theta$ | $d$
$0.0001$ | $0.562032\pm 0.000021$ | $1.1740\pm 0.0041$
$0.001$ | $0.562031\pm 0.000022$ | $1.1803\pm 0.0040$
$0.01$ | $0.562026\pm 0.000021$ | $1.1861\pm 0.0063$
$0.02$ | $0.562007\pm 0.000021$ | $1.2618\pm 0.0070$
$0.03$ | $0.561985\pm 0.000022$ | $1.441\pm 0.016$
$0.05$ | $0.561909\pm 0.000021$ | $1.548\pm 0.015$
$0.07$ | $0.561803\pm 0.000022$ | $1.699\pm 0.021$
$0.1$ | $0.561600\pm 0.000021$ | $1.750\pm 0.018$
$0.12$ | $0.561448\pm 0.000022$ | $1.867\pm 0.051$
$0.15$ | $0.561184\pm 0.000022$ | $1.973\pm 0.046$
$0.2$ | $0.560686\pm 0.000024$ | $1.987\pm 0.054$
$0.3$ | $0.559581\pm 0.000025$ | $1.964\pm 0.055$
$0.5$ | $0.557365\pm 0.000023$ | $2.044\pm 0.059$
$0.7$ | $0.555407\pm 0.000021$ | $1.838\pm 0.045$
$1.0$ | $0.553160\pm 0.000019$ | $2.024\pm 0.024$
$2.0$ | $0.549539\pm 0.000018$ | $2.015\pm 0.066$
$5.0$ | $0.547549\pm 0.000016$ | $1.989\pm 0.050$
$10.0$ | $0.547177\pm 0.000017$ | $2.016\pm 0.042$
Table 1: Mean saturated packing fractions obtained from computer simulations.
The error of packing fraction $\theta$ is the standard deviation of the mean
value. The error of the parameter $d$ was calculated using the exact
differential method applied to the result of the least square fitting of
numerical data to relation (1).

###  III.3 Density autocorrelation function

While the packing fraction describes the global structure of a set of shapes,
the local statistics of their positions can be better understood by probing
the density autocorrelation function which can be defined as follows:

$g(R)=\lim_{\text{d}R\to 0}\frac{\langle N(R,R+\text{d}R)\rangle}{\theta\,2\pi
R\,\text{d}R}$ (4)

where $\langle N(R,R+\text{d}R)\rangle$ is the mean number of shapes, whose
centers are placed in the distance between $R$ and $R+\text{d}R$ from the
center of a given figure. The presence of $\theta$ in the denominator is for
normalization $g(R\to\infty)=1$. The density autocorrelation functions for
several packings are shown in Fig. 9.

Figure 9: The density autocorrelation function for several packings. The main
panel shows the density autocorrelation function for packings built of regular
polygons of $n=3,4,5,6$, and $10$ sides. Inset shows the same function but for
packings built of rounded squares of $r=0.02$, $0.2$, and $0.5$.

The correlation functions have typical features of ones observed for RSA
packings or equilibrium liquids. It was shown that for one-dimensional
packings $g(R)$ vanishes superexponentially [28], which is also observed here.
The plots partially explain the behavior of the mean saturated packing
fraction. It is the highest for squares, while at the same time, we observe
the shortest distance between neighboring shapes of this type. On the other
hand, the plot farthest to the right corresponds to triangles, which form
looser configurations. The density autocorrelation for packings built of
rounded squares is practically the same as for squares if the rounding is
small $r<0.1$. For larger $r$ we observe a second maximum, which grows as the
shape approaches the disk.  This maximum first appears at $R=\sqrt{2}$, which
corresponds to the slight cusp in $G(R)$ for squares and appears due to its
rapid decay when squares are not in touch. However, where square corners are
rounded, this distance decreases, and the cusp transforms into the peak, which
travels left with an increasing radius of rounding $r$ and grows up to
infinity in the limit of touching disks [14, 13]. The effect of rounding the
squares on the positioning of the density autocorrelation function peak could
be interesting to study because it would reveal additional factors that could
be used to customize the growth kinetics, saturation, and tightness of
packing. It is intriguing that while both techniques of going from square to
circle finally produced similar d and saturation densities, their response
behavior in terms of the density autocorrelation function appears to be
different.

The results are  expected because, if these polygons were stacked in a lattice
pattern, one would expect squares to be the most densely packed (due to a
higher likelihood that the packing would have no corners that could
potentially leave some of the available space open). Moreover, particles with
odd numbers of sides will undoubtedly have more unoccupied space as their
nearest neighbors cannot occupy  the space near them without overlap. When the
number of sides  increases, the maximum saturation packing for the  disks
should be reached. A regular polygon’s shape may also affect the volume that
it excludes.  Because it is more rounded and has a bigger internal volume than
the pentagon in respect to its circumference, the hexagon has a lower excluded
volume than a regular pentagon of the same size.

##  IV Conclusions

The square appears to be a unique shape in terms of random sequential
adsorption as the two-dimensional oriented packings built of particles of this
shape characterize significantly different kinetics given by relation (2)
while packings built of all other regular polygons, as well as the majority of
other shapes obey the power-law (1). By studying the kinetics of  packings
built of rounded squares we show that even quite small rounding, which, in
practice, is not noticeable visually, changes the kinetics to the one
typically observed in similar settings, namely $d=2$ for shapes with two
degrees of freedom. This transition is not observed in other characteristics.
It is important to add that to study the asymptotic properties of the kinetics
of packing growth relatively large packings have to be generated, and
preferably as close as possible to their saturation points, contrary to the
packing fraction which can be quite precisely estimated using relatively small
packings, as long as periodic boundary conditions are used [10].

For rounded squares, we also observe the transition between the packing
fractions of configurations formed by squares and disks. However, the
transition occurs for significantly larger values of parameter $r$ responsible
for the amount of rounding than in the case of packing growth kinetics. The
study of density autocorrelation functions seems to give additional details
regarding packing densities with,  however, no further insight into the
asymptotic behavior.

To handle processing in real applications, such as “ Pickering emulsion” and
adsorption in  catalysts, unique “particle engineering schemes” are becoming
necessary. For instance, it has been discovered here that for adsorption
purposes, items that at a later time can be imagined as molecules or particles
with square shapes have distinct growth kinetics and saturation, allowing for
customizable levels of adsorbability. Moreover, rounding techniques and
modifications made to the same methodology may be adjusted to solve the dearth
of theoretical models for  two-dimensional adsorption using RSA.

## Acknowledgments

Numerical simulations were carried out with the support of the
Interdisciplinary Center for Mathematical and Computational Modeling (ICM) at
the University of Warsaw under grant no. GB76-1.

## References

  * Evans [1993] J. W. Evans, Random and cooperative sequential adsorption, Rev. Mod. Phys. 65, 1281 (1993).
  * Talbot  _et al._ [2000] J. Talbot, G. Tarjus, P. Van Tassel, and P. Viot, From car parking to protein adsorption: an overview of sequential adsorption processes, Col. Surf. A 165, 287 (2000).
  * Kubala  _et al._ [2022] P. Kubala, P. Batys, J. Barbasz, P. Weroński, and M. Cieśla, Random sequential adsorption: An efficient tool for investigating the deposition of macromolecules and colloidal particles, Adv. Col. Interface Sci. , 102692 (2022).
  * Torquato and Stillinger [2010] S. Torquato and F. H. Stillinger, Jammed hard-particle packings: From kepler to bernal and beyond, Rev. Mod. Phys. 82, 2633 (2010).
  * Flory [1939] P. J. Flory, Intramolecular reaction between neighboring substituents of vinyl polymers, JACS 61, 1518 (1939).
  * Feder [1980] J. Feder, Random sequential adsorption, J. Theor. Biol. 87, 237 (1980).
  * Onoda and Liniger [1986] G. Y. Onoda and E. G. Liniger, Experimental determination of the random-parking limit in two dimensions, Phys. Rev. A 33, 715 (1986).
  * Wang [1994] J.-S. Wang, A fast algorithm for random sequential adsorption of discs, Int. J. Mod. Phys. C 5, 707 (1994).
  * Haiduk  _et al._ [2018] K. Haiduk, P. Kubala, and M. Cieśla, Saturated packings of convex anisotropic objects under random sequential adsorption protocol, Phys. Rev. E 98, 063309 (2018).
  * Cieśla and Ziff [2018] M. Cieśla and R. M. Ziff, Boundary conditions in random sequential adsorption, J. Stat. Mech. Theor. Exp. 2018, 043302 (2018).
  * Vigil and Ziff [1989] R. D. Vigil and R. M. Ziff, Random sequential adsorption of unoriented rectangles onto a plane, J. Chem. Phys. 91, 2599 (1989).
  * Viot  _et al._ [1992] P. Viot, G. Tarjus, S. Ricci, and J. Talbot, Random sequential adsorption of anisotropic particles. i. jamming limit and asymptotic behavior, J. Chem. Phys. 97, 5212 (1992).
  * Pomeau [1980] Y. Pomeau, Some asymptotic estimates in the random parking problem, J. Phys. A 13, L193 (1980).
  * Swendsen [1981] R. H. Swendsen, Dynamics of random sequential adsorption, Phys. Rev. A 24, 504 (1981).
  * Brosilow  _et al._ [1991] B. J. Brosilow, R. M. Ziff, and R. D. Vigil, Random sequential adsorption of parallel squares, Phys. Rev. A 43, 631 (1991).
  * Shelke  _et al._ [2007] P. B. Shelke, M. D. Khandkar, A. G. Banpurkar, S. B. Ogale, and A. V. Limaye, Universality of the power-law approach to the jamming limit in random sequential adsorption dynamics, Phys. Rev. E 75, 060601(R) (2007).
  * Moud [2023] A. A. Moud, Precise determination of the saturation coverage of polygons in silico using exclusion assisted packing technique, Journal of Statistical Physics: In press. 10.1007/s10955-023-03097-4 (2023).
  * Akeda and Hori [1975] Y. Akeda and M. Hori, Numerical test of palasti’s conjecture on two-dimensional random packing density, Nature 254, 318 (1975).
  * Zhang [2018] G. Zhang, Precise algorithm to generate random sequential adsorption of hard polygons at saturation, Phys. Rev. E 97, 043311 (2018).
  * Cieśla  _et al._ [2019] M. Cieśla, P. Kubala, and G. Zhang, Saturated random packing built of arbitrary polygons under random sequential adsorption protocol, Physical Review E 100, 062901 (2019).
  * Cieśla [2017] M. Cieśla, Scaling properties of the number of random sequential adsorption iterations needed to generate saturated random packing, Journal of Statistical Physics 166, 39 (2017).
  * Baule [2017] A. Baule, Shape universality classes in the random sequential adsorption of nonspherical particles, Phys. Rev. Lett. 119, 028003 (2017).
  * Yousefi  _et al._ [2019] P. Yousefi, H. Malmir, and M. Sahimi, Morphology and kinetics of random sequential adsorption of superballs: From hexapods to cubes, Phys. Rev. E 100, 020602 (2019).
  * Cieśla  _et al._ [2021] M. Cieśla, P. Kubala, and K. Kozubek, Algorithms to generate saturated random sequential adsorption packings built of rounded polygons, Phys. Rev. E 103, 063308 (2021).
  * Hinrichsen  _et al._ [1986] E. L. Hinrichsen, J. Feder, and T. Jøssang, Geometry of random sequential adsorption, Journal of statistical physics 44, 793 (1986).
  * Cieśla [2013] M. Cieśla, Continuum random sequential adsorption of polymer on a flat and homogeneous surface, Phys. Rev. E 87, 052401 (2013).
  * Cieśla  _et al._ [2020] M. Cieśla, K. Kozubek, P. Kubala, and A. Baule, Kinetics of random sequential adsorption of two-dimensional shapes on a one-dimensional line, Phys. Rev. E 101, 042901 (2020).
  * Bonnier  _et al._ [1994] B. Bonnier, D. Boyer, and P. Viot, Pair correlation function in random sequential adsorption processes, J. Phys. A: Math. Gen. 27, 3671 (1994).


# MTRESS 3.0 – Modell Template for Residential Energy Supply Systems ††thanks:
This work has been funded by the German Federal Ministry of Education and
Research, Grant No. 03SF0624L.

Patrik Schönfeldt<EMAIL_ADDRESS>Institute of Networked Energy
Systems
German Aerospace Centre (DLR)
Oldenburg, Germany  Sunke Schlüters<EMAIL_ADDRESS>Institute of
Networked Energy Systems
German Aerospace Centre (DLR)
Oldenburg, Germany  Keno Oltmanns<EMAIL_ADDRESS>Institute of Networked
Energy Systems
German Aerospace Centre (DLR)
Oldenburg, Germany

###### Abstract

MTRESS is a tool that facilitates the creation of models of residential energy
supply systems by providing a template with meaningful presets. This model can
then be used to linearly optimise the operation of the energy system. Version
3.0 enables multiple locations belonging to one energy system to be defined.
Furthermore, it adds hydrogen as an energy carrier with the possibility to
convert power to gas using electrolysis and to store gas at different pressure
levels. Additionally, we reworked the Python API, giving more flexibility to
the user.

##  1 Introduction

Graph based formulation of an energy system in order to formulate a (mixed-
integer) linear optimisation problem can be considered a standard approach. In
Python, there are several tools implementing that approach. For example, there
are PyPSA [1], urbs [2], OSeMOSYS [3], PowerGAMA [4], Minpower [5], MOST [6],
Calliope [7], and oemof.solph [8]. It is also noteworthy that they have
similar dependencies, i.e. most are based on Pyomo [9, 10]. As a result,
improvements to one of those tools can be adapted for the others.

The present contribution aims to further simplify the creation of this kind of
models, even when the level of detail is increased. Also, it offers different
styles of creating energy system models, either by preparing formatted data in
text files or by scripting in Python. The implementation of these styles is
influenced by the aforementioned tools Calliope and oemof.solph. Calliope
emphasises a strict separation of data and program, and considers the
definition of a model as data that is parameterised using text files in YAML
and CSV formats. On the other hand, oemof.solph implements a ”create and add”
workflow, so that implementation of the specific model and the data are
located closely to one another in the code.

##  2 Architecture of MTRESS 3.0

The aim of MTRESS is the simple creation of energy supply system models while
keeping model accuracy at a considerably high level. To give meaningful
presets, a number of assumptions is made that are often met. With MTRESS 3.0,
a number of new concepts is introduced to allow for more flexibility compared
to previous versions. Further – as MTRESS wraps oemof.solph – energy systems
created using the template can also be extended and further refined using the
full flexibility of oemof.solph.

The overall structure is created by building blocks of the five classes
EnergySystem, Location, EnergyCarrier, Component, and Demand. One major
simplification compared to other methods to create energy system models is
that the connections do not have to be defined by the user.

###  2.1 Energy System

The EnergySystem serves as a container for the model. It holds global
information, e.g. about the time, and defaults that can be overwritten for
specific locations, e.g. weather data. Furthermore, all locations of the model
and the connections between them are registered with the EnergySystem object.
Once the energy system is about to be solved, it makes sure every location has
all the necessary connections and constraints set.

###  2.2 Location

A Location collects different energy carriers, components and demands.

To allow for automatic connections between the components and demands, every
energy carrier (e.g. electricity or heat) and every component (e.g. a heat
pump) can only be defined once per location (or left out). To define multiple
instances of one energy carrier with different configurations, multiple
locations have to be defined.

###  2.3 Energy Carrier

Each EnergyCarrier represents a form of energy in a energy systems location
and acts as a distribution hub between components. Currently, electricity,
heat, natural gas, and hydrogen are implemented. These energy carriers may
have a “grid connection”, meaning that energy may be purchased or sold.
Currently, the following energy carriers are implemented:

#### Electricity

Electricity is implemented in a way that allows the tracking of which energy
is used where. While there is no physical difference, the model distinguishes
between electricity that is produced locally and electricity coming from the
grid. This enables different fees and tariffs based on the origin of the
electricity to be considered.

#### Heat

The temperature has a significant impact on the performance of renewable
energy supply systems, e.g. for the Coefficient of Performance (COP) of heat
pumps or the output of a solar thermal collector. Thus, the energy carrier
heat allows to optimise both temperature and heat. This is done by defining
several discrete temperature levels [11]. Besides the temperature levels, a
reference temperature can be defined. This can be useful to simplify the model
by setting the reference to a return temperature, resulting in the
corresponding return flow to be considered at zero energy. Some temperatures,
i.e. the ones of sources for heat pumps, are not considered by the energy
carrier. To emphasise that fact, these sources are defined as anergy sources,
which are not connected to the energy carrier but only to the heat pump.

#### Gas

The concept of the energy carrier heat is expanded to (compressible) gas. It
can be expanded from high pressure to lower pressure or compressed (using a
compressor). The pressure level plays a role, especially when energy is stored
in gaseous form.

As hydrogen was recently implemented into MTRESS in the form of an
electrolyzer and a hydrogen compressor, the gas carrier currently finds
application in that field.

###  2.4 Component

A Component represents a technology used in the system. They can serve as
sources or connect either energy carriers to each other or different
temperature levels (in case of the heat carrier) or pressure levels (in case
of a gaseous carrier) of the carrier. The following collection of components
should give an idea of the concept:

#### Heat Pump

The heat pump uses electricity and provides heat (possibly at different
temperature levels). Further, it connects to every available anergy source at
the location. The COPs are automatically calculated based on the information
given by the heat carrier and the anergy sources.

#### Anergy Sources

There can be several anergy sources, e.g. down hole heat exchangers or air
source heat exchangers. They hold a time series for both the temperature and
the power limit that can be drawn from the source. Furthermore, additionally a
total limit can be defined. This is particularly important for geothermal
sources that need to recover.

#### Combined Heat and Power

Combined heat and power (CHP) units consume gas and supply both heat and
electricity. It is possible to choose between different models that linearize
the CHP to consider (or neglect) nonlinear part-load efficiencies.

#### Renewable Electricity Source

Renewable electricity sources are representing photovoltaic (PV), wind power,
or river-flow water generators. They contain a time series for the maximum
power that can be drawn from them. For PV, the time series can be
automatically calculated by MTRESS from a weather time series.

#### Energy storage

Energy storage is implemented per carrier. Thus, there can be electricity
storage (battery), heat storage, and gas storage. The three types incorporate
the properties of the corresponding energy carrier. The battery is represented
by a state of charge. We decided to not allow changing voltage levels for the
electricity, as this technical option is typically not implemented. For heat
storage and gas storage, stored content depends on an optimisation variable
(temperature or pressure), the used linearization is described in Sec. 3.

#### PEM Electrolyzer

In regard to the generation of hydrogen, a protone-exchange-membrane (PEM)
electrolyzer was implemented. It uses electricity and water (H2O) to produce
hydrogen (H2) and oxygen (O). For the simplicity of the model, water-input as
well as oxygen-output can be neglected. However, the usage of waste heat
generated by the electrolyzer was implemented into the model. Usually waste-
heat of PEM electrolyzers is produced at
$77\text{\,}\mathrm{\SIUnitSymbolCelsius}$ [12], while hydrogen is produced at
a pressure of around $30\text{\,}\mathrm{bar}$ [13].

#### Compressor

To consider the necessity for different pressure levels of hydrogen, e.g. for
transport or usage at the maximum density of $700\text{\,}\mathrm{bar}$ or a
storing at $350\text{\,}\mathrm{bar}$ for rocket science applications [14], a
compressor was implemented. The compressor uses electricity to rise the
pressure level of the produced hydrogen.

###  2.5 Demand

Demands contain time series of energy that is needed. Depending on the type of
demand (e.g. electricity or heat), it automatically connects to the
corresponding energy carrier. Also, a name identifying the demand has to be
given that is unique for the location. This is because multiple demands of one
type can exist for one location. The different types have different
complexity: Electricity demand does not need any further specification, heat
and gas demand need a specified temperature or pressure level, respectively.
Further, energy from electricity and the gaseous carriers is just consumed,
heat demands have a returning energy flow.

###  2.6 Helpers

Besides the part of the energy system, MTRESS offers some helpers to
facilitate working with the models.

#### Reproducible runs

It is possible to create an energy system model using the Python API and to
export it to a YAML file afterwards. This way, models created dynamically
using the Python API can be easily checked and results can be reproduced
without running the full program again.

#### Analysis of the results

Energy flows in MTRESS are categorised using tags. It is possible to filter
for these tags, e.g. to sum up heat coming from heat pumps in several
buildings. Further analysis and plotting can be done by exporting results to
Pandas [15].

#### Energy system graph plotting

For quick checks and for understanding the layout, we implemented a simple
graph plotting routine. It displays every node of the final energy system
graph and the connections between those. Trapezoid shapes signify sources
(smaller side up) or sinks (smaller size down), cycles signify busses and
octagonal shapes signify more complicated energy converters. Adding to the
same functionality in solph, the MTRESS plotter groups these nodes
corresponding to the location and further according energy carrier, component,
or demand they belong. An example is displayed in Fig. 2. For a later
publication, the plotted graphs can be exported as graphml and edited manually
in graph visualisation tools.

##  3 Linearized Formulation of Energy Storage

For the gas storage, available pressure depends on the storage content. The
latter, however, is an optimisation variable. The same argument is valid for a
(sensible) heat storage when it comes to the temperature of the medium that
can be taken from the storage. Thus, linear formulation of such storage types
is not trivial. As discussed in Sec.2.3, we implement an approach that uses
discrete levels. Using the example of a gas storage, it has varying pressure
that defines the content of the storage, i.e.

$E(t)=c_{\mathrm{E}}\cdot p(t),$ (1)

where $E$ denotes the energy content of the storage at time $t$, $p$ the
pressure inside the storage at time $t$ and $c_{E}$ is a storage specific
constant depending on the energy density of the gas and the volume of the
storage. Following the definition of the EnergyCarrier, the pressure is
discretized

$p_{n},\,n\in\\{1,\dotsc,N\\},$ (2)

and $E_{n}:=c_{\mathrm{E}}\cdot p_{n}$ denote the energy contents of the
storage at the defined pressure levels.

Now, we want that an active (or usable) level is signified by the binary
status variable $y_{n}(t)\in\\{0,1\\}$ with

$\displaystyle y_{n}(t)$ $\displaystyle=0\text{ if }E(t)<E_{n},\,\text{and}$
(3a) $\displaystyle y_{n}(t)$ $\displaystyle=1\text{ if }E(t)\geq E_{n}.$ (3b)

Note that $E$ is being optimised, so Eq. (3) cannot be read as a causal
relation. We also need a linear formulation. We suggest

$\displaystyle y_{n}(t)$ $\displaystyle\leq\frac{E(t)}{E_{n}},$ (4a)
$\displaystyle\hat{y}_{n}(t)$
$\displaystyle\geq\frac{E(t)-E_{n}}{E_{\mathrm{max}}},$ (4b)
$\displaystyle\bar{y}_{n}(t)$ $\displaystyle=1-\hat{y}_{n}(t),$ (4c) where
$E_{\mathrm{max}}$ is the maximum storage content. Equation (4a) guarantees
Eq. (3a) but relaxes Eq. (3b) in the sense that $y_{n}$ is not forced to be
active. To compensate for that, Eq. (4c), enforces $\bar{y}_{n}=0$ for the
given case. The symbol is chosen to emphasise that it can be read as an
inverse status. If this should be strictly the case, also
$1={y}_{n}(t)+\bar{y}_{n}(t)$ (4d)

has to be defined to eliminate the possibility that
${y}_{n}(t)=\bar{y}_{n}(t)$, i.e. at $E=E_{n}$, where both can be 1.

Now, note that gas leaving the storage needs to have lower pressure than the
storage and higher pressure is needed to increase the storage content. Thus,

$\displaystyle P_{\mathrm{out},n}(t)$ $\displaystyle\leq y_{n}(t)\cdot
P_{\mathrm{out,max},n},$ (5a) $\displaystyle P_{\mathrm{in},n}(t)$
$\displaystyle\leq\bar{y}_{n}(t)\cdot P_{\mathrm{in,max},n},$ (5b)

where $P_{\mathrm{out},n}$ and $P_{\mathrm{in},n}$ denote the power flow out
of and into the storage, respectively, at the pressure $p_{n}$.
$P_{\mathrm{out},\mathrm{max},n}$ and $P_{\mathrm{in},\mathrm{max},n}$ are the
absolute limits of these flows. This way, it is guaranteed that
$P_{\mathrm{out}}(t)=0$ if the storage content is not sufficient and
$P_{\mathrm{in}}(t)=0$ if the storage content is too high.

So, at each point in time, the storage content $E$ defines the pressure $p$ in
the storage and thus limits the withdrawal pressure or the feed pressure. As
multiple output and input flows can be active at the same time, it is
meaningful to also define a weighted limit

$\sum_{n}P_{\mathrm{out},n}(t)\cdot w(p_{n})\leq
P_{\mathrm{out},\mathrm{max}},$ (6)

so that the total power flow out of the storage, and analogously for the input
flows, is also constrained.

Figure 1: Working principle of the status variable $y_{n}$: If the stored
energy exceeds the limit (at the end of a time step), the corresponding level
might be active.

A further complication comes up due to discrete time steps. A naive
implementation might allow to pass a pressure level $p_{n}$ within one time
step using $P_{\mathrm{out},n}(t)>0$ – even a full storage can be completely
emptied using the highest pressure level. If either power limits as in Eq. (5)
are set low or the time resolution is high, this will not impose a problem.
However, there is a solution that explicitly solves the issue.

$\displaystyle P_{\mathrm{out},n,t}$ $\displaystyle\leq y_{n,t}\cdot
P_{\mathrm{out,max},n},$ (7a) $\displaystyle P_{\mathrm{in},n,t}$
$\displaystyle\leq\bar{y}_{n,t}\cdot P_{\mathrm{in,max},n},$ (7b)

is a time-discrete reformulation of Eq. (5), there the index $t$ denotes the
time interval between $t$ and $t+\Delta t$. If now the constraint for the
status variable is defined by the energy content at the end of that time
interval

$y_{n,t}\leq\frac{E_{t+\Delta t}}{E_{n}},$ (8)

it is no longer possible to cross a level $p_{n}$ using the corresponding
power flows. For a storage with levels at $E_{n}=[0,0.3,0.6,0.9,1]\cdot
E_{\mathrm{max}}$, the principle is depicted in Fig. 1. Lines for the highest
and the lowest level are omitted because they are always active or inactive,
respectively. This implies that at any time the lowest level cannot be used
for storing energy and energy at the highest level cannot be obtained from the
storage. For a fully mixed heat storage, this implication can be read as the
fact that a storage will never reach (exactly) the temperature that is used to
feed it.

Generally, however, inside a heat storage stratification is possible. Thus, a
second heat storage model is offered [11], that works without binary
variables. It models different temperature layers of variable height that
share the same volume. We assume that the warmest temperatures are at the top
and temperature decreases until the bottom of the storage tank. This way,
multiple temperatures can be served at the same time using the same storage.

For electricity, the presented method can be used to implement charging-rates
that depend on the state of charge.

##  4 Example

Figure 2: Example energy system graph as plotted by the automatic plotting
routine.

The case of a single-family home using a heat pump for heat supply will serve
as a basic example. First of all, the EnergySystem object has to be created.
As stated above, it will nest the locations and contains global information
about the problem. The simplest way to define the time horizon of the
optimisation is by handing the needed information as a dictionary:

⬇

energy_system = EnergySystem(time_index={

”start”: ”2021-07-10 06:00:00”,

”end”: ”2021-07-10 08:00:00”,

”freq”: ”60T”,

})

Note that the boundaries given above define two time intervals. Next, we
create an empty location. As there can be many locations, the location has to
be given a unique name as an identifier.

⬇

house = Location(name=”SFH”)

energy_system.add(house)

We add the location to the model at this point, just to have it done.
Generally speaking, the order of the individual steps of creating the model is
not important for the result. In particular, it is also possible to create and
populate the locations first, before creating the EnergySystem object.

⬇

house.add_carrier(

carriers.Electricity(costs={

”working_price”: 35, # ct/kWh

”demand_rate”: 0

})

)

house.add_demand(

demands.Electricity(

name=”electricity demand”,

time_series=[7, 8.4], # kW

)

)

By giving a working price, it is implied that energy for that carrier can be
purchased (from an external grid). The convention for the units is not
binding, e.g. using $\text{\,}\mathrm{\text{€}}\text{\,}{\mathrm{MW}}^{-1}$
and $\text{\,}\mathrm{MW}$ would also be possible, as long as it is
consistently done for all carriers and demands. As for all demands, the
electricity demand time series is given in units of power. The energy system
defined up to this point describes a complete electricity-only energy system
and could be used as such.

For the heat sector, temperature levels and a reference temperature have to be
defined. The reference temperature has to be lower than or equal to the lowest
return temperature. Besides, every temperature required by the demands has to
be defined as a level for the carrier.

⬇

house.add_carrier(

carriers.Heat(

temperature_levels=[

20, 30, 55], # °C

reference_temperature=10, # °C

)

)

house.add_demand(

demands.FixedTemperatureHeat(

name=”space heating”,

flow_temperature=30, # °C

return_temperature=20, # °C

time_series=[13.37, 42], # kW

)

)

house.add_demand(

demands.FixedTemperatureHeat(

name=”hot water”,

flow_temperature=55, # °C

return_temperature=10, # °C

time_series=[0, 12], # kW

)

)

Note that we did not define costs for the heat carrier, an equivalent
expression would be costs=None. Defining it would be interpreted as a
connection to the heating grid, i.e. setting it to zero would make it free.
So, we need an energy conversion technology. In our example, an air-source
heat-pump is used. It consists of two individual components, the actual
HeatPump and the corresponding anergy source:

⬇

house.add_component(

technologies.HeatPump(

thermal_power_limit=None

cop_0_35=3.8, # 1

)

)

house.add_component(

technologies.AirHeatExchanger(

air_temperature=[3, 9], # °C

)

)

In Figure 2, the resulting energy system is displayed as a graph. This graph
was automatically generated by MTRESS building upon using graphviz [16].

Once defined, the model is optimised using:

⬇

energy_system.optimise()

Afterwards, results can be prompted

⬇

electricity_flows = energy_system.flows(Carrier, Electricity)

The alternative is to define the energy system using a yaml file. The
following listing will result in exactly the same model as the code explained
above:

⬇

general:

timeindex:

start: 2021-07-10 06:00:00

end: 2021-07-10 08:00:00

freq: 60T

\parlocations:

SFH:

carriers:

Heat:

temperature_levels:

- 20. # °C

- 30. # °C

- 55. # °C

reference_temperature: 10 # °C

Electricity:

demand_rate: 0 # ct/kW

working_price: 35 # ct/kWh

\pardemands:

Electricity:

name: ”electricity demand”

time_series: [7, 8.4] # kW

FixedTemperatureHeat:

name: ”space heating”

time_series:

- 13.37 # kW

- 42 # kW

flow_temperature: 30 # °C

return_temperature: 20 # °C

FixedTemperatureHeat:

name: ”hot water”

time_series:

- 0 # kW

- 12 # kW

flow_temperature: 55 # °C

return_temperature: 10 # °C

\parcomponents:

AirHeatExchanger:

parameters:

air_temperature: [3, 9] # °C

HeatPump:

parameters:

cop_0_35: 3.8

Note that all comments are voluntary and are just meant to help the reader.

For longer time series, it can be convenient not to inline every value. This
is supported by giving it in the form

⬇

air_temperature: file=weather.csv:temperature (°C)

which denotes that the time series can be found in the column “temperature
($\text{\,}\mathrm{\SIUnitSymbolCelsius}$)” of the file “weather.csv”. Time
series used in models can be either defined at points in time or over time
intervals. For time-intervals, MTRESS uses left-indexed data, meaning that the
time stamp denotes the beginning of the interval the value is valid. Following
the convention of solph this is being made explicit by adding the time step
limiting the last interval with “NaN” (for “not a number”) instead of data. It
might sound a bit pedantic at first, but giving the length of the last
interval is actually required because there is no requirement that all time
intervals have the same length. Additionally, being explicit helps avoiding
common off-by-one errors.

##  5 Summary and Outlook

We presented version 3.0 of MTRESS, a tool that allows to create optimisation
models for residential energy supply systems in just a few lines of code.
These models can contain multiple temperature levels for the energy carrier
heat and multiple pressure levels for gases (like natural gas or hydrogen) in
a transparent and flexible way. This way i.e. storage capacities can be
modelled more realistically compared to traditional linear models without
levels. There are two ways to create those models, on one hand, there is a
Python API for dynamic modelling, on the other hand a YAML interface is
provided. For the future, it is planned to allow exporting models as YAML
files to facilitate reproducing results of energy system designs that were
created using a dynamic algorithm.

## Acknowledgments

We thank Uwe Krien for bringing up the topic of properly indexing data and
Lennart Schürmann for discussions about an improved storage formulation. Elif
Turhan and Herena Torio are acknowledged for proofreading the manuscript.

## References

  * [1] Thomas Brown, Jonas Hörsch, and David Schlachtberger.  PyPSA: Python for power system analysis.  Journal of Open Research Software, 6(1):4, jan 2018.
  * [2] Johannes Dorfner, Konrad Schönleber, Magdalena Dorfner, sonercandas, froehlie, smuellr, dogauzrek, WYAUDI, Leonhard-B, lodersky, yunusozsahin, adeeljsid, Thomas Zipperle, Simon Herzog, kais siala, and Okan Akca.  tum-ens/urbs: urbs v1.0.1, July 2019.
  * [3] Mark Howells, Holger Rogner, Neil Strachan, Charles Heaps, Hillard Huntington, Socrates Kypreos, Alison Hughes, Semida Silveira, Joe DeCarolis, Morgan Bazillian, and Alexander Roehrl.  Osemosys: The open source energy modeling system: An introduction to its ethos, structure and development.  Energy Policy, 39(10):5850–5870, 2011.  Sustainability of biofuels.
  * [4] Harald G. Svendsen and Ole Chr Spro.  Powergama: A new simplified modelling approach for analyses of large interconnected power systems, applied to a 2030 western mediterranean case study.  Journal of Renewable and Sustainable Energy, 8(5):055501, 2016.
  * [5] A. Greenhall, R. Christie, and J-P Watson.  Minpower: A power systems optimization toolkit.  In 2012 IEEE Power and Energy Society General Meeting, pages 1–6, 2012.
  * [6] Carlos E. Murillo-Sánchez, Ray D. Zimmerman, C. Lindsay Anderson, and Robert J. Thomas.  Secure planning and operations of systems with stochastic sources, energy storage, and active demand.  IEEE Transactions on Smart Grid, 4(4):2220–2229, 2013.
  * [7] Stefan Pfenninger and Bryn Pickering.  Calliope: a multi-scale energy systems modelling framework.  Journal of Open Source Software, 3(29):825, 2018.
  * [8] Uwe Krien, Patrik Schönfeldt, Jann Launer, Simon Hilpert, Cord Kaldemeyer, and Guido Pleßmann.  oemof.solph – a model generator for linear and mixed-integer linear optimisation of energy systems.  Software Impacts, 6, Nov 2020.
  * [9] Michael L. Bynum, Gabriel A. Hackebeil, William E. Hart, Carl D. Laird, Bethany L. Nicholson, John D. Siirola, Jean-Paul Watson, and David L. Woodruff.  Pyomo–optimization modeling in python, volume 67.  Springer Science & Business Media, third edition, 2021.
  * [10] William E Hart, Jean-Paul Watson, and David L Woodruff.  Pyomo: modeling and solving mathematical programs in python.  Mathematical Programming Computation, 3(3):219–260, 2011.
  * [11] Patrik Schönfeldt, Adrian Grimm, Bhawana Neupane, Herena Torio, Pedro Duran, Peter Klement, Benedikt Hanke, Karsten von Maydell, and Carsten Agert.  Simultaneous optimization of temperature and energy in linear energy system models.  In 2022 Open Source Modelling and Simulation of Energy Systems (OSMSES), pages 1–6, 2022.
  * [12] Joris Tiktak.  Heat Management of PEM Electrolysis: A study on the potential of excess heat from medium- to large-scale PEM electrolysis and the performance analysis of a dedicated cooling system.  Masters thesis, Delft University of Technology, Delft, 2019.
  * [13] H-TEC PEM-Elektrolyseur ME450.  https://www.h-tec.com/produkte/detail/h-tec-pem-.
  * [14] H2ORIZON – ein Meilenstein in der Energiewende, Nov 2022.  https://www.h2orizon.de/forschungsprojekt.html.
  * [15] Wes McKinney et al.  Data structures for statistical computing in python.  In Proceedings of the 9th Python in Science Conference, volume 445, pages 51–56. Austin, TX, 2010.
  * [16] Emden R. Gansner and Stephen C. North.  An open graph visualization system and its applications to software engineering.  Software: Practice and Experience, 30(11):1203–1233, 2000.