# Typhon: Automatic Recommendation of Relevant Code Cells in Jupyter Notebooks

Chaiyong Ragkhitwetsagul, Veerakit Prasertpol, Natanon Ritta, Paphon Sae-Wong,
Thanapon Noraset and Morakot Choetkiertikul  Faculty of Information and
Communication Technology, Mahidol University, Nakhon Pathom, Thailand

###### Abstract

At present, code recommendation tools have gained greater importance to many
software developers in various areas of expertise. Having code recommendation
tools has enabled better productivity and performance in developing the code
in software and made it easier for developers to find code examples and learn
from them.

This paper proposes Typhon, an approach to automatically recommend relevant
code cells in Jupyter notebooks. Typhon tokenizes developers’ markdown
description cells and looks for the most similar code cells from the database
using text similarities such as the BM25 ranking function or CodeBERT, a
machine-learning approach. Then, the algorithm computes the similarity
distance between the tokenized query and markdown cells to return the most
relevant code cells to the developers.

We evaluated the Typhon tool on Jupyter notebooks from Kaggle competitions and
found that the approach can recommend code cells with moderate accuracy. The
approach and results in this paper can lead to further improvements in code
cell recommendations in Jupyter notebooks.

##  I Introduction

Code recommendation has been a great advantage to a lot of software developers
for auto-completing the written code or suggesting the next tokens or
statements to the developers. This function has helped improve developer
productivity significantly. It works by taking code the the developer is
currently working on and suggesting the complete code snippet back to the
user. This reduces the time the developers need to complete their code
tremendously.

At present, there are code recommendation tools with two kinds of
functionalities: code-to-code recommendation and text-to-code recommendation.
A code-to-code recommendation is an approach where the recommendation tools
understand the current context of developers’ code and recommend the code
accordingly. There have been multiple studies of code-to-code recommendation
such as Aroma [1], Senatus [2], Strathcona [3], and Example Overflow [4] using
techniques such as clustering and ranking of code snippets, code structure,
Jaccard similarity and Minhas-LSH on code, and TF-IDF. Text-to-code
recommendations have developers inquire about the details and behaviors of the
code they require from the recommendation tool using natural language text.
The example of the existing approaches for text-to-code recommendation include
GitHub Copilot111https://github.com/features/copilot and Tabnine
222https://aiterms.net/tabnine/ [5].

Currently, programming is not limited to only building software, but also data
analysis. In the data science domain, there are many various kinds of IDE
tools that let developers and researchers work on their projects. The most
commonly used platform is Jupyter Notebook, which enables a simplified and
easy-to-navigate workflow, making Jupyter Notebook one of the most popular
options in data Science project development [6].

In this paper, we are interested in code recommendations for the Jupyter
Notebooks platform by reusing code snippets that are already written by data
science developers. We want to study if we can give appropriate code
recommendations based on a given markdown text in Jupyter Notebooks. This can
help reduce the developer’s time by not writing the whole code by themselves
or learning from the recommended code snippets and improving further upon
them.

To support our motivation, we performed a preliminary experiment of code cell
recommendation in Jupyter Notebooks using GitHub Copilot on the content of 5
notebooks from the KGTorrent dataset [7]. We gave the markdown cell queries
and checked the returned Python whether they were relevant to the given
markdown cells or not. The result showed that the GitHub Copilot efficiency
offers varied performance with a correctness percentage ranging from a minimum
of 0.00 percent to a maximum of 51.16 percent, with an average of 21.83
percent. We investigated the results and found that it is potentially due to
the specific environment of the Jupyter Notebook, which has mixes of natural
language (markdown text) and code. In general, Copilot will take a partial
comment, which is supposed to be relatively short, and the whole source code
file as input and suggest back possible solutions the programmers may need.
Many data science notebooks often have long descriptive text in markdown
cells. We found that the Copilot sometimes performs source code suggestions
poorly when the given input comment is large, and the whole file context
contains a long descriptive text. The given situation often produces non-
reusable source code or a comment string related to the input comment instead
of the source code.

Therefore, this paper attempts to propose an alternative source code
suggestion tool that provides recommended source code from a given descriptive
markdown text as input, regardless of the size of the description. Moreover,
our proposed technique will not be based on code generation, but on code
reuse. Our approach leverages the practice of code reuse by recommending code
cells in Jupyter notebooks using existing code in a large Jupyter notebook
code base in Kaggle, the well-known machine learning and data science
platform, using the information from the associated markdown text. The
contribution of this paper is the similarity matching techniques specifically
designed for Jupyter Notebook code cells. Our code-to-markdown cell similarity
compares a given markdown cell to other markdown cells and code cells in
Kaggle, a large-scale Jupyter Notebooks corpus.

##  II Background

###  II-A Jupyter Notebooks

Jupyter Notebook is a web-based interactive computing and documenting tool
that functions in terms of a digital document containing cells of text, code,
and computing results proposed by Kluyver et al. [6]. The concept of the
Jupyter Notebook evolves from the interactive shell or REPL (Read-Evaluate-
Print-Loop), which is the basis of interactive programming. Considering that
many academic researchers require to document their research progress, they
often need to write computer code to perform computative operations,
statistical tests, and run simulations. Delivering the composed code in
academic papers via human descriptive language is not precise and is
unreliable for reproducing the implementation code in that paper. Typical
academic papers often supply the supplement code in a separate section and are
often isolated from the written content. The readers must cross-reference the
written content and the relevant code. Consequently, this reduces the
readability of the papers and might cause inconsistencies in the code and the
prose. Jupyter Notebook aims to improve readability and increase the
document’s reproducibility and computative results. Therefore, the Notebook is
designed to support the scientific research documentation workflow by
integrating the text, code, and computative results into a single view. Thus,
reducing the effort to cross-reference between code and prose. This Notebook
lets authors document their papers via an interactive text and code cells
component, enabling later editing and re-executing of the code cells. Hence,
increases the reproducibility of the interactive academic documents. Two major
platforms provide hosting services and usage for Jupyter Notebook: Google
Colab and Kaggle, both owned by Google, and Deepnote which has free and paid
Jupyter Notebook services.

Kaggle333https://www.kaggle.com can be thought of as a large warehouse of
open-source Jupyter Notebooks and a playground for data scientists and machine
learning engineers. Kaggle is a website that allows users to find and publish
data sets and provides a web-based Jupyter Notebooks with a data-science
environment. Kaggle users can work together and participate in competitions to
solve data science challenges. Due to the popularity of Kaggle Competitions,
most users of Kaggle have participated in competitions to challenge each
other. Kaggle also has a ranking system for users, and the rank has four
categories, including Competitions, Datasets, Notebooks, and Discussions. The
rank of each category will depend on the contribution and the prize of each
category. According to a large number of competitions, numerous Jupyter
Notebooks have been available on Kaggle as public data. Thus, Kaggle is an
excellent place to learn from the works of various users, from beginners to
experts.

###  II-B Code and Text Similarity Measurement

####  II-B1 Vector Representation Models for Text

Multiple techniques in the Information Retrieval methods are used to recommend
similar text in the collection of documents.

Best Matching 25 (BM25) is a bag-of-words retrieval function, which ranks a
set of documents regarding the query terms appearing in each document
regardless of the proximity within the document. BM25 computes the similarity
between the query terms ($Q$) and the comparison documents $D$ using
$\sum_{i=1}^{n}IDF(q_{i})\cdot\frac{f(q_{i},D)\cdot(k_{1}+1)}{f(q_{i},D)+k_{1}\cdot(1-b+b\cdot\frac{fieldLen}{avgFieldLen})}$

where $(q_{i})$ is the ith query term in $Q$, $IDF(q_{i})$ is the inverse
document frequency of the ith query term, which measures how often a term
occurs in all documents and penalizes terms that are common, $f(q_{i},D)$ is
the number of times the ith query term occurs in document $D$, $k_{1}$ is a
variable that limits how much a single query term can affect the score of a
document, $b$ is a variable that amplifies the effect of the length of a
document relative to the average document length, $fieldLen$ is the length of
the field (in terms) of the document, $avgFieldLen$ is the average length of
the field (in terms) of all documents.

####  II-B2 Vector Representation Models for Code

Moreover, recently there have been studies about creating code models, which
are machine-learning models that derive vectors (i.e., code embeddings) from
code snippets. We discuss some of the recent code models below.

CodeBERT [8] is a natural language processing (NLP) pre-trained model
developed by Microsoft that can understand both programming language and
natural language. The paper presents a hybrid objective function that
incorporates the pre-training task of replaced token detection to train the
model. CodeBERT can be used for various tasks, such as natural language code
search and code documentation generation. The model achieves state-of-the-art
performance on two natural language programming language (NL-PL) applications
and performs better than previous pre-trained models on NL-PL probing. In
summary, CodeBERT is a powerful tool for understanding and generating both
programming language and natural language, which can be useful for various
applications in software engineering.

UniXcoder [9], also developed by Microsoft, is a unified cross-modal pre-
trained model for programming languages, that specializes in code
understanding and code generation tasks. UniXcoder is a pre-trained NLP cross-
modal model that facilitates the communication between natural language and
programming languages for code-related tasks. Additionally, UniXcoder is a
pre-trained model in the CodeBERT series. It is designed to understand and
generate code from natural language queries or descriptions. UniXcoder is a
unified cross-modal model that bridges the gap between natural language and
programming language by leveraging advanced machine learning techniques. It is
pre-trained on a large corpus of natural language text and programming
languages, making it more efficient and effective in code-related tasks.

Additionally, UniXcoder uses a mask attention mechanism with a prefix adapter
to control the encode-decode behavior. This allows the model to adjust its
output based on the input, improving its accuracy and relevance to the task at
hand. Additionally, the model leverages cross-modal content, such as abstract
syntax trees and code comments, to enhance code representation. This improves
the quality of the code generated by the model and makes it more natural and
readable. UniXcoder is capable of both code-related understanding and
generation tasks. This means that it can understand natural language queries
or descriptions and generate the corresponding code, or it can take code as
input and generate natural language descriptions or comments about the code.
This versatility makes it a powerful tool for developers, researchers, and
other professionals working with natural language and programming languages. A
similarity score between two UniXcoder vectors $A$ and $B$ can be computed
using cosine similarity $\cos(\theta)=\dfrac{A\cdot
B}{\left\|A\right\|\left\|B\right\|}$.

##  III Proposed Methodology: Typhon

In this paper, we propose a novel methodology, called Typhon, for recommending
code cells based on existing code cells from Jupyter notebooks in the Kaggle
dataset.

###  III-A Overview of the Approach

As mentioned previously, existing code recommendation tools may not work well
with the Jupyter Notebook environment. Where the markdown description cells
are varied in terms of length and detail. The recommendation tool often cannot
recommend reliable code cells for users because the code and text in the local
file’s context sometimes provide vague and unclear context. Therefore, our
proposed approach focuses on suggesting code cell(s) based on the similarity
of the markdown cell’s content to the existing markdown cells or code cells.
The concept of the code cell recommendation by the approach is presented in
the Figure 1. The approach will search through the existing corpus of Jupyter
notebooks for markdown cells with the most similarity with the input markdown
cell. Then, it suggests the code cell with the most similarity to the input
markdown cell back.

Figure 1: Typhon’s code cell recommendation

###  III-B Jupyter Notebook Corpus: KGTorrent

We leverage KGTorrent, an extensive dataset of computational notebooks with
dense metadata retrieved from the Google-owned platform Kaggle, as the corpus
of Jupyter notebooks in this paper. The collection process of Jupyter
notebooks in KGTorrent covers the time period of November 2015 to October
2020. The data is collected from around 5,598,921 users and 2,910
competitions. KGTorrent has a size of 175 GB and is compressed to the size of
76 GB and consists of 248,761 Jupyter notebooks in the Python programming
language.

###  III-C Markdown-to-Code Cell Recommendation Techniques

The following are the techniques to locate the most similar code cells
according to a given markdown cell that we adopt in this paper.

####  III-C1 BM25

The flow of the BM25 recommendation is depicted in Figure 2. The flow starts
with the system taking a query markdown cell as input, and then computing the
similarity score with markdown cells from Kaggle notebooks in the database
using BM25. The more similar the query markdown cell is with the Kaggle
markdown cells, the higher the similarity score. After that, the system
delivers the code cells corresponding to the matched markdown cells with the
highest similarity. We use Elasticsearch for data indexing, the Python script
was used to upload the data that we processed from the raw data of KGTorrent
by mapping the markdown cell and code cell next to the markdown cell(s) to be
a pair. Subsequently, we get the set of markdown cells as a query to retrieve
comparable markdown pairs from Elasticsearch. We use the default setting of
Elasticsearch where BM25 is the similarity and ranking computation method.

Figure 2: Code cell recommendation using BM25 Figure 3: Code cell
recommendation using UniXcoder

####  III-C2 UniXcoder

The flow of UniXcoder recommendation is as depicted in Figure 3 A query
markdown cell is taken into a UniXcoder model to produce an embedding vector
representing the input. Similarly, the code cells in Kaggle notebooks are also
passed to UniXcoder to create embedding vectors and store them in a database
for querying. For UniXcoder, we search for code cells by computing the
similarity of the code cells directly with the markdown cell because UniXcoder
is a cross-model. Typhon utilizes the Weaviate444https://weaviate.io, a vector
search engine database for accommodating the vectorizing process and storing a
collection of code datasets. Both the similarity-finding method and returning
such results happen within Weaviate. Then, our script returns the recommended
code cells. We use the default similarity setting of Weaviate which is cosine
similarity.

##  IV Evaluation

We evaluate Typhon in two ways: recommendations of code cells containing
Matplotlib plots and a sanity check.

TABLE I: Matplotlib visualization types and the associated query terms Plot type |  Sub plot type | Query term
---|---|---
Basic |  Scatter | plot data using scatter visualization
|  Bar | plot data using bar visualization
|  Stem | plot data using stem visualization
|  Step | plot data using step visualization
|  Fill_between | plot data using fill_between visualization
|  Stackplot | plot data using stackplot visualization
Plots of Arrays and Fields |  Imshow | plot data using imshow visualization
|  Pcolormesh | plot data using pcolormesh visualization
|  Contour | plot data using contour visualization
|  Contourf | plot data using contourf visualization
|  Barbs | plot data using barbs visualization
|  Quiver | plot data using quiver visualization
|  Streamplot | plot data using streamplot visualization
Statistics Plots |  Hist | plot data using hist visualization
|  Boxplot | plot data using boxplot visualization
|  Errorbar | plot data using errorbar visualization
|  Violinplot | plot data using violinplot visualization
|  Eventplot | plot data using eventplot visualization
|  Hist2d | plot data using hist2d visualization
|  Hexbin | plot data using hexbin visualization
|  Pie | plot data using pie visualization
Unstructured Coordinates |  Tricontour | plot data using tricontour visualization
|  Tricontourf | plot data using tricontourf visualization
|  Tripcolor | plot data using tripcolor visualization
|  Triplot | plot data using triplot visualization
3D |  3D Scatterplot | plot data using 3D scatterplot visualization
|  3D Surface | plot data using 3D surface visualization
|  Triangular 3D Surface | plot data using triangular 3D surface visualization
|  3D Voxel , Volumetric Plot | plot data using 3D voxel , volumetric plot visualization
|  3D Wireframe Plot | plot data using 3D wireframe plot visualization

###  IV-A Recommendations of Code Cells Containing Matplotlib Plots

For the recommendation of the code cells containing Matplotlib plots, we first
studied the available plot types from the Matplotlib website. This is followed
by creating a template query text as a representative of the markdown cell for
all the plot types, which is shown in Table I. The next step is to start
querying each suggested code and determining whether the suggested code is
related to the query chart type.

Before starting the evaluation process, the codebase data of both BM25 and
UniXcoder were prepared and organized into three groups based on their
corresponding ranks of the notebooks’ authors in Kaggle. The numbers of code-
markdown pairs in each rank group were as follows: 2,670 pairs for Rank
Grandmaster, 4,098 pairs for Rank Master, and 10,855 pairs for Rank Expert.
These numbers are approximate because some pairs were discarded during the
indexing operation in the Machine Learning model. The data in the codebase was
indexed from the KGTorrent dataset by mapping raw markdown-code cells and
filtering for only code-markdown pairs related to plotting and charts to
reduce the scope of the evaluation.

The third author who has a background in Data Science is tasked with comparing
the suggested code with the query chart type. If the suggested code is related
to the query chart type, then the recommendation is marked as correct. On the
other hand, if it is not related, the recommendation is marked as incorrect.
This process is carried out for all the plot types on the Matplotlib website.
Finally, after completing querying recommendations for all plot types, the
results are used for evaluating the accuracy of recommendations.

The result from the evaluation is shown in Table II, where it included 14
chart types collected from the Matplotlib library
555https://matplotlib.org/stable/plot_types/indexwebsites. Table II contains
the Plot Type column, 3 rank-level columns; Grand Master, Master, and Expert,
and 2 sub-types for model columns; UniXcoder and BM25. At the Grand Master
rank, UniXcoder has the capability to suggest 6 code snippets that are related
to the charts and 3 code snippets for BM25. As for the Master rank, UniXcoder
can recommend 7 related code snippets, and the BM25 approach can recommend for
4 recommendations. Finally, for the Expert rank, UniXcoder can recommend 9
related code snippets and 7 related code snippets for the BM25 approach.

TABLE II: Typhon’s first recommendation contains the code cells related to the given visualization type. | Grand Master | Master | Expert
---|---|---|---
Plot Type | UniXcoder | BM25 | UniXcoder | BM25 | UniXcoder | BM25
scatter | ✓ | ✓ | ✓ | ✓ | ✓ | ✓
bar | ✓ |  | ✓ | ✓ | ✓ | ✓
step | ✓ |  |  |  | ✓ | ✓
imshow | ✓ |  | ✓ |  |  | ✓
contour |  |  |  |  | ✓ |
hist |  |  |  |  |  | ✓
boxplot |  | ✓ |  |  | ✓ | ✓
errorbar |  |  | ✓ |  |  |
violinplot |  |  |  | ✓ | ✓ |
pie | ✓ |  | ✓ | ✓ |  |
tripcolor | ✓ |  |  |  |  |
3d scatterplot |  | ✓ | ✓ |  | ✓ | ✓
3d surface |  |  | ✓ |  | ✓ |
triangular 3d surface |  |  |  |  | ✓ |
Total Correct | 6 | 3 | 7 | 4 | 9 | 7

###  IV-B Sanity check

TABLE III: Sanity check result Rank | Type | Total Items | Total Correct | Total Correct (%)
---|---|---|---|---
Grand Master | UniXCoder | 2,517 | 254 | 10.01
BM25 | 2,399 | 95.31
BM25 + stemming and lemmatization | 2,132 | 84.72
Master | UniXCoder | 3,744 | 377 | 10.07
BM25 | 3,391 | 90.57
BM25 + stemming and lemmatization | 3,007 | 80.32
Expert | UniXCoder | 9,553 | 605 | 6.33
BM25 | 8,644 | 90.48
BM25 + stemming and lemmatization | 7,193 | 75.30

We also performed a sanity-checking test aiming to make sure that Typhon can
provide recommendation functionality, at the very least, as correct as it
should be. The steps are in the following order. First, each code-markdown
pair in the code base is used to query for recommendations from the same
Kaggle dataset. Then, those recommended code cells for UniXCoder, BM25, and
BM25 with the pre-processed query using stemming and lemmatization are
compared with the original code from the query code-markdown pair.

During the sanity check, the data used for both the databases and queries were
the same, therefore they were identical. We scope down the data in this sanity
check. Again, this data consisted of code-markdown pairs specifically related
to plotting and charts. There were 2,670 code-markdown pairs in Rank
Grandmaster, 4,098 pairs in Rank Master, and 10,855 pairs in Rank Expert. The
markdown in each code-markdown pair was used as the query.

Our assumption expects that the recommendation result should return the exact
copy of the original code in the query code-markdown pair. The sanity-check
result is shown in the Table III. The result comprises 3 primary ranks of the
notebook authors in Kaggle: Grand Master, Master, and Expert. Each level
contains 3 types of the recommendation model approach available for use in the
Typhon tool, with the Total Correct, Total Correct (%), and Total Items
columns, explaining the total number of correct recommendations, the total
number of items in the codebase, and the percentage of correct items.

We analyze the result according to the rank of the Kaggle notebook’s authors.

  1. 1.

In the Grand Master rank, UniXCoder has the least correct recommendation, with
254 items or 10.01 percent. The correct recommendation is the BM25 model, with
2,399 items, or 95.31 percent. In comparison, BM25 with stemming and
lemmatization has the correct recommendation of 2,132 items or 84.72 percent.

  2. 2.

In the Master rank, the ranking is the same; UniXcoder has the least correct
recommendation items, the BM25 model is the highest, and BM25 has a pre-
processed query in the middle. Where the UniXCoder has 377 items or 10.07
percent, BM25 has 3391 items or 90.57 percent, and BM25 with pre-processed
query has 3,007 items or 80.32 percent.

  3. 3.

In the Expert rank, the rank is also the same. UniXCoder has the least correct
recommendation items, the BM25 model the highest, and BM25 with the pre-
processed query in the middle. UniXcoder has 605 items or 6.33 percent,
Elasticsearch has 8,644 items or 90.48 percent, and Elasticsearch with pre-
processed query has 7,193 items or 75.30 percent.

The low recommendation score in the UniXCoder could be because UniXcoder is a
cross-modal model that contains only code cell data in the codebase and
returns recommendations by processing a natural language query with the code
in the codebase, unlike Elasticsearch, which has pairs of markdown cells and
code cells for cross-referencing. On the contrary, the score in BM25 and BM25
with pre-processed queries using stemming and lemmatization is significantly
higher than BM25 and UniXcoder. BM25 suggests the code by comparing the
natural language in the query with the natural language in the markdown in the
objects stored in the Elasticsearch codebase and pre-processing helps increase
the accuracy.

We also performed a further investigation of the BM25 recommendations that
were wrong. The authors found that there were several issues that made the
BM25 approach recommend different codes, even though the associated markdown
was used for querying already existed in the codebase. The following section
discusses the topics found in the problem.

  1. 1.

General used words: The query markdowns are not scoped into any particular
topic or context. These markdowns are short in length and convey very broad
meaning. Examples of these markdowns are “Other” and “Worldwide”.

  2. 2.

Few keywords or short phrase markdown: The query markdown consists of cases
where the markdown used for querying contains only a few keywords or short
phrases. While the markdowns in the recommendation are often longer and
contain more of the keywords from the original markdowns.

  3. 3.

Reused and duplicated markdowns: There are many cases in which the author
reuses and replicates the same markdown multiple times, both in the same
notebook and in different notebooks. There are also cases where different
authors use the same markdown in their own notebooks, which contain different
codes. Often, the markdown in such cases is short or has few keywords.

  4. 4.

General inaccurate recommendation: Various common inaccurate results can be
noticed from the results, in which there were cases where the query markdown
and recommendation markdown shared the same keywords that are often unique
among notebooks, or significantly common that can be found in any notebook.

The investigation clearly shows that clear and detailed instructions in the
markdown can directly improve code recommendation quality and improve the
likelihood of generating relevant and accurate suggestions.

However, while the BM25 recommendation approach provides valuable insights and
potential code recommendations, it is important to note that the algorithm may
not always capture the specific context or subtle requirement in the markdown
query. Human experts are still required to manually intervene to verify and
fine-tune the recommended code to ensure that actually relevant recommended
codes are selected from the recommendation. To enhance the BM25 approach and
address the observed issues, further research and refinement of the
recommendation algorithm, as well as incorporating additional contextual
information from the markdown, could potentially lead to more accurate and
reliable code suggestions. We leave this as future work.

##  V Related Work

According to the study of Ritta et al. [10], they investigated if code reuse
occurs in Jupyter Notebooks of Kaggle Competition. They selected three
competitors that have high-level skills in writing and programming in Jupyter
Notebooks. After that, they extracted all notebooks from all the competitions
of three competitors. The code cells extracted from each competitor’s latest
competitions will be used as queries to find the reused code cells via
NCDSearch666https://github.com/takashi-ishio/NCDSearch. NCDSearch is a grep-
like tool. One command of a Unix-like operating system named grep is used for
searching plain text from lines that match a regular expression in files.
Thus, NCDSearch is created from the grep concept to find similar source code
fragments in specific files. Any code cell with a similarity distance from 0
to 0.5 was counted as reused code. They also classified and found the common
type of reused code to see which type is the most code that is reused. They
discovered that competitors might have different most reused code types, but
the most common type that everyone reused is Import Package, which is code
used for importing Python’s packages. Additionally, the authors suggest that
their work has the potential to be fundamental for the automatic code-
recommending tool because code recommendation can come from reusing the
existing codes from open-source projects.

Aroma is a source code recommendation tool that is implemented with a code-to-
code search concept from Luan et al. [1]. The core idea of Aroma is
recommending any possible complete and working or extended code snippets,
having the input as partially written source code. Aroma’s purposes are to
address the extensibility opportunity of the existing code snippet, such as
error handling parts and setup of the code, and comparison with the common
pattern other developers have made. It takes a code snippet as input, and then
Aroma will index the given corpus of code and retrieve a small set of code
snippets, where the retrieved code snippet will approximately contain the
input code snippet. Then Aroma will take these retrieved snippets through a
series of pruning, ranking, and clustering. After clustering, Aroma filters
these clustered snippets with intersecting operations to derive a set of the
most common snippets among the retrieved snippets. These intersected snippets
are recommended to the user as a set of alternative extended code snippets.
Although Aroma shares the same concept of recommending source code, the input
of Aroma is taken as a snippet of code, whereas our paper focuses on the input
as markdown cells.

Senatus is a code-to-code recommendation engine for developers that aims to
enhance their productivity, efficiency, and the reliability and consistency of
the code. The team builds Senatus by following the definition given by Luan et
al. [1]. The authors propose a new method of code recommendation by indexing
the code repository with Minhash-Locality Sensitive Hashing or Minhas-LSH.
Minhash is generally used for maintaining the Jaccard similarity of sets by
representing those sets as a lower dimensional similarity and preserving
features of their vectors. This technique provides much better efficiency at
the query time. Minhash can be applied in many applications such as web
search, graph sampling, earthquake detection, and efficient clustering of a
very large collection of DNA sequences. However, by using Minhash in the code-
to-code recommendations, they provide poor retrieval quality because of the
length of skewness in the code snippet. Senatus fixes this issue by operating
directly on the abstract syntax tree (AST) structure representation of the
code which decreases the time to query and the effectiveness of the retrieval.

Strathcona:  Another benefit of code recommendation is provided by the
Strathcona Example Recommendation Tool from Holmes et al. [3], a tool that
helps developers understand and use API. Incomplete or outdated documentation
about API usage causes problems for developers, so the tool aims to reduce the
gap between source code and documentation. In the case of understanding API
usage, developers must spend a lot of time manually searching the relevant
source code showing how to use the API. With the example recommendation tool,
developers can write code and get recommended examples for API usage. The
authors of the work intended to make the developers use less effort to get
examples. The code written by developers will be a query for searching. Then
the tool will locate structurally relevant examples by comparing the context
of the structure between the query and the existing repository. Moreover,
after the tool finds the relevant examples and returns them, developers are
able to navigate the example to inspect it in more detail as desired. Returned
examples can be navigated visually and textually.

Example Overflow:  Using the idea of social media or wisdom of crowd
recommendations, Zagalsky et al. [4] propose Example Overflow, a code search,
and recommendation tool. Example Overflow is a generic recommendation system
that utilizes the code snippet and textual data on Stack Overflow. The
approach’s core idea is to retrieve the relevant code snippets based on human
evaluation. Other existing search tools such as Strathcona, Aroma, and Senatus
recommend relevant code snippets using the machine processing power to compute
the rank and similarity. On the other hand, Example Overflow leverages the
crowd wisdom from answers to questions from the Stack Overflow website.
Intuitively, many programmers of any experience often learn and code by
following examples of code snippets in the tool documentation or code examples
on the Internet. Therefore, Example Overflow offers two core practices by the
system. The first practice is comparing multiple examples simultaneously
without switching developing context, either by switching the browser tabs or
searching for code examples. The system aims to serve this key point by
presenting the top five most relevant results, all in the same view. In the
second practice, the system takes minimal context switching, according to the
first practice, where the programmers are presented with the top five results.

There are also some of the existing work in text-to-code recommendation tools
and techniques as discussed below.

CodeSearchNet is a collaboration between GitHub and the Deep Program
Understanding group at Microsoft Research [11]. It is a dataset and benchmark
for code retrieval techniques. The goal of CodeSearchNet is to allow
developers and researchers to develop and improve code retrieval techniques
which are essential for improving software development productivity and
quality. CodeSearchNet dataset is collected from publicly available open-
source repositories on GitHub. The dataset includes over 6 million methods
consisting of GO, Java, JavaScript, Python, PHP, and Ruby programming
languages. Of these, 2 million methods have associated documentation. This
results in a set of pairs between the method of code and its documentation.
CodeSearchNet also provides a benchmark framework to measure and compare the
performance of different code retrieval techniques. Its purpose is to
encourage researchers and practitioners to further study semantic code search
tasks, which involve retrieving relevant code given a natural language query.
The process is generally as follows: 99 queries from the natural language with
its corresponding possible programming language results in each considered
language (Go, Java, JavaScript, PHP, Python, and Ruby). Then, these
query/results pairs are labeled by human experts to indicate the relevance of
the result and the query. Finally, the experts annotate the result for
relevance on a zero to three scale. The framework supports exact match
retrieval, semantic retrieval, and cross-lingual retrieval [11]. On the
leaderboard of the CodeSearchNet benchmark 4 out of the top 5 results use the
NeuralBoW (Neural Bag-of-Words). The basic idea behind NeuralBoW is to first
represent each word in a document using a one-hot encoding to create a vector
of zeros except for a single 1 in the position corresponding to the index of
the word in a pre-defined vocabulary. These vectors are then averaged together
to produce a fixed-length vector representation of the entire document. This
document vector can then be used as input to a neural network classifier to
predict the document’s class label.

We compare our Typhon approach to existing code-to-code and text-to-code
recommendation techniques as shown in Table IV. The seven techniques,
including Typhon, have different ways of recommending code. Copilot and
Tabnine use machine-learning techniques to generate new code from scratch. The
other five techniques; Aroma, Strathcona, Senatus, ExampleOverflow, and
Typhon; are based on searching from existing code bases by using the other
code snippets or search keywords as queries. Aroma recommends clustering and
ranking code snippets categories, Strathcona recommends from the similarity
between the user’s IDE structural details and code structural detail in
repositories, Senatus recommends from the similarity of input description and
database using Jaccard similarity, Example Overflow recommends from fine-tuned
TF-IDF weight algorithm, and Typhon suggests code cells using BM25 and
UniXcoder.

TABLE IV: Comparison of Typhon to Previous Work Name |  Approach
---|---
Typhon |  Search and recommend code cells based on the similarity of markdown text cells using BM25 or Machine Learning.
Copilot |  Generate code cells using the OpenAI Codex model trained on large open-source projects in GitHub
Tabnine |  Generate code snippets based on AI-based proprietary algorithm
Aroma |  Search and recommend code snippets based on the clustered and ranked code snippet categories
Strathcona |  Search and recommend code snippets based on the user’s IDE structural details with relevant to structural details in code repositories
Senatus |  Search and recommend code snippets based on Jaccard similarity using Minhash-LSH
ExampleOverflow |  Search and recommend code snippets by keyword search using fine-tuned TF-IDF weight

##  VI Threats to Validity

Internal Validity: The number of code-markdown pairs in the codebase is
inconsistent due to the limited amount of available data in the dataset and
filtering process. Where the Expert rank has the most data, with approximately
10,800 pairs of code and markdown; the Master rank with approximately 4,090
pairs; and the Grand Master rank with approximately 2,660 pairs. Therefore,
the present imbalance could explain the noticeably slightly higher accuracy in
Master and Expert, where they can potentially recommend a more variety of code
due to more available data. Furthermore, the charts presented in Table II show
that there are some charts that rarely appeared in the recommendation results.
Additionally, considering the nature of the dataset, which is extracted from
Kaggle competition, there is a high possibility that the codebase contains
some group of visualizations significantly more than other groups of
visualization. Therefore, it explains why some charts appear more often than
others appear in the recommendation. This may affect the validity of the
results.

External Validity: The proposed approach works with only the data from Kaggle
and may not be generalized to Jupyter notebooks on other platforms. Moreover,
the similarity computations are based only on BM25 and UniXcoder on English
text in the markdown and Python code which may not be generalized to other
techniques and languages.

##  VII Conclusion and Future Work

Code recommendation tools have become a big part of many software developers.
Tools like GitHub Copilot and ChatGPT have become increasingly favored both in
and outside the industry. Our proposed approach, Typhon, recommends similar
code cells in Jupyter notebooks using BM25 (Information Retrieval Approach)
and UniXcoder (Machine Learning Approach). Typhon is designed to improve the
workflow of those in the Data Science field, primarily on Jupyer Notebook.

The evaluation results show that Typhon can recommend code cells involving
visualization using Matplotlib with moderate accuracy. UniXcoder can recommend
more correct code cells than BM25. Nonetheless, our sanity check shows that
UniXcoder has lower precision than BM25.

In future work, we plan to compare the performance of Typhon to other
approaches such as Copilot. M oreover, we plan to improve the accuracy of the
recommendation by increasing the number of markdown cells that are used to
query for similar code cells and study its effect on the recommendation
accuracy. We also plan to explore other code embedding techniques besides
UniXcoder.

## Acknowledgement

This work was financially supported by the Office of the Permanent Secretary,
Ministry of Higher Education, Science, Research and Innovation (OPS MHESI)
Grant No. RGNS 64-143, and the Faculty of Information and Communication
Technology, Mahidol University, Thailand.

## References

  * [1] S. Luan, D. Yang, C. Barnaby, K. Sen, and S. Chandra, “Aroma: Code recommendation via structural code search,” _Proceedings of the ACM on Programming Languages_ , vol. 3, 10 2019.
  * [2] F. Silavong, S. Moran, A. Georgiadis, R. Saphal, and R. Otter, “Senatus - a fast and accurate code-to-code recommendation engine,” in _MSR ’22_ , 2022, pp. 511–523.
  * [3] R. Holmes, R. J. Walker, and G. C. Murphy, “Strathcona example recommendation tool,” in _ESEC/FSE ’13_ , 2005, pp. 237–240.
  * [4] A. Zagalsky, O. Barzilay, and A. Yehudai, “Example overflow: Using social media for code recommendation,” in _RSSE ’12_ , 2012, pp. 38–42.
  * [5] AI Terms, “What is TabNine?” https://aiterms.net/tabnine/, online; accessed 4 November 2022.
  * [6] T. Kluyver, B. Ragan-Kelley, F. Pérez, B. Granger, M. Bussonnier, J. Frederic, K. Kelley, J. Hamrick, J. Grout, S. Corlay, P. Ivanov, D. Avila, S. Abdalla, and C. Willing, “Jupyter notebooks—a publishing format for reproducible computational workflows,” in _ELPUB ’16_ , 2016, pp. 87–90.
  * [7] L. Quaranta, F. Calefato, and F. Lanubile, “Kgtorrent: A dataset of python jupyter notebooks from kaggle,” in _MSR ’21_ , 2021, pp. 550–554.
  * [8] Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, and M. Zhou, “CodeBERT: A Pre-Trained Model for Programming and Natural Languages,” _EMNLP ’20_ , pp. 1536–1547, Feb 2020.
  * [9] D. Guo, S. Lu, N. Duan, Y. Wang, M. Zhou, and J. Yin, “Unixcoder: Unified cross-modal pre-training for code representation,” _arXiv_ , 3 2022. [Online]. Available: http://arxiv.org/abs/2203.03850
  * [10] N. Ritta, T. Settewong, R. G. Kula, C. Ragkhitwetsagul, T. Sunetnanta, and K. Matsumoto, “Reusing My Own Code: Preliminary Results for Competitive Coding in Jupyter Notebooks,” in _APSEC ’22_ , 2022.
  * [11] H. Husain, H.-H. Wu, T. Gazit, M. Allamanis, and M. Brockschmidt, “CodeSearchNet Challenge: Evaluating the State of Semantic Code Search,” _arXiv_ , 9 2019. [Online]. Available: http://arxiv.org/abs/1909.09436


# Understanding the Humidity Sensitivity of Sensors with TCAD Simulations

I-S. Ninca  I. Bloch  B. Brüers  V. Fadeyev  J. Fernandez-Tejero  C. Jessiman
J. Keller  C. T. Klein  T. Koffas  H. M. Lacker  P. Li  C. Scharf  E. Staats
M. Ullan  Y. Unno  DESY Zeuthen, Platanenallee 6, 15738 Zeuthen, Germany
Santa Cruz Institute for Particle Physics (SCIPP), University of California,
Santa Cruz, CA 95064, USA  Department of Physics, Simon Fraser University,
8888 University Drive, Burnaby, B.C. V5A 1S6, Canada  TRIUMF, 4004 Wesbrook
Mall, Vancouver V6T 2A3, BC, Canada  Physics Department, Carleton University,
1125 Colonel By Drive, Ottawa, Ontario, K1S 5B6, Canada  Institut für Physik,
Humboldt-Universität zu Berlin, Newtonstraße 15, 12489 Berlin, Germany
Instituto de Microelectrónica de Barcelona (IMB-CNM), CSIC, Campus UAB-
Bellaterra, 08193 Barcelona, Spain  Institute of Particle and Nuclear Study,
High Energy Accelerator Research Organization (KEK), 1-1 Oho, Tsukuba, Ibaraki
305-0801, Japan

###### Abstract

The breakdown voltage of silicon sensors without special surface is known to
be affected by the ambient humidity. To understand the sensor’s humidity
sensitivity, Synopsys TCAD was used to simulate n-in-p test structures for
different effective relative humidity. Photon emission of hot electrons was
imaged with a microscope to locate breakdown in the edge-region of the sensor.
The Top-Transient Current Technique was used to measure charge transport near
the surface in the breakdown region of the sensor. Using the measurements and
simulations, the evolution of the electric field, carrier densities and
avalanche breakdown in the periphery of p-bulk silicon sensors is presented.

###### keywords:

ATLAS Experiment , Silicon Sensors , TCAD Simulations , Top-TCT

††journal: Nuclear Instruments and Methods in Physics Research Section A

##  1 Introduction

The ATLAS ITk will be upgraded to sustain the harsh radiation levels due to
the increase in luminosity foreseen at the High-Luminosity Large Hadron
Collider (HL-LHC). In the prototyping phase for the new ATLAS ITk Strip
detector, silicon sensors showed electrical breakdown at lower bias voltages
when exposed to higher humidity [1]111To mitigate this issue the sensors are
stored in a dry environment. Once installed, the ITk will be in dry
environment and humidity sensitivity will not be an issue during ATLAS
operation..

The mechanisms that cause the early breakdown of sensors in humid conditions
are not yet fully understood. To gain insight into the physical processes that
trigger the early avalanche breakdown of sensors due to exposure to humidity,
charge transport in the guard-ring region of the test structure was
investigated by generating localized free charge carriers near the surface
with picosecond pulses of laser light using the Top-Transient Current
Technique (TCT). The electrical behavior of test structures was simulated
using TCAD from Synopsis [2] and the simulations were crosschecked with the
experimental data.

Figure 1: Microscope picture of an n-in-p silicon diode operated in avalanche
breakdown. Photon emission from hot electrons in the breakdown region is
visible as a bright spot in the bottom of the image. The measurement was
performed at an RH of $\approx$50 %. The breakdown current between the
backside and the GR was $\mathrm{\mathcal{O}(1)~{}$\mathrm{\SIUnitSymbolMicro
A}$}$.

##  2 Experimental Methods

###  2.1 Breakdown imaging

A consumer CMOS camera sensitive in the near infrared spectrum was used to
locate avalanche breakdown in the edge region of the test structure. Fig. 1
shows an example $4\times 4$ mm2 n-in-p diode, biased at
$\mathrm{-~{}750~{}V}$ in electrical breakdown. The outermost aluminum ring is
called the edge ring (ER), then there is the guard ring (GR) and finally the
pad metal. The bias voltage was applied on backside, which has a conductive
coupling to the ER via the undepleted region along the dicing edge. The GR and
pad were kept at ground. The bright spot is Bremsstrahlung from hot electrons
[3, 4] accelerated in a high electric field between the ER and the GR.

###  2.2 Device under test

For this study, an $\mathrm{8~{}mm}$ by $\mathrm{8~{}mm}$, n-in-p diode called
the monitor diode (MD8) was investigated. This type of diode comes from the
same ATLAS18 ITk wafer as the strip sensor [5]. The MD8’s geometry is similar
to the diode illustrated in Fig. 1 but it has a special feature of an
additional p-stop between the GR and the pad metal. The active thickness
calculated from capacitance-voltage measurements [6] is $\mathrm{295~{}\upmu
m}$ with a full depletion voltage of $\approx-280$ V and the effective p-bulk
doping is $\approx 4.2\cdot 10^{12}$cm -3.

###  2.3 TCAD geometry implementation

Structure Device Editor (SDE) was used to implement the MD8 geometry for the
TCAD simulations using representative parameters. The cross-section of the
test structure is shown in Fig. 2 with a focus on the top edge of the diode.
There is a p-implant underneath the ER, while the GR and pad have n-implants.
The doping concentration of both the p- and n-implants is
$\mathrm{10^{\text{19}}~{}cm^{-3}}$. There is a $\mathrm{0.6~{}\upmu
m~{}SiO_{2}}$ layer in direct contact with the silicon that goes beneath the
metal overhangs. A $\mathrm{0.6~{}\upmu m~{}Si_{3}N_{4}}$ layer goes on top of
the $\mathrm{SiO_{2}}$ and the electrodes. The concentration of fixed oxide
charges at the interface between the silicon and $\mathrm{SiO_{2}}$ is
$\mathrm{\approx~{}10^{11}~{}cm^{-2}}$ [7]. The values for the parameters
implemented are listed in table 1.

Table 1: TCAD Parameters Values Parameter | Value | Unit
---|---|---
Si p-bulk thickness | $295$ |  $\upmu$m
Si p-bulk doping | $4.2\cdot 10^{12}$ | cm-3
p- and n-implant doping | $10^{19}$ | cm-3
p-stop doping | $10^{16}$ | cm-3
Fixed oxide charge | $10^{11}$ | cm-2
SiO2/Si3N4 thicknesses | $0.6$ |  $\upmu$m

To model the humidity effects on the surface of the test structure, a 0.1
$\upmu$m polysilicon layer was added on top of the passivation which acts as a
resistive layer. It is directly connected to the GR and pad, but it is not in
direct contact with the ER since the ER only has passivation openings in the
corners of the diode. In the presence of humidity, the sheet resistance
$R_{\square}$ of the passivation surface changes. To simulate a specific RH
value, $R_{\square}$ was adjusted based on literature values [8]. For
$\mathrm{RH~{}=~{}40~{}\%}$, $R_{\square}$ of the passivation surface of a
similar test structure was measured [8] to be $\mathrm{9.7\cdot
10^{15}~{}\Omega}$. To set $R_{\square}$ of the polysilicon layer in TCAD to
this value, the mobility of both electrons and holes in the polysilicon was
set to $\upmu_{e,h}=6.7\cdot 10^{-3}$ cm2/Vs.

Figure 2: View of the diode geometry used in TCAD. The backside p-implant (not
shown) is at $Y=297.2\,\upmu$m and extends along $X$.

The carrier mobilities in silicon were described by the Canali model [9] for
the high-field saturation and the University of Bologna model [10] for the
doping dependence. Shockley–Read–Hall (SRH) recombination with the lifetime
$\tau=10^{-6}$ s and Auger recombination were used. The doping dependence of
$\tau$ was described using the Scharfetter relation. Charge carrier
multiplication was simulated using the van Overstraeten-de Man model [11],
which relies on the Chynoweth law [12].

The evolution of the electric field, impact ionization, and the carrier
densities over time was simulated as a transient simulation and investigated
for several time stamps.

###  2.4 Top-TCT

The Top-TCT method was used to study the charge transport in the region
between the ER and the pad of the MD8 diode. Laser pulses of
$\mathrm{660~{}nm}$ wavelength were focused at the sensor’s surface with a
full width at half maximum of the laser beam $FWHM=7.33\,\upmu$m$\pm
0.16\,\upmu$m. The laser intensity was $\mathrm{2550~{}mV}$ corresponding to a
few MIPs. The backside was connected to high voltage (HV) and read out via a
Bias-T. The ER was also at HV through a conductive channel at the dicing edge
to the backside. The GR and pad were connected to ground222Normally, the GR is
floating to maximize the breakdown voltage. as depicted in Fig. 3.

Figure 3: Top-TCT sketch showing electron-hole pair generation by a
$\mathrm{660~{}nm}$ laser pulse between the ER and the GR. During a TCT
measurement, the laser beam is moved along the surface of the diode and the
induced transients are recorded for each position.

The laser was moved across the diode’s surface and the resulting transient
current from the drifting charge carriers was recorded at the backside, which
has a conductive connection to the edge ring. A Rohde and Schwarz RTO1024
oscilloscope with a sampling rate of $\mathrm{10~{}Gsample/s}$ and a bandwidth
of $\mathrm{2~{}GHz}$ was used to acquire the current transients averaged over
1000 laser pulses. The humidity inside the TCT set-up was monitored using a
humidity sensor placed in the vicinity of the diode. The humidity was
controlled by flushing the TCT set-up with dry air until the desired RH value
was reached. During the measurements RH was stable within
$\mathrm{\pm~{}1~{}\%}$.

Prior to TCT scans, the MD8 diodes were measured in a humidity-controlled
probe station. For each RH value, current-voltage (IV) measurements were
performed to determine a possible breakdown voltage. However, none of the
available MD8 diodes broke down and hot-electron emission could not be
recorded for those diodes yet. Previous studies have shown that small
structures like mini sensors or diodes show less humidity sensitivity than
large ones, such as the ATLAS ITk strip sensors. After each IV is taken, Top-
TCT measurements were performed for the exact same RH value. The humidity was
ramped up throughout the tests, starting at $\mathrm{RH~{}=~{}10~{}\%}$ to
avoid hysteresis effects which can occur when the sensor is biased at high
humidity and the humidity is subsequently lowered, freezing a possible charge
at the surface [13]. The results shown are from a single diode, MD8 32418-14.
For all experimental results, the applied bias voltage was
$\mathrm{-~{}1000~{}V}$.

The transient current produced by the drift of electrons and holes is given
by:

$I(t)\ =\ e_{0}\cdot
N_{e,h}(t)\cdot\upmu_{e,h}\cdot(\vec{E}(\vec{r}(t)))\cdot\vec{E}(\vec{r}(t))\cdot\vec{E_{w}}(\vec{r}(t))$
(1)

where $\mathrm{e_{0}}$ is the elementary charge, $N_{e,h}(t)$ is the number of
electron-hole pairs produced, $\upmu_{e,h}(\vec{E}(\vec{r}(t)))$ is the
mobility, $\vec{E}(\vec{r}(t))$ is the electric field and
$\vec{E}_{w}(\vec{r}(t))$ is the weighting field.

##  3 Top-TCT Results

Fig. 4 shows the transient currents for RH$=40\,\%$ and multiple positions
along a straight line between the ER and the pad metal. It should be noted
that the $X$ positions are not aligned and differ in the plots shown in this
section. By integrating the transient currents for 43 ns $\leq t\leq$ 80 ns,
the total collected charge can be obtained as a function of the laser
position, shown in the charge profiles in Fig. 5. Between the ER and GR, the
charge profile decreases with increasing the RH. At the same time, the
collected charge between the GR and the pad does not change. The high
collected charge between the ER and the GR for low humidity is an indication
for charge multiplication in the high field at the edge of the GR. In order to
test this hypothesis we have measured the collected charge with different
laser intensities. The efficiency of charge multiplication strongly depends on
the laser intensity [14] as a high concentration of free charge carriers will
shield the peak electric field. This can be observed in Fig. 6, where the
signal was normalized relative to the values in the pad window. The collected
charge near the outer edge of the GR increases with decreasing laser
intensity, as expected for charge multiplication in this region.

Figure 4: Measured transient currents at different positions in the edge-
region of the diode for 40 % RH and $V_{bias}=-1000$ V. The position at
$X=3650\,\upmu$m is close to the ER while the position at $X=4010\,\upmu$m is
close to the pad. The position of the GR is around $X=3900\,\upmu$m. Figure 5:
Charge profiles between the ER and pad metalization for different values of RH
and $V_{bias}=-1000$ V.  Figure 6: Normalized collected charge between the ER
and pad window at $\mathrm{RH~{}=~{}8~{}\%}$, $V_{bias}=-1000$ V and different
laser intensities (a larger DAC value corresponds to a lower intensity). The
collected charge was normalized to the charge collected in the pad window,
which starts at $X=4010\,\upmu$m, for each laser intensity.

Using a short integration window at the beginning of the current transient,
43,5 ns $\leq t\leq$ 43,9 ns, the prompt current profiles [15] are obtained as
shown in Fig.,7. The maximum of the prompt current is near the GR and it
decreases as the laser moves away from this electrode. Additionally, the
maximum amplitude of the prompt current experiences a decrease with increasing
the RH. These observations are an indication that there is a very high
electric field (Eq. 1) near the GR and the prompt current in this region might
be affected by charge multiplication.

Figure 7: Prompt currents between the ER and pad metalization for different RH
and $V_{bias}=-1000$ V.

##  4 TCAD Results

In the TCAD simulation, the bias voltage was ramped from $\mathrm{0~{}V}$ in
steps of 10 V/10 s. After $\mathrm{900~{}s}$, the bias voltage was kept
constant at $\mathrm{-~{}900~{}V}$. All simulations presented here were
performed at a RH of $\mathrm{40~{}\%}$. Fig. 8 shows the simulated reverse
current which still increases when the bias voltage is kept constant
$\mathrm{-~{}900~{}V}$. It is expected that the time constant of this
evolution depends strongly on the $R_{\square}$ [13, 8]. Therefore, the
simulations were so far only performed for a relatively high RH and cannot be
compared to the measurements directly. However, since the measurements were
performed first at low RH and the RH was subsequently increased with each
measurement, effectively decreasing the time constant, it is expected that the
measurements at low RH should be comparable to the simulation right after
ramping and the measurements at high RH should be comparable to the simulation
after waiting some time at constant bias.

The outcome was analysed in more details at four points in time. The first
time stamp, at $\mathrm{900~{}s}$, coincides with reaching the final bias
voltage of $\mathrm{-~{}900~{}V}$. The second time stamp, at
$\mathrm{2500~{}s}$, was chosen as an intermediate and the third one, at
$\mathrm{4500~{}s}$, after the leakage current started increasing slowly while
the last time stamp, at $\mathrm{6500~{}s}$, was chosen after the current
started rising rapidly.

Figure 8: Simulated GR current for $\mathrm{RH~{}=~{}40~{}\%}$ and
$V_{const.}=-900$ V for $t>900$ s. Soft breakdown starts at $t=4372$ s. The
arrows point to the four time stamps chosen to highlight changes in the
simulation.

The electrostatic potential for $t=900$ s is illustrated in Fig. 9. The bias
voltage was applied on the backside and on the ER, while the GR and the pad
were grounded. The biasing scheme is the same as for the Top-TCT measurements.

Figure 9: Simulated electrostatic potential for $\mathrm{RH~{}=~{}40~{}\%}$
and $V_{bias}=-900$ V. The electrostatic potential was extracted immediately
after the ramping was finished at $t=900$ s.

The absolute electric field shown in Fig. 10 shows a high field peak near the
GR which coincides with the location of the maximum measured prompt current in
Fig. 7. The TCAD simulation predicts an additional high field peak near the ER
which is not visible in the measurement. Once the leakage current starts
increasing slowly at $\mathrm{t~{}=~{}4372~{}s}$ (soft breakdown) the high
field at the ER increases. At $\mathrm{t~{}=~{}6500~{}s}$, where the leakage
current starts rising drastically, new high field regions appear near the ER
and GR implants (see Fig. 11). The origin of these maxima and the effect on
the breakdown need to be investigated further, but it appears these regions
are generating a large amount of free charge carriers by impact ionization.

Figure 10: Simulated electric field for $\mathrm{RH~{}=~{}40~{}\%}$,
$V_{bias}=-900$ V and different time stamps. Figure 11: Absolute electric
field along $X$ axis $100$ nm under the oxide.

The electron density is shown in Fig. 12. Due to the fixed positive oxide
charge a conducting electron inversion layer is present at the Si-SiO2
interface before a bias voltage is applied. The electron inversion layer is
dispersed by the electric field when the interface is depleted and the
remaining electron concentration at the interface is low after ramping to
$\mathrm{-~{}900~{}V}$, as seen in Fig. 12a. The increase of the current after
some time in humid conditions coincides with an increase of the electron
concentration in the inversion layer, as visible in Fig. 12c. It appears that
the free electrons generated by impact ionization in the high field region
developing at the ER metal edge (see Fig. 10c) replenish the inversion layer.
When the current starts to increase rapidly the electron concentration at the
interface increases even more (Fig. 12d) and free electrons are also generated
at the ER implant.

Figure 12: Simulated electron densities for $\mathrm{RH~{}=~{}40~{}\%}$,
$V_{bias}=-900$ V and different time stamps.

The evolution of the concentration of free holes in the bulk is comparable to
the evolution of the electron concentration, as shown in Fig. 13. A high hole
concentration is visible near the peaks of the electric field once the current
increases. Fig. 13a also shows a high density of holes present in the
polysilicon layer on top of the ER after the ramping of the device has
finished. It is important to note that the polysilicon layer is capacitively
coupled to the ER, while it is in direct electrical contact with the GR and
the pad. Over time, a high concentration of holes in the polysilicon layer
moves laterally towards the GR, visible at $X\approx 110\,\upmu$m in Fig. 13c.
It seems to be this positive charging up of the passivation surface that is
driving the breakdown. It should be noted that this charging up was not
observed in simulations for lower RH, which are not presented here.

Figure 13: Simulated hole densities for $\mathrm{RH~{}=~{}40~{}\%}$,
$V_{bias}=-900$ V and different time stamps.

Electron-hole pair production due to impact ionization requires a certain
electric field strength and is only effective in few regions. After ramping,
there is an impact ionisation region underneath the metal overhang of the GR
as illustrated in Fig. 14a. At $\mathrm{t~{}=~{}2500~{}s}$, Fig. 14b shows a
new impact ionization region developing underneath the ER overhang. Following
the slow increase of leakage current, the two impact ionization regions start
increasing as presented in Fig. 14c. Once the leakage current rises fast, new
impact ionization regions develop at the ER and GR implants as highlighted in
Fig. 14d.

Figure 14: Generation rate of free charge carriers from impact ionization for
$\mathrm{RH~{}=~{}40~{}\%}$, $V_{bias}=-900$ V and different time stamps. For
comparison, the bulk generation rate is $U=4.78\cdot 10^{15}$ cm-3s-1.

##  5 Summary and discussion

Top-TCT measurements and TCAD simulations of silicon sensors with a focus on
the guard ring region have been performed at bias values of 900 V and above to
study humidity-related breakdown. Electrical breakdown is observed in the TCAD
simulation at high humidity after waiting some time at a constant bias
voltage. In the sensor used for TCT measurements, no reproducible breakdown
stable enough to perform a long term measurement was observed.

The total collected charge in TCT measurements between the guard ring and the
edge was observed to decrease with rising relative humidity. The maximum
prompt current location observed in the measurement coincides with an electric
field peak observed in the TCAD simulation which is sufficiently high to
induce charge multiplication of electrons. The signal profile which is
observed in the TCT measurement at the edge of the GR at low RH (Fig. 5) can
be explained by screening of the charge multiplication [14], also seen in Fig.
6. For high RH less impact ionization is observed at the GR in the
measurements. This could be due to less efficient charge multiplications
because there are much more free carriers from the electron inversion layer
available. The free electrons are generated in a new high field region at the
ER in the simulation (Fig. 12c) and drift at the interface into the high field
region at the GR, where they can generate more free charge carriers. The
development of the new high field region at the ER seems to be caused by a
positive charging of the surface of the passivation at the ER. This appears to
be a possible cause for humidity-related avalanche breakdown in the guard ring
region for the p-bulk sensors at high bias investigated here, assuming the
resistive polysilicon layer on the passivation is directly coupled to the GR.

Further measurements in breakdown conditions as well as further simulation
studies on the influence of the coupling of the polysilicon layer and the
fixed oxide charge density are planned.

## Acknowledgments

The work at CNM is part of the Spanish R&D grant PID2021-126327OB-C22, funded
by MCIN/ AEI/10.13039/501100011033 / FEDER, UE. The work at SCIPP was
supported by the US Department of Energy, grant DE-SC0010107. The work at SFU,
TRIUMF and Carleton University was supported by the Canada Foundation for
Innovation and the Natural Sciences and Engineering Research Council of Canada

## Declaration of AI and AI-assisted technologies in the writing process

During the preparation of this work the author(s) used ChatGPT 3.5 in order to
improve the clarity of the language. After using this tool/service, the
author(s) reviewed and edited the content as needed and take(s) full
responsibility for the content of the publication.

## Copyright

Copyright 2024 CERN for the benefit of the ATLAS Collaboration. Reproduction
of this article or parts of it is allowed as specified in the CC-BY-4.0
license.

## References

  * [1] J. Fernandez-Tejero et al., Humidity sensitivity of large area silicon sensors: Study and implications, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 978 (2020) 164406.
  * [2] Synopsys, TCAD - Technology Computer Aided Design, https://www.synopsys.com/manufacturing/tcad.html.
  * [3] N. Akil et al., A multimechanism model for photon generation by silicon junctions in avalanche breakdown, IEEE Transactions on Electron Devices 46 (5) (1999) 1022–1028.
  * [4] J. Bude et al., Hot-carrier luminescence in Si, Physical Review B 45 (11) (1992) 5848.
  * [5] Y. Unno et al., Specifications and pre-production of n+-in-p large-format strip sensors fabricated in 6-inch silicon wafers, ATLAS18, for the Inner Tracker of the ATLAS Detector for High-Luminosity Large Hadron Collider, Journal of Instrumentation 18 (03) (2023) T03008.
  * [6] F. Riemer, Reverse current, capacitance and thermal runaway of irradiated silicon diodes, Master’s Thesis, Humboldt-Universität zu Berlin, https://bib-pubdb1.desy.de/record/471495 (2021).
  * [7] M. Ullán et al., Quality Assurance methodology for the ATLAS Inner Tracker strip sensor production, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 981 (2020) 164521.
  * [8] D. Brueske et al., Investigation of the insulator layers for segmented silicon sensors before and after X-ray irradiation, in: IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), 2014, pp. 1–5.  doi:10.1109/NSSMIC.2014.7431261.
  * [9] C. Canali et al., Electron and hole drift velocity measurements in silicon and their empirical relation to electric field and temperature, IEEE Transactions on Electron Devices 22 (11) (1975) 1045–1047.
  * [10] S. Reggiani et al., Electron and hole mobility in silicon at large operating temperatures. I. Bulk mobility, IEEE Transactions on Electron Devices 49 (3) (2002) 490–499.
  * [11] R. Van Overstraeten et al., Measurement of the ionization rates in diffused silicon pn junctions, Solid-State Electronics 13 (5) (1970) 583–608.
  * [12] A. Chynoweth, Ionization rates for electrons and holes in silicon, Physical Review 109 (5) (1958) 1537.
  * [13] W. Shockley et al., Mobile electric charges on insulating oxides with application to oxide covered silicon pn junctions, Surface Science 2 (1964) 277–287.
  * [14] E. Currás et al., Gain reduction mechanism observed in low gain avalanche diodes, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 1031 (2022) 166530.
  * [15] G. Kramberger et al., Investigation of irradiated silicon detectors by edge-TCT, IEEE Transactions on Nuclear Science 57 (4) (2010) 2294–2302.