# Turtle Score - Similarity Based Developer Analyzer

Sanjjushri Varshini
<EMAIL_ADDRESS>Ponshriharini V
<EMAIL_ADDRESS>Santhosh Kannan
<EMAIL_ADDRESS>Snekha Suresh
<EMAIL_ADDRESS>Harshavardhan Ramesh
<EMAIL_ADDRESS>Rohith Mahadevan
<EMAIL_ADDRESS>Raja CSP Raman
<EMAIL_ADDRESS>
###### Abstract

In day-to-day life, a highly demanding task for IT companies is to find the
right candidates who fit the companies’ culture. This research aims to
comprehend, analyze and automatically produce convincing outcomes to find a
candidate who perfectly fits right in the company. Data is examined and
collected for each employee who works in the IT domain focusing on their
performance measure. This is done based on various different categories which
bring versatility and a wide view of focus. To this data, learner analysis is
done using machine learning algorithms to obtain learner similarity and
developer similarity in order to recruit people with identical working
patterns. It’s been proven that the efficiency and capability of a particular
worker go higher when working with a person of a similar personality.
Therefore this will serve as a useful tool for recruiters who aim to recruit
people with high productivity. This is to say that the model designed will
render the best outcome possible with high accuracy and an immaculate
recommendation score.

Keywords: Machine Learning, IT companies, Recommendation, developer
similarity, learner similarity, learner analysis.

##  1 Introduction

Companies hire candidates with the traditional methods, new techniques that
each specific organization follows. But in most of the scenarios, there is an
issue of hiring candidates who do not fit the culture of the specified
organization. Only a third of companies report that they monitor whether their
hiring leads to a satisfactory result. But there is a greater percentage of
companies who do not follow up on their employees regarding their skills.

Moreover, it has become difficult to assess the quality of the candidate and
whether they would fit right in with the company culture. A lot of variables
and factors come into consideration regarding the quality of a candidate the
company wants to hire and these vary from different companies concerning
different job positions. Predominantly, the quality of a candidate depends
upon different factors such as the candidate’s perks, competency, CCI
(Continuous Capability Improvement), participatory culture, and so on. Also,
the hires are not only for entry-level jobs but across most levels. As the job
level increases, the requirements for the candidates the company demands also
increase respectively. This has brought in a new problem of ensuring the level
of a candidate is on par with the already employed. Also, there is a vast set
of competition between the candidates solely over the chances of going to a
higher rank in their company. In these types of scenarios, there comes a
comparison metric between the preceding candidates who are trying to move for
a higher rank.

To address these issues, this research provides a new metric system for the
companies to help in identifying the right set of candidates for employability
within their company. Data collected from various candidates is analyzed and a
recommendation system based on similarity is created. For this, parameters
like turtle score - a score based on the learning analytics score, GitHub
score, and error archive score are used. Kaggle’s score is also taken into
account for the machine learning candidates. Kaggle and Github are very
popular among most of the leading companies across the globe and these
companies take those scores into account as one of the candidate’s
requirements. The reason behind this is the scores that each candidate
possesses represent their technical skills over the set of domains they have
worked on. These scores act as a good metric to evaluate a candidate and
compare them with candidates with similar scores.

##  2 Literature Survey

[5] Employability is a major factor in the sustainable competitiveness within
the company as well for the personal growth of the candidate. As a result, a
question arises in the mind of everyone - Is the candidate chosen the best
option from the pool of choice? Companies have a hard time answering the said
question due to a lack of monitoring and bias in the hiring process itself.
[9] Implicit bias towards characteristics like race, gender, and sexual
orientation has always resulted in persistent inequality and reduces the
utility of the employer. So, is there any process that is free from the said
problems that can be both reliable and efficient? Various machine learning
models are analyzed to find an answer.

[8]In the modern competitive world, employers are looking into machine
learning models to help them fill their spot of vacancy.[2] One among the
models and the most prominent ones uses recommendation systems. A
recommendation system is a system that uses information filtering to predict
the preferences of a user.[6] A recommendation system does so by using the
previous history of the user or by finding similarities with other users. It
helps to create dynamic recommendations personalized to each user.
Collaborative filtering is the widely used method for creating a
recommendation system.[4] Let us consider two users A and B. If A and B are
interested in the same product, then their preferences for other products may
align more compared to any random user. In the modern data-driven world,
recommendation systems play a huge role in supporting the process of decision-
making.[11]An organization needs to hire candidates without any discrimination
based on gender, color, and many other factors. To overcome this using an ML
model will effectively solve this crisis and decrease the bias that human
beings cause. One of the major reasons to use the ML model in the hiring
methodology is that the model can be acclimated over time to adjust to
changing priorities. [7] The search for similar-minded people has been over
many decades but the application of Machine Learning makes this job simpler
and better. The hashtag-based clustering method with a combination of several
rounds of the k-means algorithm and the DBSCAN algorithm is used to form
clusters of data that belong to like-minded people. Chaotic data of millions
of people can be filtered out easily and working partners for each worker can
be assigned by the hirer thus increasing the efficiency of both the workers.

[12] Recruiters in the information technology domain have met the problem of
finding appropriate candidates by their skills. In the resume, the candidate
may describe one skill in different ways or skills could be replaced by
others. The recruiters may not have the domain knowledge to know if one’s
skills are fit or not, so they can only find ones with matched skills.

[1] Academic credentials include but are not restricted to diplomas, degrees,
certificates, and certifications, which act as a way to attest completion of
training or education undertaken by the student. Broadly speaking, these
credentials may also attest to the successful completion of any test or exam.
Ultimately, they serve as a model of independent validation of the said
individual’s possession of the knowledge, skill, and ability needed to carry
out a particular task or activity.

The most competent skills that are meant to be present among the candidates
that are proclaimed by larger companies are based on Kaggle, Github score,
etc., Information regarding one’s Kaggle or Github helps the company to
identify the level of skill-set of the candidate. [3] Kaggle is a platform
service for data analysis competition that was established in 2010.
Corporations and data scientists provide the data and challenges, and users
propose such things as analysis models.[10] Insights such as the quality of
contributions made by a developer on GitHub could provide an initial
understanding of a potential candidate’s coding capabilities. Considering the
rising popularity of social recruiting and the kind of potential GitHub
platform offers, GitHub can be leveraged for software developer recruitment.

##  3 Existing Model

###  3.1 Avrio

Avrio on its own has its own set of hiring processes. At first, they basically
use a messenger bot in their hiring process to communicate with their
candidates. This is sort of their pre-screening process for their interview.
They generate a complete profile from their profile analysis of those
candidates who submitted their resumes. Avrio has its own Fitscore for each
candidate to compare them based on various factors and the generated profile
will be run through every job offering database to see if there are any
matches. Also, they have their very own virtual assistant called Rio which is
responsible for the candidates’ personal profile as it is programmed to record
and present the analysis of the candidate by asking different sets of
questions that will reflect their skills, capability, and interest set for the
available job.

###  3.2 Ideal

Somen and Shaun are the co-founders of Ideal. The idea of this started off
from their own personal experience as they were struggling to make data-backed
hiring decisions in their decisions. The idea of them having a thought that
there might be a better idea never left off and now here they are with their
solution. They claim to have their own virtual assistant which directly
integrates with the existing clients’ applications. The assistant also trains
itself from their past hiring decisions from millions of candidates. They
adapt to two main processes. At glance, the first part of the process is
Resume screening which is by default performed manually. They don’t
specifically use any standardized criteria for every resume which is why this
system exhibits inherent biasing. For an ideal candidate, the virtual AI
quickly identifies the required elements using pattern recognition. The second
part is Candidate sourcing. The virtual assistant of Ideal uses client
credentials to automatically connect to various third-party candidate
websites. The required parameters will be made by the clients so that the
algorithm consistently searches those third-party websites for any matching
for the candidates.

###  3.3 Entelo

Entelo is one such example that revolves around a talent sourcing software
platform using AI to help companies identify quality candidates. The algorithm
that the company claims is something that is capable of doing some extreme
tasks in a short period of time. To achieve this factor, Entelo has identified
over 70 predictive variables which are further used to analyze the data of
different candidate profiles. One among the examples of the variables is
something like the Linkedin profile of the candidate. The company specifically
targets individuals who are not actively seeking employment but still are open
to exploring new opportunities.

##  4 Distance Similarity Used

###  4.1 Chebyshev

The Chebyshev distance, also called the Tchebyshev distance, is referred to as
the maximum absolute distance. It examines the absolute value of the distance
between any two points on the axes. The Chebyshev Distance is mainly used in
the game of chess and used in warehouse logistics. The Chebyshev distance is
given by the mathematical equation:

$d(p,q)=max(\left|q_{x}-p_{x}\right|,\left|q_{y}-p_{y}\right|)$

###  4.2 Canberra

The Canberra Distance is the measure of any two points of a set in a vector
space. The selected pair consists of non-negative real numbers. The Canberra
distance is used for comparing ranked lists and used in computer systems to
check for any malicious activities or any violation of policy. The
mathematical expansion is given as:

$d(p,q)=\frac{\left|q_{x}-p_{x}\right|}{\left|q_{x}\right|+\left|p_{x}\right|}+\frac{\left|q_{y}-p_{y}\right|}{\left|q_{y}\right|+\left|p_{y}\right|}$

###  4.3 Euclidean

Euclidean distance is the length of the line segment joining two points in an
Euclidean space. If the Cartesian coordinates of the points are known, the
Euclidean distance can be calculated using the Pythagorean theorem. So it is
sometimes called the Pythagorean distance. The Euclidean distance is symmetric
and it is positive for any two distinct points and zero for any point and
itself.

$d(p,q)=\sqrt{(q_{x}-p_{x})^{2}+(q_{y}-p_{y})^{2}}$

###  4.4 Manhattan

Manhattan distance is the sum of the projections of the line segment joining
two points onto the coordinate axes in an n-dimensional vector space. In
simple words, it is the sum of the absolute difference between all dimensions
of two points It is extensively used in a vast array of fields from regression
analysis to frequency distribution.

$d(p,q)=\left|q_{x}-p_{x}\right|+\left|q_{y}-p_{y}\right|$

###  4.5 Minkowski

The Minkowski distance can be considered as a generalization of the Euclidean
distance and the Manhattan distance. Minkowski distance of order p can be
found using the formula

$d(p,q)=(\sum_{i=1}^{n}|p_{i}-q_{i}|^{p})^{\frac{1}{p}}$

Where p is an integer. For p=1 and p=2, the Minkowski distance is reduced to
the Manhattan distance and Euclidean distance respectively. In this paper, the
Minkowski distance of order 3 has been implemented.

###  4.6 Bray-Curtis

Bray-Curtis distance is used to find how similar the given two candidates are.
It is calculated as follows:

$d(p,q)=\sum|u_{i}-v_{i}|/\sum|u_{i}+v_{i}|$

The differences between the respective values of the candidates are added
together and taken as the numerator. The summation of the sum of respective
values is taken as the denominator. The result provides the Bray-Curtis
distance. If the distance is close to 0, then it means that the given two
candidates are closely related.

###  4.7 Cosine

Cosine similarity and Cosine distance are inversely related to each other. As
the distance decreases, similarity increases. So, the value of Cosine distance
should be closer to 0 to get a highly similar result. Cosine similarity
projects the given two vectors on a multi-dimensional plane and calculates the
cosine angle between them.

$d(p,q)=cos(\theta)=\frac{\textbf{A}.\textbf{B}}{||\textbf{A}||\,||\textbf{B}||}=\frac{\sum_{i=1}^{n}A_{i}B_{i}}{\sqrt{\sum_{i=1}^{n}A_{i}^{2}}\sqrt{\sum_{i=1}^{n}B_{i}^{2}}}$

Here, A and B are the points projected on the multi-dimensional plane. The
cosine angle between A and B is calculated to find the cosine similarity.
Cosine distance = 1 - Cosine similarity If the distance is close to 0, then it
means that the given two candidates are highly similar to each other.

##  5 Proposed Method

To overcome this problem, the model tries to find the similarity between
candidates. In this paper, a new standardization metric is introduced for
measuring a candidate’s potential skills on the technical scale for the
recruitment process from a company’s point of view. This metric is termed as “
Turtle Score“.

###  5.1 Turtle-Score

A score based on the learning analytics score, GitHub score, Puzzle meter, Job
Shadowing meter, and error archive score is the metric that is used to measure
one’s potential skills in the technical scale for the recruitment process.
Various Machine Learning schemes are integrated to make the process much more
subtle. These scores each help to analyze the characteristics of a candidate
in specific problems.

###  5.2 Github Turtle Score

Github plays an important role in every developer’s career. Being a code-
hosting platform, Github can decide the experience and coding knowledge of a
person. A person can be evaluated just by looking at the contributions made by
them and the repositories along with the projects they had worked on. Github
score is the sum total of all these outcomes and it mainly focuses on the
number of commits that a particular person had contributed to all the
repositories over a span of time. This shows the consistency and the working
nature of a person within a few seconds thereby saving time.

###  5.3 Learning Analytics Turtle Score

LA score is derived from Learning Analytics which is based on the collection
and analysis of different websites and articles that the candidate had come
across while working. Learning Analytics is an extension that helps in
collecting and storing articles, research papers, websites, etc., which seems
to be helpful for a person to solve an error or to complete the coding
process. This score can help in determining the interests and skill set of a
candidate in a particular area of knowledge by analyzing all the data
collected by the individual using the Learning Analytics extension.

###  5.4 Kaggle Score

Kaggle is a well-known website for Machine Learning candidates which offers
datasets and Jupyter Notebooks to publish and run ML models. Kaggle is the
best option if one is going for machine learning-based tasks. Thus, the Kaggle
score would reveal the Machine Learning proficiency of a candidate and can
help determine the right candidate for the job accordingly.

###  5.5 Error Archive Score

Errors occupy an important position in the career of a person as it is a part
of the learning process and it helps change the perspective of viewing a
particular problem. This is why it is essential to keep a note of all the
errors and the solution used to overcome them. This score can help in
analyzing the problem-solving ability and the technical skill to overcome that
error.

###  5.6 Job Shadowing Meter

Job shadowing meters are derived based on how interactive and learnable a
particular candidate is during their job shadowing period. A candidate is
asked to observe someone else working on a particular technology and gain
knowledge from them. During this period, if the candidate is capable of
understanding what exactly is going on and if they are able to work on the
said technology without any guidance, then they get a high score. This is used
to see where a person stands on their learning and observation scale.

###  5.7 Puzzle Meter

Puzzle meters are an exclusive tool to identify candidates with an urge to
solve problems given to them no matter how long it takes or how hard the task
is. A candidate can be assigned with tasks from different areas of expertise
thereby the Manager can assess them by considering the time taken to solve the
particular problem and the interest of the candidate in that particular task
which made them work off the clock. This helps in filtering out the candidates
with talent in a particular field of interest. The company can assign each and
every candidate with a custom-made score according to their performance in the
tasks assigned. Thus, deserved candidates can be chosen for the jobs
accordingly.

Therefore by combining all these scores, a person would be able to determine
the total skill-set of a candidate and also find similar candidates with the
same talents. The similarity between any two candidates is found by comparing
the above parameters between any two candidates. This similarity found can in
turn be used to find the potential job position any given candidate might be
able to get.

##  6 Technology Used

###  6.1 Flask

This web framework is used to display the result in a web interface. The
application must be brought in this interface form in order to deploy it using
docker. Flask framework in this model takes in a user’s details as input and
provides with similar candidate list as output along with its similarity
percentile. It also predicts and displays the job position which might be
suitable for the candidate based on the given candidate data. This comparison
takes place in the background using the code present in a python file.

###  6.2 Docker

A simple docker deployment is done in order to avoid any errors related to
versions. The current machine learning model runs in a particular version with
which it is built. When it is used in a different system, it might throw some
error depending on the version used in that user’s system. To overcome this
problem, a docker image with all the versions specified is created. Since the
docker image installs the required versions, the application runs perfectly
fine in any system without errors related to its version.

##  7 Working

###  7.1 Data Flow

Figure 1: Data Flow

####  7.1.1 Raw Data

Real-time data of each candidate is piled and dumped into the database.

####  7.1.2 Data Pre-Processing

The data would be filtered and would undergo a few processes to remove
outliers and to fill the null values.

####  7.1.3 Data Manipulation

The data is visualized for a better understanding of the workflow of each
candidate and to compare among the candidates to find similar ones.

####  7.1.4 Distance-Similarity Calculation

The difference among each candidate’s data is calculated and an overall
similar percentile score is found.

####  7.1.5 ML Integration

This includes the prediction of similar candidates using the related similar
score accordingly using the machine learning tools and the accurate results
are produced.

###  7.2 Data Architecture

Figure 2: Architecture Chart

####  7.2.1 Training Flow

The inputs for the training model will be loaded. The input data set that has
been loaded for the training model is the company dataset of all candidates.
Now, the data is prepared for training where it will undergo some process to
fill null values and remove outliers in the provided data set. Now the
prepared data set will be tested and used in different distance similarity
measures for calculating the distance-similarity percentile. Now the machine
learning algorithm will be trained for some similar worker prediction. Now as
the model is further trained for job prediction under linear SVC. The
resulting data will be sent for final evaluation and the final model will be
presented.

####  7.2.2 Prediction Flow

As the training flow gets completed, the system will now load inputs for the
prediction model. The Input data set that has been loaded for the training
model is the company dataset of all candidates. Now, the data is prepared for
training where the data will undergo some process to fill null values and
remove outliers in the provided data set. Now the prepared data set will be
tested and used in different distance similarity measures for calculating the
distance-similarity percentile. Now the resulting values will be sent for
prediction analysis for finding a similar worker using the final model. Now as
the model is further trained for similar jobs to the predicted model. The
system will now present the result that was produced by the prediction model.

##  8 Result Analysis

To find the most similar candidate to any given candidate, different models to
get varying results are used. Each model has its own ups and downs, so these
models here are analyzed and their results are compared to one another. These
results are calculated based on the turtle score. Therefore the comparison is
done based on this and similarity is then calculated accordingly. It depends
on how closely they are related, that is, the smaller the difference between
the scores of any two candidates, the closer is the relation between them.

Figure 3: Similarity percentile table

Fig. 3 indicates the similarity percentile for 6 different people when they
are compared to a given candidate. This table shows how the result varies
depending on the distance calculation function used. Thus, based on distance
and percentile range, models can be ordered as least to most similar candidate
finder:

  1. 1.

Manhattan

  2. 2.

Euclidean

  3. 3.

Minkowski

  4. 4.

Canberra

  5. 5.

Chebyshev

  6. 6.

Bray-Curtis

  7. 7.

Cosine

Also, the most appropriate job for the candidate is predicted. For this, a
classification model is trained on the turtle score of the company employees.
Next, it is fed with the candidate’s turtle score to predict the suitable job
for the candidate in the company. Several supervised classification models
were tested and Linear Support Vector Classifier (SVC) was the best among them
with an accuracy of 87

##  9 Future Works

Over this course of Turtle score research, the comparison is done using
Distance Similarity measures such as Euclidean, Manhattan, Minkowski, and many
more. There are over 9 different Distance similarity measures used in this
research and further research can be done with many more similarity measures.
Further research can also be done by bringing up many more entities. This
might lead to a much closer similarity. Also, the addition of different unique
entities makes the turtle score more compatible and customizable as
organizations from different sectors can use it as well.

Data can also be derived from various candidates belonging to various age
groups. This will provide the dataset with diversity which in turn will
improve the application range of the model.

##  10 Conclusion

The primary ideology behind this applied research is that the process of
recruitment is made to be efficient in order to hire candidates like whom
exactly the company wishes to have. This can be achieved by the innovative
algorithm methodology this paper deals with. This can even study the student
and help them to identify their own standard and can sustain in IT industries.
This will make the job of the recruiting strategy more manageable in such a
way that companies can employ the individuals they hope to. Various algorithms
and testing were implemented to deliver the best outcomes.

One of the biggest pros of Turtle score is that it is compatible with any
environment. This turtle score can also be implemented in more than one set of
organizations such as

  * •

It can be modified in such a way that the schools can track the progress of
their students over the years,

  * •

Helps medical organizations to understand the patient’s health in terms of
recovering from any disease or any syndrome and many more.

It can also be used to find what kind of jobs are compatible for the given
candidate. This can be done by getting a similar candidate and checking their
current profession or job post. Therefore, this model can be used to predict a
person’s future job using their current data. This makes it possible to apply
the turtle score model to any college-going or school student and predict
their future job. It also helps to provide a progress report which tells them
where to improve in order to land a particular job post. This in turn
encourages students to stay on their feet and improve.

Finding similar candidates not only makes it possible to recruit people based
on the requirements but also provides more data about a person as similar data
is stored in the dataset. It’ll serve as an effective tool in identifying
potential amongst people in the IT sector or it can even spread out to other
sectors.

##  11 Acknowledgement

We would like to show our gratitude to Featureprenuer for sharing their
valuable data for the development of this application.

## References

  * [1] Chaitanya Bapat.  Blockchain for Academic Credentials.  arXiv e-prints, page arXiv:2006.12665, June 2020.
  * [2] Hrisav Bhowmick, Ananda Chatterjee, and Jaydip Sen.  Comprehensive Movie Recommendation System.  arXiv e-prints, page arXiv:2112.12463, December 2021.
  * [3] Teruaki Hayashi, Takumi Shimizu, and Yoshiaki Fukami.  Collaborative Problem Solving on a Data Platform Kaggle.  arXiv e-prints, page arXiv:2107.11929, July 2021.
  * [4] Heidy Hazem, Ahmed Awad, and Ahmed Hassan.  A Distributed Real-Time Recommender System for Big Data Streams.  arXiv e-prints, page arXiv:2204.04633, April 2022.
  * [5] Claudia M. Van Der Heijde and Beatrice I. J. M. Van Der Heijden.  A competence-based and multidimensional operationalization and measurement of employability.  Human Resource Management, 45(3):449–476, 2006.
  * [6] Jun Hu, Shengsheng Qian, Quan Fang, and Changsheng Xu.  MGDCF: Distance Learning via Markov Graph Diffusion for Neural Collaborative Filtering.  arXiv e-prints, page arXiv:2204.02338, April 2022.
  * [7] Philipp Kappus and Paul Groß.  Finding clusters of similar-minded people on twitter regarding the covid-19 pandemic.  In Machine Learning Techniques and Data Science. Academy and Industry Research Collaboration Center (AIRCC), nov 2021.
  * [8] Ali A. Mahmoud, Tahani AL Shawabkeh, Walid A. Salameh, and Ibrahim Al Amro.  Performance predicting in hiring process and performance appraisals using machine learning.  In 2019 10th International Conference on Information and Communication Systems (ICICS), pages 110–115, 2019.
  * [9] Anay Mehrotra, Bary S. R. Pradelski, and Nisheeth K. Vishnoi.  Selection in the Presence of Implicit Bias: The Advantage of Intersectional Constraints.  arXiv e-prints, page arXiv:2202.01661, February 2022.
  * [10] Hauff C. Nagaram A.R.  Using Github Profiles in Software Developer Recruitment and Hiring.  TUDelft, August 2021.
  * [11] Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, and Ece Kamar.  Investigations of Performance and Bias in Human-AI Teamwork in Hiring.  arXiv e-prints, page arXiv:2202.11812, February 2022.
  * [12] Le Van-Duyet, Vo Minh Quan, and Dang Quang An.  Skill2vec: Machine Learning Approach for Determining the Relevant Skills from Job Description.  arXiv e-prints, page arXiv:1707.09751, July 2017.


# Artificial neural network as an effective tool to calculate parameters of
positron annihilation lifetime spectra

M. Pietrow Corresponding Author<EMAIL_ADDRESS>Institute of Physics,
M. Curie-Skłodowska University, Pl. M. Curie-Skłodowskiej 1, 20-031 Lublin,
Poland  A. Miaskowski  Faculty of Production Engineering, University of Life
Sciences, Akademicka 13, 20-950 Lublin, Poland

###### Abstract

The paper presents the application of the multi-layer perceptron regressor
model for predicting the parameters of positron annihilation lifetime spectra
using the example of alkanes in the solid phase. A good agreement of
calculation results was found when comparing with the commonly used methods.
The presented method can be used as an alternative quick and accurate tool for
decomposition of PALS spectra in general. The advantages and disadvantages of
the new method are discussed.

##  1 Introduction

Positron Annihilation Lifetime Spectroscopy (PALS) is one of the useful
experimental methods using positrons for studying structural details in a wide
spectrum of materials, in particular in the solid state [1]. This method is
based on the annihilation of positrons where their lifetime and annihilation
intensity in the sample is dependent on some properties of the material in the
nano scale, including local electron density, bound electron energy, and the
density and size of free volumes in the sample.

Depending on the material, besides the process of direct annihilation, a
positron can form a meta-stable atomic state with an electron, called a
positronium (Ps), which can exist in two spin states referred to as _para_ \-
and _ortho_ -Ps differing in properties (especially, their lifetimes differ in
vacuum by three orders of magnitude) [2]. A number of conditions must be met
for the Ps to be formed in matter. One of them is that free volumes of a
sufficiently large size must be present. For these materials, a Ps is
extremely useful in material science since its lifetime can be related to the
size of free volumes [3]. Depending on the structure of the sample, there is
possibly a variety of Ps components which annihilate with characteristic
lifetimes. All these populations give their own account to the positron
annihilation spectrum measured experimentally. PALS spectra require
decomposition in the post-measuring procedure of decomposition resulting in
both the calculation of lifetimes for particular species of positrons and the
relative amplitudes for these processes (so-called spectrum inversion problem)
[4].

Many algorithms used for data processing require assuming an exponential
character of positron decay. They also require fixing the number of components
used during the decomposition. For example, the method used by one of the
adequate software, the LT programme [5] or PALSfit [6], consists in fitting
the PALS experimental spectrum to a sum of a given number of exponential
functions usually convoluted with the (multi) gaussian apparatus resolution
curve.

The PALS spectra used here were measured for normal alkanes (n-alkanes), i.e.
the simplest organic molecules where carbon atoms form a straight chain of the
molecule and are saturated by hydrogen atoms. The n-alkanes with a different
number $n$ of carbon atoms in the molecule form a homologous series described
by the general chemical formula CnH2n+2 (C$n$ is used as an abbreviation).
Alkanes in the solid phase form molecular crystals where the trains of
elongated molecules are separated by gaps called the inter-lamellar gaps. Ps
can be formed in the free volumes made by both the gaps and the spaces
generated by changes in the conformation with temperature [3]. Using the PALS
technique, the size of these free volumes can be determined from the lifetime
and the relation between both being given by the Tao-Eldrup formula or its
modification [7]. According to our previous analysis of alkanes carried out
with the use of the PALS technique, the best results of the spectrum
decomposition are achieved assuming only one population of _ortho_ \- and
_para_ -Ps, whereas the ratio of the _ortho_ to _para_ intensity is fixed at
3/1.

Tools of machine learning like genetic algorithms or artificial neural
networks have been used to perform numerical calculations in a variety of
aspects in positron science [8, 9, 8, 10, 11, 12]. They have also been used
for unfolding the lifetimes and intensities from PALS spectra [13, 14, 15,
16]. Possibly due to the low computing power of the hardware and the low time
resolution of PALS spectrometers at the time when the neural network
algorithms for decomposition of PALS spectra were proposed, most of the
spectra used in these calculations are simulated by the software but not
measured directly. For the same reason, the neural network architecture used
there does not allow changing parameters as much as is allowed by algorithms
developed today. Furthermore, no procedure allowing application for the same
calculations of spectra registered for different time constants per channel
has been presented since then. Thus, the preferred software used for spectrum
decomposition is still based on non-linear fitting algorithms which do not
include a possibility of establishing the result based on a multi-spectra set
at the same time.

Here, we present an approach to analysis of PALS spectra based on the multi-
layer perceptron (MLP) model, which is one of the tools of machine learning
[17]. The model assumes a network of inter-connected neurons grouped in the
input layer ($\mathtt{In}_{i}$), the hidden neurons layers
($\mathtt{h}_{i}^{k}$) and the output neurons layer ($\mathtt{Out}_{i}$),
where $i$ goes over the neurons in a given layer and $k$ numbers the hidden
layers. A graphical diagram of the network used is shown in fig. 1. The
numbers of $\mathtt{In}$ and $\mathtt{Out}$ neurons are determined by the
amount of the independent input data introduced to the network and the data
defined to be the results of calculation in a given problem, respectively. The
number of hidden layers and the number of neurons within these layers are set
experimentally to optimise the network to give required results. To each layer
(excluding the output layer), one bias neuron is attached for technical
reasons [18]. The tool assumes the learning process first, where the
$\mathtt{In}$ neurons are fed with the data for which the result of the
$\mathtt{Out}$ neurons is known in advance. During this process, the weight
coefficients for pairs of inter-connected neurons are adjusted by an
algorithm, so that the output of the MLP can give results most similar to the
expected ones. The MLP becomes to be trained after a number of iterations of
training. Once the MLP results of learning are satisfied, the MLP can be used
to calculate the output for the input data never used in the training process.

Figure 1: Schematic view of the MLP applied. The PALS data from consecutive
channels of the MCA are transferred as the amplitudes of the consecutive input
neurons $\mathtt{In}_{i}$. $\mathtt{h}_{i}$ denote neurons in the $i$-th
hidden layer whereas $\mathtt{Out}_{i}$ denote output neurons returning chosen
PALS decomposition parameters.

The MLP type of network can be applied to solve the problem of both
classification and regression. For the first group of problems, it is required
from the MLP to ascribe the values of the output parameters in the form of
well separated categories. These so-called _labels_ can always be parametrised
by a discrete set of numbers. The problem described in this paper is
classified as rather a regression problem (MLPR) where the values of the
output at each $\mathtt{Out}$ neuron are characterised by a continuous set of
values. Consequently, the output may contain values approaching these
appearing during the learning process but may not necessarily be exactly of
the same value. The internal algorithms of the MLPR allows regarding the
learning process as a way of finding the quasi-continuous output function of
input parameters. In our case, based on the data from the PALS spectra applied
as the input values of the perceptron, the MLPR is used for solving the
regression problem of finding the values of key PALS parameters on the output.

##  2 Method

The _scikit-learn_ library was used to estimate the PALS parameters for
alkanes [19]. In our case, the MLP regressor class (called _MLPRegressor_),
which belongs to one of the supervised neural network models, was implemented.
In this class, the output is a set of continuous values. It uses the square
error as the loss function. This model optimises the squared error using the
Broyden–Fletcher–Goldfarb–Shanno algorithm (LBFGS) [20], which belongs to
quasi-Newton methods. Some MLPRegressor parameters playing a key role are
mentioned below. Their values require to be tuned, especially the
$\mathtt{alpha}$ hyper-parameter, which helps in avoiding over-fitting by
penalising weights with large magnitudes. A full list of parameters of
MLPRegressor is defined in [21].

In the learning process here, we used spectra collected for years from an
analog spectrometer for several alkanes (in the range of C6 – C40) measured at
several temperatures (-142∘C – 100∘C). Irrespective of both the goal of the
particular experiment and the length of the alkane chain used as a sample, the
initial assumptions made for starting the analysis of the spectra made by the
LT programme [5] were the same. Each measurement resulting in the spectra used
was performed with a sample prepared in a similar way, i.e. the sample was
degassed, and the rate of cooling or heating was the same. In each case, the
measurement at constant temperature took place for at least one hour which
gave some hundreds of thousands of annihilations (the strength of the
radioactive source was similar in each case). During some experiments the
temperature was changed stepswise but each spectrum was collected at constant
temperature. The most important issue here is that the post-experimental
analysis of the spectra was conducted under the same general assumptions every
time. Especially, for the decomposition of these spectra, we used LT supposing
that the time resolution curve can be approximated by one-gaussian curve.
Every time it was assumed that the annihilation process in the Kapton envelope
accounted for 10% (so-called _source correction_). Additionally, only one
component was always assumed for _para_ \- and _ortho_ -Ps, whereas their
intensity ratio was fixed at the value 3/1 (see [22] for details of the
experimental procedure).

Taking into account these assumptions, each spectrum was decomposed into three
exponential curves for which the intensities ($I$) and lifetimes ($\tau$) were
calculated for the following sub-populations of positrons: the free positron
annihilation ($I_{2}$, $\tau_{2}$), _para_ ($I_{1}$, $\tau_{1}$), and _ortho_
-Ps ($I_{3}$, $\tau_{3}$)111Numbering of the indices is related to the length
of $\tau$. The increasing values of the indices correspond to the rising
length of lifetime.. The database collected in this way contained 7973 PALS
spectra, wherein about 75% were used in the neural network training process
and the rest were used as a testing set for checking the accuracy of the
results given by the learned network.

The number of input neurons is determined by the number of channels of the
Multi-channel Analyser (MCA) module of the PALS spectrometer recording PALS
spectra. Furthermore, the number of the output neurons is related in this
model to the number of PALS parameters, which are supposed to be predicted for
further studies of physical processes in the sample. The decomposition of the
PALS spectrum made by commonly used programs, like LT, allows determining
($I$,$\tau$) pairs for all assumed components of a given spectrum. However,
often, not all these parameters are needed for further analysis. Furthermore,
some of these parameters are inter-dependent. For example, in the case of PALS
spectra for the alkanes discussed here, one assumes that the spectrum is built
up by events from the three populations of positrons mentioned above
($\tau_{1}$ – $\tau_{3}$, $I_{1}$ – $I_{3}$ parameters). However, from the
practical view point, only $\tau_{2}$, $I_{2}$, $\tau_{3}$, and $I_{3}$ are
then used for studying physical processes and the structure of the sample.
Furthermore, in this case, $I_{i}$ are inter-dependent and fulfil the
following relations $I_{1}$+$I_{2}$+$I_{3}$=100%222Annihilation in Kapton was
subtracted in advance. and $I_{3}/I_{1}$=3. Thus, effectively, the parameters
considered as the $\mathtt{Out}$ parameters of MLPR are only $I_{2}$,
$\tau_{2}$, and $\tau_{3}$. According to this, we declared in our modelling
only three output neurons for receiving values for these three parameters.

##  3 Preparation of input and output data

During the PALS measurements, the time constant per channel ($\Delta$) varied,
depending on the internal properties and settings of the spectrometer. Most of
the data used here were collected with $\Delta$=11.9 ps; however, some spectra
were measured with $\Delta$=11.2 ps, 13.2 ps, 11.6 ps, and 19.5 ps (fig. 2).

Figure 2: Number of spectra (horizontal axis) with a given value of the time
constant per channel $\Delta$ (vertical axis) used as a data set in the
presented calculations.

Therefore, it is important for the $\mathtt{In}$ neurons to code the PALS
amplitude samples not in the relation to the channel numbers but in the scale
of time. Hence, in addition to the spectrum amplitudes, the regressor has to
learn the times associated with these amplitudes. Thus, one half of the
$\mathtt{In}$ neurons is fed with time values for consecutive channels of a
spectrum, whereas the second half is fed with the values of their amplitude.
The advantage of the regression approach applied here is the ability to test
spectra measured even for a time sequence that has never appeared in an
extreme case in the training process.

This method requires setting correctly a common zero-time for each spectrum.
To achieve this, the original data from the left slope of the spectrum peak
(and only a few points to its right) were used to interpolate the resolution
curve, which is assumed to be in the gaussian form. The position of this peak
defines a zero-time for a spectrum. One-gaussian interpolation is compatible
with previous LT analysis assumptions. Based on the common starting position
for all spectra established in this way, the values of time for each channel
on the right to the peak were re-calibrated for each spectrum depending on
$\Delta$ for which the spectrum was measured. Finally, for further analysis,
we took the same $N$ number of consecutive channels for each spectrum on the
right to its peak (points $p_{i}$ in fig. 3). The $\delta$ parameter shown in
fig. 3 denotes the distance (in time units) between the first point on the
right to the peak and the calculated time position of the peak. The number $N$
taken for further analysis was established experimentally. Finally, the
spectrum data for the MLPR input are the $N$ points $p_{i}$ with their two
values: the re-calibrated number of counts in a given channel (see below) and
their re-calibrated times of annihilation.

Figure 3: Schematic view of a peak region of the PALS spectrum. The bullets
indicate $(t,log(A))$ pairs saved in the MCA channels, whereas the star
indicates a position of a peak calculated assuming a gaussian shape of an
apparatus distribution function. Only the points to the right of the star
($p_{1}$, $p_{2}$, …) are taken as data introduced to MLPR. The $\delta$
parameter denotes a time distance between the calculated peak and the first
point, whereas $\Delta$ is the time distance between two points.

Then, to minimise errors, the original input data were transformed before
application. Each original spectrum was stored in 8192 channels of MCA.
Firstly, starting from the first channel on the right to the spectrum maximum
($p_{1}$ in fig. 3), 2k channels were taken from the original spectrum. This
means that the spectra were truncated at about 25 ns of the registration time
(varying to some extent, depending on the $\Delta$ for a given spectrum).
Secondly, to smooth random fluctuations, the data were smoothed in most cases.
One of the examples of smoothing is averaging over five consecutive channels.
In this case, the number of samples in each spectrum shrank from the original
2k channels to the amount of 400. Since the $\mathtt{In}$ neurons transfer
information about the pair of values – times ($t$-part) and amplitudes
($A$-part), 800 input neurons that fed the MLPR with the data in this case
were declared. Thirdly, to standardise the range of the input data values, the
set of the PALS amplitudes was normalised to the maximum value of the
amplitude and then logarithmised. According to these transformations, the
$A$-part data covered the numerical range [-9,0] – fig. 4. Furthermore, to
adjust the range of the values in the $t$-sector, the values of time were
divided by -2.5. As a result, all data transferred to the $\mathtt{In}$
neurons were in the range of [-10,0].

Figure 4: Input data directed to $\mathtt{In}$ neurons can be divided into two
sub-sets: $t$-part which is a set of time values for points $p_{1}$, $p_{2}$,
… (see fig. 3) and $A$-part coding the log function of their normalised
amplitudes. In special cases, these data are smoothed or compressed before use
in MLPR.

Additionally, we applied some transformation of the original values for the
$\mathtt{Out}$ neurons in order to have their values at each neuron scaled to
the same range. Initially, the first output neuron is related to $I_{2}$,
whereas its original value range is typically tenths (in % units). The second
neuron transfers the information related to $\tau_{2}$ whose original values
are of the order of 0.1 (of ns), whereas the order of $\tau_{3}$ related to
the third neuron is originally 1 (of ns). In order to have the uniform order
of numerical values on all $\mathtt{Out}$ neurons, the data that finally feed
with them are [$I_{2}$/10, $\tau_{2}\cdot$10, $\tau_{3}$].

The criterion of acceptance of training the network was the best value of the
score validation function defined for this regressor as

$\mathcal{S}=1-\frac{\sum_{N}(\mathcal{O}_{\text{true}}-\mathcal{O}_{\text{pred}})^{2}}{\sum_{N}\mathcal{O}_{\text{true}}^{2}},$
(1)

where $\mathcal{O}_{\text{true}}$, $\mathcal{O}_{\text{pred}}$ – expected
(known) and calculated (predicted) values of the result, respectively [19,
21]. $\mathcal{S}$ is calculated for both the learning and testing sets
separately. $N$ here denotes the number of spectra in the trained or tested
set. The optimum value of $\mathcal{S}$ is $\approx$1.

##  4 Results

The MLPRegressor used in these calculations requires establishing some key
parameters [21] influencing the ability to learn and a speed of the learning
process. We performed some tests trying to optimise these parameters. The best
results we obtained by the settings shown in tab. 1.

Table 1: Values of the MLPRegressor parameters applied for producing the final MPLR results. Parameter | value
---|---
$\mathtt{hidden\\_layer\\_sizes}$ = | 7$\times$150
$\mathtt{activation}$ = | _relu_
$\mathtt{solver}$ = | _lbfgs_
$\mathtt{alpha}$ = | 0.01
$\mathtt{learning\\_rate}$ = | _invscaling_
$\mathtt{power\\_t}$ = | 0.5
$\mathtt{max\\_iter}$ = | 5e+9
$\mathtt{random\\_state}$ = | None
$\mathtt{tol}$ = | 0.0001
$\mathtt{warm\\_start}$ = | True
$\mathtt{max\\_fun}$ = | 15000

Both the names and the meaning of the technical parameters shown in the table
are identical to these defined in the routine description [21]. Once the key
parameters of the MLPR were established (especially the $\mathtt{solver}$), we
performed tests of credibility of the network changing the number of hidden
layers, the number of neurons within ($\mathtt{hidden\\_layer\\_sizes}$
parameter), and the $\mathtt{alpha}$ parameter. The results in tab. 2 show
examples of the results. For these networks, we specified the mean validation
score parameter for both the training $\langle\mathcal{S}_{tr}\rangle$ and
testing $\langle\mathcal{S}_{te}\rangle$ sets separately with their variation
$\delta\mathcal{S}$. Averaging was made over the results of ten runs of the
training process for identical networks differing by initially random weights.
We did not notice any rule giving a ratio of the numbers of neurons that
should be declared in the consecutive hidden layers (especially as the number
of neurons should decrease proportionally in the consecutive layers). A few
initial examples shown here suggest that the accuracy of results increases
when both the number of hidden layers and the number of neurons inside
increase. However, the last two rows of the table show that a further increase
in these parameters does not give better results. Finally, the network that
gave a nearly best result was chosen (marked in bold
$\langle\mathcal{S}_{tr}\rangle$ in the table). It was checked for this
network that an increase in the iterations of training ($\mathtt{max\\_iter}$
parameter) beyond about 5$\cdot$109 did not improve
$\langle\mathcal{S}\rangle$.

Table 2: Valuation score for chosen values of some MLPR parameters. $\mathcal{S}$ values are averages over 10 runs with random initial neuron weights. A nearly optimum case of parameters is placed in a row with $\mathcal{S}$ marked in bold. $\mathtt{hidden\\_layer\\_sizes}$ | $\mathtt{max\\_iter}$ | $\mathtt{alpha}$ | $\langle\mathcal{S}_{tr}\rangle$ | $\delta\mathcal{S}_{tr}$ | $\langle\mathcal{S}_{te}\rangle$ | $\delta\mathcal{S}_{te}$
---|---|---|---|---|---|---
$30\times 25\times 15$ | 106 | 0.7 | 0.950 | 0.003 | 0.942 | 0.004
$3\times 100$ | 108 | 0.7 | 0.969 | 0.005 | 0.965 | 0.008
$3\times 100$ | 108 | 0.1 | 0.974 | 0.005 | 0.968 | 0.008
$3\times 100$ | 5$\cdot$ 108 | 0.1 | 0.975 | 0.003 | 0.976 | 0.003
$4\times 100$ | 108 | 0.1 | 0.978 | 0.004 | 0.975 | 0.006
$500\times 400\times 300\times 200$ | 5$\cdot$109 | 0.01 | 0.977 | 0.003 | 0.974 | 0.007
$7\times 150$ | 5$\cdot$108 | 0.01 | 0.985 | 0.002 | 0.975 | 0.013
$\begin{array}[]{rl}&500\times 500\times 400\times 400\times\\\ &\times 300\times 300\times 200\times 200\end{array}$ | 5$\cdot$109 | 0.01 | 0.978 | 0.005 | 0.977 | 0.008
$8\times 500$ | 5$\cdot$109 | 0.01 | 0.982 | 0.004 | 0.980 | 0.005

For several finally tested networks, the spectrum of the magnitude of inter-
neurons weights was checked. It is expected that weights that differ
significantly from the average range of values may affect the stability of the
results. In this case, the range of weight values seems to be quite narrow. As
shown in fig. 5, the weight magnitude order (exponent of weights) for the
chosen network ranges from 10-5 to 100, while the relative number of cases in
these subsets changes exponentially. The lack of values outside the narrow set
of values suggests that self-cleaning of the resultant weights is performed by
the MLPRegressor algorithm itself.

Figure 5: Number of cases (log scale) of exponents of weights for a network
with 7$\times$100 hidden layers. For this network, the key parameters are:
$\mathtt{solver}$=_lbfgs_ , $\mathtt{max\\_iter}$=5$\cdot$1011,
$\mathtt{alpha}$=0.005, $\mathtt{learning\\_rate}$=_invscaling_ , and
$\mathtt{activation}$=_relu_.

The number of all PALS spectra used as a database for the network was 7973,
and 6500 were used to learn the output values (training set) by the network,
while the rest were used for checking the results of learning (testing set).
Tab. 3 shows a few examples of randomly taken results given by one of the
networks finally used.

Table 3: Examples of a few randomly taken results of calculations (prediction) of the $I_{2}$, $\tau_{2}$, and $\tau_{3}$ parameters compared to the expected values calculated by LT. Here, $\mathtt{hidden\\_layer\\_size}$=7$\times$150, $\mathcal{S}$=0.985 for both training and testing sets. Example |  $I_{2}$ [%] |  $\tau_{2}$ [ns] |  $\tau_{3}$ [ns]
---|---|---|---
expected | predicted | expected | predicted | expected | predicted
1 | 68.0 | 68.8 | 0.27 | 0.28 | 1.21 | 1.20
2 | 47.2 | 47.6 | 0.38 | 0.39 | 3.23 | 3.18
3 | 65.4 | 65.5 | 0.30 | 0.29 | 1.35 | 1.38
4 | 78.3 | 78.3 | 0.23 | 0.24 | 1.11 | 1.12
5 | 52.4 | 52.4 | 0.35 | 0.34 | 2.93 | 2.93
6 | 60.5 | 60.4 | 0.31 | 0.30 | 1.25 | 1.20
7 | 59.3 | 58.8 | 0.29 | 0.29 | 1.19 | 1.22
8 | 61.0 | 62.3 | 0.30 | 0.31 | 1.91 | 1.91
9 | 70.2 | 69.5 | 0.21 | 0.21 | 1.06 | 1.10
10 | 39.9 | 38.8 | 0.23 | 0.22 | 1.15 | 1.13

The results given by the trained network were compared to the expected values
known from the LT analysis. Although $\mathcal{S}$ for both the trained and
tested sets in this case is not the highest one obtained in our tests, the
result of the use of this network is satisfactory in a practical sense because
the deviation of the predicted and expected result is in the range of
deviation given by LT itself.

The problem of pre-preparation of spectra for calculations by MLPR is worth
mentioning. The main problems are where the spectrum should be cut and to what
extent it is acceptable to smooth the spectra by averaging their consecutive
values. As for the first problem, it was determined by series of runs for
which the spectra were cut at other that mentioned limit of 2k channels that
this number of channels was almost the best choice.
$\langle\mathcal{S}\rangle$ was found to worsen in the case of a shorter cut
(say, 1.5k channels), and did not improve significantly in the case of the
longer ones (e.g. 3k channels) (but it took longer to compute the result
because of an increase in the number of $\mathtt{In}$ neurons).

The accuracy of the prediction increases when the learning and testing
processes are limited to one only $\Delta$ with all parameters of the network
kept constant. In this case, the set of values in the $t$-part for every
spectrum varies in a much narrower range (only $\delta$ changes). In this
case, the training process is more effective even if the size of the training
set is reduced. To show this, we separated the set of spectra measured for
only one $\Delta$=11.9 ps. Consequently, the whole set of samples under
consideration shrank to 4116 members, and 3000 of them were used for training
after the transformations described above. The score $\mathcal{S}$ obtained in
this case was much greater than $\mathcal{S}$ for an identical network applied
to spectra with all possible $\Delta$. The comparison of these two cases is
shown in tab. 4 (the last row of the table). Here, to have the result reliable
for networks with different (random) initial weights, the score was averaged
over 30 runs.

Table 4: Comparison of MLPR validation score $\mathcal{S}$ for different formats of the input data. Comparison of the results for ’raw’ data (log of normalised and adjusted data according to the procedure described in section 3) and data on which the moving average and compressing average (by each 3 and 5 separate spectrum points) are applied. The result for the network fed with the data collected for one chosen $\Delta$ is added in the last row of the table. MLPR and spectra parameters | $\langle\mathcal{S}_{tr}\rangle$ | $\langle\mathcal{S}_{te}\rangle$
---|---|---
$\mathtt{solver}$=_lbfgs_ $\mathtt{activation}$=_relu_ $\mathtt{learning\\_rate}$=_invscaling_ $\mathtt{alpha}$=0.01, $\mathtt{In}$=800 $\mathtt{hidden\\_layer\\_sizes}$=7$\times$150 $\mathtt{max\\_iter}$=5$\times$109 averaged over 30 trials |  Several $\Delta$s $N_{tr}$=6500 ($\sim$ 80%) $N_{te}$=1473 | unsmoothed data | 0.984$\pm$0.005 | 0.963$\pm$0.006
moving average | 0.984$\pm$0.005 | 0.977$\pm$0.007
k=3 | 0.981$\pm$0.005 | 0.976$\pm$0.007
k=5 | 0.981$\pm$0.006 | 0.974$\pm$0.010
Fixed $\Delta$=11.9 ps $N_{tr}$=3000 ($\sim$73%) $N_{te}$=1116 | k=5 | 0.993$\pm$0.002 | 0.989$\pm$0.003

The validation score $\mathcal{S}$ is sensitive to smoothing the spectrum
which reduces to some extent the information given by the PALS spectrum. In
tab. 4, two cases are compared where each 3- and 5-tuples of points of the
spectrum (forming non-overlapping windows) were taken to calculate their
average amplitude. For example, $N$=3500 points of the initial spectrum are
reduced to 700 points when averaging over $k$=5 points; when the remainder of
the division of $N$ by $k$ is not zero, an integer quotient is taken.
$\langle\mathcal{S}\rangle$ calculated for these two cases shows that both of
them give the same results statistically. However, further shrinking the
spectrum by setting $k$=6 or more produces worse $\mathcal{S}$.

Tab. 4 also shows the $\mathcal{S}$ parameter when the moving average is
applied during preparation of spectra. The sampling window applied here is 10.
The comparison of this result to the result of calculation with unsmoothed
data shows that the application of the moving average does improve predictions
for the testing data set.

##  5 Conclusions

We have shown in this paper that the easy-to-reach machine learning
MLPRegressor tool enhanced with some programming in _Python_ making some
preparation of data, can be used as an alternative method of solving the
problem of inversion of PALS spectra. The main disadvantage of the presented
method is the need of decomposition of training spectra by other software to
have $\mathtt{Out}$ values for training. Once the training set is collected
and the network is trained, the algorithm works very quickly, giving the
result for the tested spectrum. The training process used here is based on
results given by LT, i.e. a method producing results with some uncertainty
itself. The uncertainty produced by the LT is caused by the use of numerical
methods to compute the fit in particular cases. On the other hand, since the
MLPR prediction bases on information from a large set of spectra, this
approach seems to be less sensitive to the specific shape of a given spectrum
and may be more accurate in predicting parameters. Furthermore, the presented
method seems to be faster than the referenced ones, since calculations made by
a trained network are reduced to simple transformations of matrices and
vectors, which is not demanding computationally and less sensitive to
numerical problems.

Although the model presented here is similar to that described in [14] (and
repeated in [15]), there are significant differences indicated in tab. 5.

Table 5: Comparison of key parameters and results of the MLPR modelling applied in this study (_skLearn_) and a three-component spectrum analysis published previously (presented in [14] and [15]). |  Pázsit [14] |  An [15] | skLearn
---|---|---|---
Type of training spectra | simulated | simulated | real (alkanes)
No. of training spectra | 575 | 920 | 7973
Type of spectra tested | simulated | simulated, silicon | alkanes
No. of test spectra | 50 | 100 (30) | 1473
Type of network | one-layer perc. | one-layer perc. | multi-layer perc.
Channel width [ps] | 23.2 | 24.5 |  some (11.2-19.5)
No. of MCA/taken channels | 1500/1500 | 1024/1024 | 8192/3500
Approx. no. of counts in spec. | 10M | 10M |  $\sim$400k
Solver | backward error prop. | backward error prop. | some
No. of hidden layers | 1 | 1 | some
$I_{2}$, $\tau_{3}$ average error [%]
on tested simulated spectra | 7.3, 1.0 | 1.07-3.52, 0.55-1.21 | -, -
$I_{2}$, $\tau_{3}$ average error [%]
on tested real spectra | -, - | -, - | 1.03, 1.70

Our experimental data are collected by spectrometers differing in functional
properties, especially differing in time resolution. Even for one
spectrometer, this parameter should be re-calibrated periodically due to
changes in experimental conditions, especially temperature. In the algorithm
presented in [14], the same resolution curve for all spectra is assumed. In
our data preparation procedure, the parameters of the resolution curve are
interpolated for each case. Based on this, the $\delta$ parameter is
calculated and the value of the shift in time is established for consecutive
channels. Although one-gaussian resolution curve was assumed here, it is
possible to extend this algorithm for much more complicated cases where the
distribution curve consisted in a sum of gaussians, for example. As already
mentioned in [14], in that case, a possibility of recognising a distribution
function would give compatibility to MELT [23]. Such an extension requires
extending the calculations by applying another neural network, working in
advance, which returns the parameters of the resolution curve in a given case.
This problem has been solved by application of a Hopfield neural network [16].
Taking into account our collection of spectra, it was checked with the use of
LT and (occasionally) with MELT that the apparatus resolution curve is one-
gaussian for our spectra. Hence, they do not allow testing such an extended
model.

Furthermore, the MCA module of spectrometers may differ in the time constant
per channel $\Delta$. Thus, spectra used as a training data set may be
collected for different channel widths. Taking into account the method
presented in [14] for fixed $\Delta$, the training result is of little use for
spectra collected with another $\Delta$. Oppositely, we have shown the
possibility of application of an improved algorithm to data collected for
different $\Delta$s. The data collected from many spectrometers may contribute
to a large training data set, which allows solving the inversion problem for
any PALS spectrum and, thus, may be a universal tool that can be used in
different laboratories. Although the set of $\Delta$ used here is small, the
accuracy of the results is quite good. To use this tool to determine the real-
world spectrum parameters, the training process should be extended by adding
the spectra measured for a wider range of $\Delta$.

For greater generalisation, it is possible in principle to attach spectra
collected for other compounds to the training data set. For consistency, it
suffices for a training database to keep the same number of components (three
here) in spectrum decomposition. However, in practice, some incompatibilities
of the spectra for different compounds may arise because decomposition into a
few exponential processes is probably always a simplification of a real case
where some distribution of the size and shape of free volumes should be taken
into account as well as other Ps formation details.

Although the approach presented here is reduced to the analysis of alkanes
solely, the algorithm can be applied in calculation of PALS parameters of
other types of samples as well.

## References

  * [1] D. Manara, A. Seibert, et al.  Positron annihilation spectroscopy.  In M.H.A. Piro, editor, Advances in Nuclear Fuel Chemistry, chapter 2.3.5, pages 126–131. Elsevier Ltd., Woodhead Publishing, 2020.
  * [2] W. Greiner and J. Reinhardt.  Field Quantization.  Springer, 2013.
  * [3] T. Goworek.  Positronium as a probe of small free volumes in crystals, polymers and porous media.  Ann. Univ. Mariae Curie Sklodowska, sectio AA – Chemia, LXIX:1–110, 2014.
  * [4] Y.C. Jean, J.D. Van Horn, W-S Hung, and K-R Lee.  Perspective of positron annihilation spectroscopy in polymers.  Macromolecules, 46:7133–7145, 2013.
  * [5] J. Kansy.  Microcomputer program for analysis of positron annihilation lifetime spectra.  Nucl. Instr. Meth. A, 374:235–244, 1996.
  * [6] Jens V. Olsen, Peter Kirkegaard, Niels Jørgen Pedersen, and Morten Mostgaard Eldrup.  Palsfit: A new program for the evaluation of positron lifetime spectra.  Physica Status Solidi (C) Current Topics in Solid State Physics, 4(10):4004–4006, 2007.
  * [7] K. Wada and T. Hyodo.  A simple shape-free model for pore-size estimation with positron annihilation lifetime spectroscopy.  J. Phys. Conf. Ser., 443:012003, 2013.
  * [8] J. Jegal, D. Jeong, E.S. Seo, et al.  Convolutional neural network-based reconstruction for positronium annihilation localization.  Sci Rep, 12:8531, 2022.
  * [9] J.L. Herraiz, A. Bembibre, and A. López-Montes.  Deep-learning based positron range correction of pet images.  Applied Sciences, 11(1), 2021.
  * [10] M. Wędrowski.  Artificial neural network based position estimation in positron emission tomography.  PhD thesis, Interuniversity Institute for High Energies, Vrije Universiteit Brussel, Belgium, 2010.
  * [11] W.J. Whiteley.  Deep Learning in Positron Emission Tomography ImageDeep Learning in Positron Emission Tomography Image ReconstructionReconstruction.  PhD thesis, University of Tennessee, Knoxville, U.S.A., 2020.
  * [12] D. Petschke and T.E.M. Staab.  A supervised machine learning approach using naive gaussian bayes classification for shape-sensitive detector pulse discrimination in positron annihilation lifetime spectroscopy (pals).  Nucl. Instrum. Methods. Phys. Res. B, 947:162742, 2019.
  * [13] N.H.T. Lemes, J.P. Braga, and J.C. Belchior.  Applications of genetic algorithms for inverting positron lifetime spectrum.  Chem. Phys. Lett., 412(4):353–358, 2005.
  * [14] I. Pázsit, R. Chakarova, P Lindén, and F. Maurer.  Unfolding positron lifetime spectra with neural networks.  Appl. Surf. Sci., 149:97–102, 08 1999.
  * [15] R. An, J. Zhang, W. Kong, and B-J. Ye.  The application of artificial neural networks to the inversion of the positron lifetime spectrum.  Chinese Physics B, 21(11):117803, nov 2012.
  * [16] V.C. Viterbo, J.P. Braga, A.P. Braga, and M.B. de Almeida.  Inversion of simulated positron annihilation lifetime spectrum using a neural network.  J. Chem. Inf. Comput. Sci., 41:309–313, 2001.
  * [17] G. Rebala, A. Ravi, and S. Churiwala.  An Introduction to Machine Learning.  Springer Nature, 2019.
  * [18] J. Heaton.  Introduction to the Math of Neuroal Networks.  Heaton Res., 2012.
  * [19] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al.  Scikit-learn: Machine learning in python.  J. Mach. Learn. Res., 12:2825–2830, 2011.
  * [20] Broyden–fletcher–goldfarb–shanno algorithm.  https://en.wikipedia.org/wiki/Broyden-Fletcher-Goldfarb-Shanno_algorithm.  Accessed: 2022-11-20.
  * [21] sklearn.neural_network.mlpregressor.  https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html.
  * [22] T. Goworek, M. Pietrow, R. Zaleski, and B. Zgardzińska.  Positronium in high temperature phases of long-chain even n-alkanes.  Chem. Phys., 355:123–129, 2009.
  * [23] A. Shukla, M. Peter, and L. Hoffmann.  Analysis of positron lifetime spectra using quantified maximum entropy and a general linear filter.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 335(1):310–317, 1993.