# Formation of Regression Model for Analysis of Complex Systems
Using Methodology of Genetic Algorithms

Anatolii V. Mokshin1, Vladimir V. Mokshin2,3 and Diana A. Mirziyarova1
1Institute of Physics, Kazan Federal University, 420008 Kazan, Russia
2Institute for Computer Technologies and Information Protection, Kazan
National Research Technical University named after A.N. Tupolev-KAI, 420111
Kazan, Russia
3Almetyevsk branch of Kazan National Research Technical University named after
A.N. Tupolev-KAI, 423461 Kazan, Russia
E-mail<EMAIL_ADDRESS>
###### Abstract

This study presents the approach to analyzing the evolution of an arbitrary
complex system whose behavior is characterized by a set of different time-
dependent factors. The key requirement for these factors is only that they
must contain an information about the system; it does not matter at all what
the nature (physical, biological, social, economic, etc.) of a complex system
is. Within the framework of the presented theoretical approach, the problem of
searching for non-linear regression models that express the relationship
between these factors for a complex system under study is solved. It will be
shown that this problem can be solved using the methodology of _genetic
(evolutionary)_ algorithms. The resulting regression models make it possible
to predict the most probable evolution of the considered system, as well as to
determine the significance of some factors and, thereby, to formulate some
recommendations to drive by this system. It will be shown that the presented
theoretical approach can be used to analyze the data (information)
characterizing the educational process in the discipline ”Physics” in the
secondary school, and to develop the strategies for improving academic
performance in this discipline.

artificial intelligence, machine learning, genetic algorithms, regression
model, data analysis, complex system, statistical physics

##  I Description of the Approach

###  I.1 Factors and non-linear regression model.

Let evolution of a complex system be characterized by a finite set of time-
dependent factors. By _external factors_ , we mean those factors that are
affected by the system. Then, the internal factors are those factors that
contain information generated by the system itself. Using external factors, it
is possible to drive the considered system by setting the values of these
factors. Internal factors characterize the response of the system. Therefore,
their values can only be adjusted through external factors. To clarify this,
we give the following example from physics. Let some polymer material undergo
external mechanical stress, namely, shear deformation. In this case, the force
causing the shear deformation can be considered as an external factor. The
values of this force can be adjusted. In turn, as a result of the shear
deformation, some structural changes in this system may occur tempbib6 . These
structural changes will be characterized by the internal factors: local
density, local temperature, and local tangential stresses. In the general
case, this situation corresponds to a nonequilibrium physical process.
Therefore, the relationship between these factors cannot be described in terms
of any currently known physical laws; these relationships can be nontrivial
tempbib5 ; tempbib20 .

In accordance with the standard methodology of regression analysis tempbib3 ,
it is necessary to introduce the concepts of input and output factors. By an
output factor we mean a factor by which the evolution of the system is
monitored. It is necessary to note that the number of the output factors may
be more than one, but it must be finite. All other factors $x_{1}(t)$,
$x_{2}(t)$, $x_{3}(t)$, $\ldots$, $x_{M}(t)$ will be defined as input factors.
Here, $M$ is the number of the input factors. The choice of an output factor
is determined by the purpose of the study. Therefore, the attribution of the
factors to input or output factors may be a matter of convention.

Let there be a relationship between the output factor $y(t)$ and the input
factors $x_{1}(t)$, $x_{2}(t)$, $x_{3}(t)$, $\ldots$, $x_{M}(t)$, which is
defined as the generalized Kolmogorov-Gabor polynomial tempbib7 :

$\displaystyle
y(t)=a_{0}+\sum_{i=1}^{M}a_{i}x_{i}(t)+\sum_{i=1}^{M}\sum_{l>i}^{M}a_{il}x_{i}(t)x_{l}(t)+$
$\displaystyle\sum_{i=1}^{M}\sum_{l>i}^{M}\sum_{k>l}^{M}a_{ilk}x_{i}(t)x_{l}(t)x_{k}(t)+\ldots,$
(1)

where $a_{0}$, $a_{i}$, $a_{(i,l)}$, $\ldots$ are weight coefficients whose
values are initially unknown. These coefficients determine the contribution of
the corresponding input factors $x_{i}(t)$, $i=1,\;2,\;\ldots,\;M$ or the
product of these input factors in Eq. (I.1) to obtain a correct value of the
output factor $y(t)$. In other words, the weight coefficients can be
considered as a quantitative measure of the significance of the input factor
or the corresponding product of the input factors. Obviously, the greater the
number of the input factors, the more precisely it is possible to reproduce
the target values of the output factor $y(t)$. However, as can be seen from
expression (I.1), the greater the number of the input factors, the more
complex this expression. In general, in order to be able to formulate
expression (I.1), it is necessary that value of $M$ be finite.

Since expression (I.1) contains all the possible combinations of the products
of the input factors, then the problem of finding expression (I.1) in its
explicit form reduces to finding values of the weight coefficients $a_{0}$,
$a_{i}$, $a_{il}$, $\ldots$.

###  I.2 Search for the type of regression model as an optimization problem.

Expression (I.1) can be rewritten in a form which will be common for the
products of the input factors $x_{1}(t)$, $x_{2}(t)$, $x_{3}(t)$, $\ldots$,
$x_{M}(t)$:

$\displaystyle y(t)$ $\displaystyle=$ $\displaystyle a_{1}\cdot
x_{1}^{\delta_{1}^{(1)}(t)}\cdot x_{2}^{\delta_{2}^{(1)}(t)}\cdot
x_{3}^{\delta_{3}^{(1)}(t)}\cdot\ldots\cdot x_{M}^{\delta_{M}^{(1)}(t)}+$ (2)
$\displaystyle+a_{2}\cdot x_{1}^{\delta_{1}^{(2)}(t)}\cdot
x_{2}^{\delta_{2}^{(2)}(t)}\cdot x_{3}^{\delta_{3}^{(2)}(t)}\cdot\ldots\cdot
x_{M}^{\delta_{M}^{(2)}(t)}+$ $\displaystyle+a_{3}\cdot
x_{1}^{\delta_{1}^{(3)}(t)}\cdot x_{2}^{\delta_{2}^{(3)}(t)}\cdot
x_{3}^{\delta_{3}^{(3)}(t)}\cdot\ldots\cdot
x_{M}^{\delta_{M}^{(3)}(t)}+\cdots=$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{2^{M}}a_{i}\cdot x_{1}^{\delta_{1}^{(i)}(t)}\cdot
x_{2}^{\delta_{2}^{(i)}(t)}\cdot x_{3}^{\delta_{3}^{(i)}(t)}\cdot\ldots\cdot
x_{M}^{\delta_{M}^{(i)}(t)}=$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{2^{M}}a_{i}\prod_{j=1}^{M}x_{j}^{\delta_{j}^{(i)}(t)},$

where $i$ is the order number of the product of the input factors, and $j$ is
the index of the input factor in the corresponding product, and the quantity
$\delta_{j}^{(i)}$ characterizes the exponent and can take the values $0$ or
$1$, i.e. $\delta_{j}^{(i)}=0$ or $\delta_{j}^{(i)}=1$. We introduce the next
notation:

$\displaystyle\bar{X}_{i}(t)=\prod_{j=1}^{M}x_{j}^{\delta_{j}^{(i)}(t)}=$
$\displaystyle=x_{1}^{\delta_{1}^{(i)}(t)}\cdot
x_{2}^{\delta_{2}^{(i)}(t)}\cdot x_{3}^{\delta_{3}^{(i)}(t)}\cdot\ldots\cdot
x_{M}^{\delta_{M}^{(i)}(t)},$ (3)

where the factors in this product are written in ascending order of their
indices. Then, expression (I.1) takes the form

$y(t)=\sum_{i=1}^{2^{M}}a_{i}\bar{X}_{i}(t).$ (4)

From a mathematical point of view, expression (4) is the scalar product of two
vectors

$\bar{\bar{a}}=\\{a_{1},\;a_{2},\;a_{3},\ldots,a_{2^{M}}\\}$ (5)

and

$\bar{\bar{X}}(t)=\\{\bar{X}_{1}(t),\;\bar{X}_{2}(t),\;\bar{X}_{3}(t),\;\ldots,\;\bar{X}_{2^{M}}(t)\\},$
(6)

and can be written as

$y(t)=(\bar{\bar{a}},\bar{\bar{X}}(t)).$ (7)

Then, the search for expression (I.1) [or equivalent expression (2)], which
would correctly reproduce values of the output factor $y(t)$, is reduced to
the search for the exponents $\delta_{j}^{(i)}$ in the products
$\bar{X}_{i}(t)$. We note that this is a typical optimization problem tempbib8
; tempbib22 ; tempbib25 ; tempbib32 . In the case when the number of the input
factors is not large, such problems are solved by the so called brute force
method tempbib9 . However, when there are a sufficiently large number of the
input factors, solution to this problem can be obtained using the machine
learning methods, for example, using the genetic algorithms tempbib4 ;
tempbib23 ; tempbib26 ; tempbib28 ; tempbib29 ; tempbib30 ; tempbib31 .

###  I.3 Genetic Algorithms. Basic definitions and methodology.

Genetic algorithms is aimed to solve the optimization problem and are based on
the ideas typical for a natural evolutionary process. This allows one to avoid
to perform a consequent consideration of all the possible forms of the
equation for the output factor $y(t)$ with various combinations of the input
factors to find the optimal solution tempbib4 . The common GA methodology
includes the use of basic notions such as individual, genome and population as
well as the two specific operations tempbib4 . As applied to solving the
problem of finding the optimal regression model for $y(t)$, we introduce the
following definitions:

(i) We define a model polynomial of the form (2) for the output factor $y(t)$
as an individual. In general case, the concrete form of an individual $y(t)$
can be specified arbitrarily or in accordance with some rule. It is reasonable
to take a model polynomial for $y(t)$ as an individual, because the this
quantity is of the main interest in the given problem and this quantity should
be found directly by means of the genetic algorithms tempbib10 .

(ii) According to the GA methodology, uniqueness of an individual should be
determined by a genome. As seen from expression (2) and Eq. (I.2), all input
factors are contained in each product $\bar{X}_{i}(t)$, and these products are
included in the model polynomial for the output factor $y(t)$. Account for the
concrete input factor (say, the factor $x_{j}$) in a product $\bar{X}_{i}(t)$
is due to a value in exponent $\delta$ at this factor; recall that this value
can be $0$ or $1$. So, for example, when we need to take into account the
input factor $x_{j}$ in the product $\bar{X}_{i}(t)$, we should take the
exponent $\delta_{j}^{(i)}=1$. If we need to exclude the input factor $x_{j}$
from consideration, then we take the exponent $\delta$ at the corresponding
factor as $\delta_{j}^{(i)}=0$. Thus, each sequence of products of input
factors will be characterized by a unique set of ones and zeros. Then, it is
reasonable to define this unique set of zeros and units as a genome. To
clarify this, we provide the following example. Let there be four input
factors $x_{1}$, $x_{2}$, $x_{3}$ and $x_{4}$. Then, a some product, say,
$\bar{X}_{5}$ takes the following form:

$\bar{X}_{5}=x_{1}^{0}x_{2}^{1}x_{3}^{0}x_{4}^{1}\equiv
x_{2}^{1}x_{4}^{1}\equiv x_{2}x_{4},$ (8)

where the set of exponent values $\\{0,\;1,\;0,\;1\\}$ is a binary notation
for the digit $5$, i.e. for the index $i=5$. In other words, a natural
correspondence appears between the product index (in the given example with
$\bar{X}_{5}$, this is $i=5$) and the input factors, which are included in
this product. Recall that the input factors in expression (8) should be
written in order of increasing index.

(iii) By population we mean a set of individuals:

$\displaystyle y^{(1)}(t)$ $\displaystyle=$
$\displaystyle\sum_{i}a_{i}^{(1)}\bar{X}_{i}(t),$ $\displaystyle y^{(2)}(t)$
$\displaystyle=$ $\displaystyle\sum_{i}a_{i}^{(2)}\bar{X}_{i}(t),$
$\displaystyle y^{(3)}(t)$ $\displaystyle=$
$\displaystyle\sum_{i}a_{i}^{(3)}\bar{X}_{i}(t),$ $\displaystyle\cdots,$
$\displaystyle y^{(s)}(t)$ $\displaystyle=$
$\displaystyle\sum_{i}a_{i}^{(s)}\bar{X}_{i}(t),$ $\displaystyle\cdots,$

where $s$ is the individual index in the population. Each model polynomial
$y^{(s)}(t)$ of set (I.3) can be considered as some possible solution for the
output factor $y(t)$.

In addition, in accordance with the GA methodology, two basic evolutionary
operations must be defined: crossover and mutation operations tempbib12 ;
tempbib11 ; tempbib34 ; tempbib27 .

(iv) We define the crossover operation as the operation of obtaining a new
pair of the ”child” individuals by exchanging the right fragments of the
vectors $\bar{\bar{X}}^{(j)}$ and $\bar{\bar{X}}^{(k)}$, which belong to two
”parent” individuals $y^{(j)}$ and $y^{(k)}$ of the same population [see Eqs.
(6) and (7)]. Here, $j$ and $k$ are the indices of the individuals in the
population. Namely, to generate a new pair of individuals, it is necessary to
specify the so-called separation point, which will be a natural number $n$,
and $n\in[1,\;2^{M}+2]$. Here, the largest value of $n$ is $2^{M}+2$, because
there are $2^{M}$ various forms of the products $\bar{X}$ of the input factors
[see Eq. (2)] and two contributions $\bar{X}_{*}=0$ and $\bar{X}_{0}=1$ in the
same population. The point corresponding to this natural number $n$ will
divide the vectors $\bar{\bar{X}}^{(j)}$ and $\bar{\bar{X}}^{(k)}$ into left
and right parts. By swapping the right-hand sides of these vectors, we get a
new pair of the vectors and, thereby, we get a pair of the ”child”
individuals. This operation is illustrated by the following scheme (see Fig.
1)

Figure 1: The crossover operation

(v) Mutation is an operation due to which a new individual with a changed gene
appears tempbib13 . Recall that, in accordance with our methodology, the gene
is represented as a set of ones and zeros [see, for example, (8)], and this
set characterizes the presence or absence of a factor $x_{j}$ in a product
$\bar{X}_{i}$. Therefore, to obtain a new individual by means of the mutation,
we need to realize inversion of a random digit in this set: namely, zero
inverts to one, or vice versa. So, for example, in the case of the product
$\bar{X}_{5}$, that was given above, a possible result of the mutation can be

$\displaystyle\bar{X}_{5}=x_{1}^{0}x_{2}^{1}x_{3}^{0}x_{4}^{1}\;\Rightarrow\;\bar{X}_{7}=x_{1}^{0}x_{2}^{1}x_{3}^{1}x_{4}^{1}\equiv$
$\displaystyle x_{2}^{1}x_{3}^{1}x_{4}^{1}\equiv x_{2}x_{3}x_{4}.$ (10)

Here, the product $\bar{X}_{7}$ appears directly as a result of the mutation
with inverse procedure of the exponent for the input factor $x_{3}$.

###  I.4 Implementation of GA Methodology.

The main purpose of the above methodology is searching the most optimal
polynomial $y(t)$, which is expressed as (I.1) and is able to reproduce the
experimental (target) values of the output factor. Let we have a dataset for
the time period $[0,\;T]$. To implement the GA, it is necessary to define two
ranges that correspond to learning sampling and test sampling tempbib12 ;
tempbib11 . A rigorous approach requires solving the problem of finding the
optimal size of these samplings. It should be noted that if the sampling sizes
turn out to be small, then the available data in these samplings will not be
enough for the proper statistical analysis, and, therefore, the resulted model
for $y(t)$ will not be of a sufficient accuracy. On the other hand, the larger
the sizes of these samplings, the more computing resources are required. Thus,
the solution to this problem implies finding an optimal size of the samplings.
In the simplest implementation, one can set the learning and test samplings of
a same size. Moreover, to have a possibility to do a forecast with the data,
the overall time range $[0,\;T]$ can be divided into three ranges of the same
size: $[0,\;T/3]$, $[T/3,\;2T/3]$ and $[2T/3,\;T]$, which will be associated
with learning sampling, test sampling and forecast sampling, respectively.

The first stage of the methodology implies the formation of a trial population
of individuals $y^{(s)}(t)$, where $s=1,2,\ldots$. Data corresponding to the
learning sampling (for example, the time range $[0,\;2T/3]$) will be used to
generate a set of model polynomials $y^{(s)}(t)$. This is a learning regime.
”Experimental” data for the output factor $y(t)$ from the test sampling (for
example, from the time range $[2T/3,\;T]$) should be used to test how the
resulting models $y^{(s)}(t)$ are able to reproduce the “experimental” data.
To estimate quantitatively the accuracy of a resulting model, a matching
criterion $\Delta$ – e.g., the Fisher criterion, the standard deviation, the
coefficient of variation or the coefficient of determination – should be used.
The matching criterion $\Delta$ is computed for each polynomial $y^{(s)}(t)$
of a population, and, thus, the set of $\Delta^{(s)}$ is formed tempbib11 .

The second stage of the GA implies numerical calculation of the set of vectors
$\bar{\bar{a}}$ [see expression (7)], as well as finding the explicit form of
the polynomials $y^{(s)}(t)$ of a population and finding the values of the
corresponding criteria $\Delta^{(s)}$. Based on the found values of the
criterion $\Delta$, all the individuals of the population are ranked in order
of increasing values of $\Delta$. Namely, the individuals $y^{(s)}$, whose the
matching criterion $\Delta$ takes the highest values, are at the end of the
set. Then, the lower half of the ranked population is discarded. Thus, the
ranked set of values of $\Delta^{(s)}$ makes a sense of the so-called fitness
function, which is used in the GA to determine an acceptable solution domain
tempbib4 .

At the third stage of the algorithm, a population is replenished by means of
the crossover operation for the individuals and the mutation applied for the
genes, as was described above. We note that the crossover operation applied
for a random pair of individuals from the remaining half of the population is
carried out until the original size of the population will be restored. The
mutation for different pairs of individuals is performed with a certain given
probability.

The second and third stages are cyclically repeated until the first polynomial
$y^{(s)}$ of the population ranked by the values of the matching criterion
will corresponds with the required accuracy to “experimental” data for the
output factor $y(t)$.

Table 1: Input and output factors characterizing a current lesson Factor |  |
---|---|---
notation | Factor Description | Group of factors
$x_{1}$ | Factor indicating whether new knowledge was given by teacher in a lesson |
$x_{2}$ | Number of schoolchildren in a lesson |
$x_{3}$ | Factor indicating whether test work was in a lesson | Organization
$x_{4}$ | Factor indicating whether independent work was in a lesson | of a lesson
$x_{5}$ | Factor indicating whether demonstration experiment was given |
$x_{6}$ | Factor indicating whether laboratory work was in a lesson |
$x_{7}$ | Number of schoolchildren who received the assessment ”satisfactory” | Assessments
$x_{8}$ | Number of schoolchildren who received the assessment ”unsatisfactory” | of academic performance
$x_{9}$ | Have tasks been set for homework? |
$x_{10}$ | Have repetition of theoretical knowledge been given as homework? | Homework
$x_{11}$ | Have tasks associated with the search of an additional information |
| been given as homework? |
$x_{12}$ | Noise factor | Noise factors
$x_{13}$ | Noise factor |
$y_{1}$ | Number of schoolchildren who received | Output factor
| the assessments ”good” and ”excellent” in a lesson |

##  II Analysis of data characterizing the educational process in the
discipline of ”physics”

Applying the machine learning methods in an educational process represents a
completely new research field, which can be aimed at improving the efficiency
and quality of education tempbib35 . The interest in using the machine
learning algorithms for solving various educational problems is growing, as
evidenced by the regular appearance of new studies offering various original
approaches tempbib14 ; tempbib15 ; tempbib17 ; tempbib16 ; tempbib18 . In
particular, in the review paper tempbib14  it is proposed to use these methods
to predict student performance. At the same time, it is proposed in Ref.
tempbib14  to use these methods to make various forecasts on the basis of an
information relating to the social, age and psychological features of
students. It is noteworthy that the ideas given in Ref. tempbib14  are
consistent with the ideas proposed before in Refs. tempbib15 ; tempbib17 . It
is important to note that applying the machine learning methods in a
completely different context for the educational process was discussed in
Refs. tempbib16 ; tempbib18 . Namely, as demonstrated in Refs. tempbib16 ;
tempbib18 , it is possible to use the machine learning methods on the basis of
the so-called flexible tree-based algorithms to provide an accurate and
maximally objective assessment of the schoolchildren and students performance
on the basis of the test results. The corresponding results were obtained from
a comparative analysis of schoolchildren/student performance from nine
different countries.

In this work, we are implementing the GA methodology presented in Section I to
analyze the data for the educational process related with the physics
discipline in the concrete secondary school in Kazan (Russia). The analyzed
temporal period is covered the data for one and a half academic years. This
period includes $97$ lessons for the humanitarian designated class (H-class)
and the $146$ lessons for the class designated to natural sciences (NS-class).
The data contained the following information, namely, (i) the information
about the students assessments for each lesson, (ii) the information about
whether the test work, independent work, laboratory work was in a lesson,
whether a demonstration experiment was given during a lesson, and also (iii)
the information about the type of hometask for the schoolchildren.

Figure 2: Time-dependent factors for the H-class: input factor $x_{2}$
indicating the number of students in a lesson, and output factor $y_{1}$ which
indicate the number of schoolchildren who received the assessments “good” and
“excellent” in a lesson. Note that the number of schoolchildren in the full
class is $24$. Solid (blue) lines represent experimental data; red stars
connected by solid (red) line is a model result with Eq. (II). The learning,
test and forecast samplings are separated by vertical dashed lines.

The main aim of this study is to evaluate the efficiency of teaching physics
in these concrete lessons at this school. As a result of this, the factor
$y_{1}$ characterizing the number of schoolchildren received the assessments
”good” and ”excellent” during a lesson was chosen as an output factor. All
other factors were assigned to the input factors (for detail, see Tab. 1). As
seen from this table, all the input factors $x_{i}$, $i=1,\;2,\;\ldots,\;11$,
can be grouped as the follows: (a) the group characterizing the organization
of a lesson, (b) the group characterizing the assessments of academic
performance and (c) the group that characterizes the type of homework. Such a
division of the input factors into the groups may make it possible to
understand how the certain groups of factors influence the academic
performance. In addition, it is worth noting that we artificially introduced
two additional noise factors, $x_{12}$ and $x_{13}$, which represented the
arrays of random natural numbers. This was done in order to verify the
accuracy of the algorithm used. So, if the algorithm works correctly, then the
algorithm should recognize that these noise factors are not associated with
the physics teaching data. Note that the factors $x_{1}$, $x_{3}$, $x_{4}$,
$x_{5}$, $x_{6}$, $x_{9}$, $x_{10}$ and $x_{11}$ can only take two wordings:
”yes” or ”no”. In our numerical analysis, we apply the next correspondence
between these wordings and the numeric denotations: $0$ – ”no” and $1$ –
”yes”.

Figure 3: (a) Occurrence frequency $P(x_{i})$ of the considered input factor
$x_{i}$ depending on the index of this factor. Note that these frequencies
were obtained from the procedure of the regression model formation [see Eq.
(II)]. Various groups of the input factors are separated by vertical dashed
lines. (b) Occurrence frequencies presented in order of decreasing their
values.

Let us consider the results of analysis for the H-class. According to the
algorithm presented in Section I.4, on the basis of the data corresponding to
the H-class the following regression model was obtained:

$\displaystyle y_{1}(t)$ $\displaystyle=$ $\displaystyle
a^{(1)}+a^{(2)}x_{5}(t)x_{8}(t)+a^{(3)}x_{3}(t)x_{11}(t)+$
$\displaystyle+a^{(4)}x_{3}(t)x_{5}(t)+a^{(5)}x_{8}(t)x_{9}(t)+$
$\displaystyle+a^{(6)}x_{11}(t)+a^{(7)}x_{1}(t)x_{3}(t)+a^{(8)}x_{8}(t)+$
$\displaystyle+a^{(9)}x_{9}(t)+a^{(10)}x_{4}(t)x_{6}(t)+$
$\displaystyle+a^{(11)}x_{3}(t)x_{6}(t)+a^{(12)}x_{1}(t)x_{8}(t)+$
$\displaystyle+a^{(13)}x_{9}(t)x_{11}(t)+a^{(14)}x_{11}(t)+$
$\displaystyle+a^{(15)}x_{6}(t)x_{7}(t)+a^{(16)}x_{10}(t)x_{11}(t)+$
$\displaystyle+a^{(17)}x_{1}(t)+a^{(18)}x_{6}(t)x_{11}(t)+$
$\displaystyle+a^{(19)}x_{1}(t)x_{2}(t)+a^{(20)}x_{6}(t)x_{9}(t)+$
$\displaystyle+a^{(21)}x_{9}(t)x_{10}(t)+a^{(22)}x_{2}(t)x_{3}(t)+$
$\displaystyle+a^{(23)}x_{5}(t)x_{6}(t)+a^{(24)}x_{1}(t)x_{7}(t)+$
$\displaystyle+a^{(25)}x_{2}(t)x_{13}(t)+a^{(26)}x_{4}(t)x_{11}(t)+$
$\displaystyle+a^{(27)}x_{13}(t)+a^{(28)}x_{5}(t)x_{11}(t)+$
$\displaystyle+a^{(29)}x_{4}(t)x_{9}(t)+a^{(30)}x_{3}(t)+$
$\displaystyle+a^{(31)}x_{3}(t)x_{8}(t)+a^{(32)}x_{9}(t)x_{12}(t)+$
$\displaystyle+a^{(33)}x_{7}(t)+a^{(34)}x_{10}(t)x_{13}(t)+$
$\displaystyle+a^{(35)}x_{2}(t)x_{10}(t)+a^{(36)}x_{7}(t)x_{13}(t)+$
$\displaystyle+a^{(37)}x_{2}(t)x_{9}(t)+a^{(38)}x_{5}(t)x_{13}(t)+$
$\displaystyle+a^{(39)}x_{7}(t)x_{11}(t)+a^{(40)}x_{1}(t)x_{5}(t)+$
$\displaystyle+a^{(41)}x_{11}(t)x_{12}(t)+a^{(42)}x_{2}(t)x_{5}(t).$

This regression model was obtained as follows. Since the total time domain was
$T=97$ corresponding to $97$ lessons, then we have defined the learning
sampling of the size $[0,\;60]$ lessons and the test sampling of the size
$[60,\;92]$ lessons. Further, the sampling of the size $[92,\;97]$ lessons was
taken for the forecast (see Fig. 2). It is remarkable that, as seen from Fig.
2, the obtained regression model (II) reproduces correctly the experimental
data of the output factor $y_{1}(t)$ for the test sampling as well as for the
forecast sampling.

As a result of constructing the regression model for $y_{1}(t)$ [see Eq.
(II)], the occurrence frequency $P(x_{i})$ of each the input factor $x_{i}$,
$i=1,\;2,\;\ldots,\;13$, was obtained. In fact, the occurrence frequency,
normalized to unity, indicates the probability of the appearance of a certain
input factor during the procedure for finding the model polynomial $y_{1}(t)$.
The found frequencies for the input factors are presented in Fig. 3(a), and
the number near each symbol (circle) indicates the index of the input factor
corresponding the occurrence frequency. Obviously, the higher the occurrence
frequency of the input factor $x_{i}$, the more significant this factor for
the behaviour of the output factor $y_{1}(t)$. Fig. 3(b) shows the same
occurrence frequencies in order of decreasing their values. As seen, the
frequencies are grouped into three separate regions. The region with the
highest frequency values located in the upper left corner in Fig. 3(b)
contains the input factors $x_{6}$ and $x_{4}$ that have the greatest impact
on the behaviour of the output factor $y_{1}(t)$. Recall that these factors
characterize whether the laboratory and independent works were given in a
lesson. The second group of the input factors, which can be designated as less
significant factors, is located in the middle of the graph shown in Fig. 3(b).
This group includes the input factors that provide the following information:
whether the test work was given in a lesson ($x_{3}$), whether new knowledge
was given by teacher in a lesson ($x_{1}$), the number of schoolchildren who
received the assessment ”satisfactory” in a lesson ($x_{7}$), and, finally,
whether the demonstration experiment was given in a lesson ($x_{5}$). Other
factors located in the lower right corner in Fig. 3(b) appears to be the least
significant. It is important to note that the similar results were obtained
from the analysis for the NS-class.

Based on the performed analysis, we come to the following interesting
conclusions:

(i) It turned out that the factors characterizing the organization of the
lesson, namely, the factors associated with test work, independent work and
laboratory work, most strongly affect the academic performance of
schoolchildren in physics in this secondary school.

(ii) It was obtained the nontrivial result that the academic performance is
not strongly dependent on how many schoolchildren attend in a lesson. Thus,
there is a contradiction with the well known opinion that the less students
are present at the lesson, the higher their academic performance should be.
Nevertheless, it is important to take into account that the non-trivial result
obtained by us in this study follows from the data for the concrete discipline
in the concrete school, and, therefore, this conclusion is relevant only to
this data.

(iii) As turned out, the GA algorithm allows one to find the mathematical
model for the academic performance, on the basis of which a realistic forecast
for the nearest future can be done.

(iv) Unexpectedly, the group of factors related with the homework has a little
effect on the student academic performance.

(v) Finally, the noise factors $x_{12}(t)$ and $x_{13}(t)$ have no impact on
the academic performance. This conclusion follows directly from the obtained
distribution $P(x_{i})$ presented in Fig. 3(b): these noise factors have the
lowest values of the occurrence frequency and are at the right bottom part of
the distribution. Consequently, the GA algorithm recognizes these factors as
not related to the real input factors for the considered complex system. This
is evidence that the algorithm is resistant to artificially introduced
”disturbances”.

##  III Conclusion

In this study, we present the approach for analyzing the evolution of a
complex system. The approach is based on the GA-algorithms and allows one to
solve the specific problems associated with identifying the significant
factors by means of which it is possible to drive by a complex system, with
optimizing the dynamics of a complex system, and with predicting the evolution
of a system.

As an example, we have considered the data associated with teaching the
discipline “Physics” in the concrete secondary school in Kazan (Russia). On
the basis of the regression model generated for the schoolchildren academic
performance, the some results were obtained. In particular, as shown, it is
possible to identify the main factors affecting the educational process,
namely, teaching the discipline “Physics” in a concrete school. In addition,
as it turned out, it is possible to make a probabilistic forecast of student
performance for some immediate period of time, provided that the general
educational scheme for the discipline remains unchanged. The results generated
by means of the GA-algorithm for the education data can be applied to yield
the recommendations to improve the efficiency of a specific educational
process. It should be noted that the original approach presented in this study
can be applied to any other school discipline. Finally, this approach can be
generalized to solve the problems associated with improving the efficiency of
the educational institutions as a whole.

## Acknowledgements

This work is supported by the Russian Foundation for Basic Research (the
project No. $18$-$02$-$00407$).

## References

  * (1) M. Gell-Mann, What is complexity? Complexity. 1, 16 (1995).
  * (2) A. Bunde, M.I. Bogachev, Nonlinear memory and risk estimation in financial records, in Econophysics approaches to large-scale business data and financial crisis (edited by M. Takayasu et al.), New-York, Springer. 27 (2010).
  * (3) V.V. Sychev, Complex thermodynamic systems, New-York, Springer, 2009.
  * (4) D.A. Freedman, Statistical Models: Theory and Practice, Cambridge, Cambridge University Press, 2009.
  * (5) L.A. Gladkov, V.V. Kureichik, V.M Kureichik, Genetic Algorithms, Moscow, Fizmatlit, 2006.
  * (6) V.V. Mokshin, I.R. Saifudinov, L.M. Sharnin, M.V. Trusfus, P.I. Tutubalin, Parallel genetic algorithm of feature selection for complex system analysis, J. Phys.: Conf. Series. 1096(1), 012089 (2018).
  * (7) P.I. Tutubalin, V.V Mokshin, Structural and Functional Model of the On-board Expert Control System for a Prospective Unmanned Aerial Vehicle, Lecture Notes in Electrical Engineering. 641, 262 (2020).
  * (8) A.V. Mokshin, B.N. Galimzyanov, J.-L. Barrat, Extension of Classical Nucleation Theory for Uniformly Sheared Systems, Phys. Rev. E. 87 (2013) 062307.
  * (9) L.F. Cugliandolo, Dynamics of glassy systems, arXiv:cond-mat/0210312. 135 (2002).
  * (10) Z. Jiang, T. Zhao, S. Wang, F. Ren, A novel risk assessment and analysis method for correlation in a complex system based on multi-dimensional theory, Applied Sciences (Switzerland). 10, 3007 (2020).
  * (11) A.N. Kolmogorov, W. L. Doyle and I. Selin, Interpolation and extrapolation of stationary random sequences, Santa Monica, CA: RAND Corporation, 1962.
  * (12) D.E. Kvasov, Y.D. Sergeyev, Deterministic approaches for solving practical black-box global optimization problems, Advances in Engineering Software. 80, 58 (2015).
  * (13) H. Tang, X. Guo, L. Xie, S. Xue, Experimental validation of optimal parameter and uncertainty estimation for structural systems using a shued complex evolution metropolis algorithm, Applied Sciences (Switzerland). 9, 4959 (2019).
  * (14) H. Hu, Y. Li, Y. Bai, J. Zhang, M. Liu, The Improved Antlion Optimizer and Artificial Neural Network for Chinese Influenza Prediction, Complexity. 2019, 1480392 (2019).
  * (15) XF. Li, ZM. Lu, Optimizing the controllability of arbitrary networks with genetic algorithm, Physica A: Statistical Mechanics and its Applications. 4471, 422 (2016).
  * (16) G. Lindfield, J. Penny, Numerical Methods, Cambridge, Academic Press, 2012.
  * (17) C. Guo, Z. Yang, X. Wu, T. Tan, K. Zhao, Application of an adaptive multi-population parallel genetic algorithm with constraints in electromagnetic tomography with incomplete projections, Applied Sciences (Switzerland). 9, 2611 (2019).
  * (18) N.S. Pyko, S.A. Pyko, O.A. Markelov, O.V. Mamontov, M.I. Bogachev, Quantification of the feedback regulation by digital signal analysis methods: Application to blood pressure control efficacy, Applied Sciences (Switzerland) 10, 209 (2020).
  * (19) I. Grigorenko, M. E. Garcia, Ground-state wave functions of two-particle systems determined using quantum genetic algorithms, Physica A: Statistical Mechanics and its Applications. 291, 439 (2001).
  * (20) C. O. Stoico, D. G. Renzi, F. Vericat, A genetic algorithm for the 1D electron gas, Physica A: Statistical Mechanics and its Applications. 387, 159 (2008).
  * (21) J. Wu, X. Shao, J. Li, G. Huang, Scale-free properties of information flux networks in genetic algorithms, Physica A: Statistical Mechanics and its Applications. 391, 1692 (2012).
  * (22) Z. Li, J. Liu, A multi-agent genetic algorithm for community detection in complex networks, Physica A: Statistical Mechanics and its Applications. 4491, 336 (2016).
  * (23) J.R. Koza, Genetic programming: A paradigm for genetically breeding populations of computer programs to solve problems, Stanford, Computer Science Department, 1990.
  * (24) A.V. Mokshin, V.V. Mokshin, L.M. Sharnin, Adaptive genetic algorithms used to analyze behavior of complex system, Commun. Nonlinear Sci. Numer. Simulat. 71, 174 (2019).
  * (25) M. Srinivas, L.M. Patnaik, Adaptive probabilities of crossover and mutation in genetic algorithms, IEEE Trans. Syst. Man Cybern. 24, 656 (1994).
  * (26) S. L. Podvalny, M. I. Chizhov, P. Y. Gusev, K. Y. Gusev, The Crossover Operator of a Genetic Algorithm as Applied to the Task of a Production Planning, Procedia Computer Science. 150, 603 (2019).
  * (27) Y. Lei, S. Zhang, X. Li, C.M. Zhou, MATLAB Genetic Algorithm Toolbox and Applications, Xian, XiDian University Press, 2014.
  * (28) M. Chassignol, A. Khoroshavin, A. Klimova, A. Bilyatdinova, Artificial Intelligence trends in education: a narrative overview, Procedia Computer Science. 136, 16 (2018).
  * (29) S. B. Kotsiantis, Use of machine learning techniques for educational proposes: a decision support system for forecasting students grades, Artificial Intelligence Review. 37, 331 (2011).
  * (30) P. Cortez, A. Silva, Using data mining to predict secondary school student performance, EUROSIS, A. Brito and J. Teixeira (Eds.). 5 (2008).
  * (31) A. A. Saa, Educational Data Mining & Students Performance Prediction, International Journal of Advanced Computer Science and Applications. 7, 212 (2016).
  * (32) C. Masci, G. Johnes, T. Agasisti, Student and school performance across countries: A machine learning approach, European Journal of Operational Research. 269, 1072 (2018).
  * (33) S.K. Althaf Hussain Basha, Y.R. Ramesh Kumar, A. Govardhan, M. Zaheer Ahmed, Predicting Student Academic Performance Using Temporal Association Mining, International Journal of Information Science and Education. 2, 21 (2012).