# SnakeSynth: New Interactions for Generative Audio Synthesis

Eric Easthope

University of British Columbia  Vancouver, British Columbia, Canada
<EMAIL_ADDRESS>
###### Abstract

I present _SnakeSynth_ , a web-based lightweight audio synthesizer that
combines audio generated by a deep generative model and real-time continuous
two-dimensional (2D) input to create and control variable-length generative
sounds through 2D interaction gestures. Interaction gestures are touch and
mobile-compatible with analogies to strummed, bowed, and plucked musical
instrument controls. Point-and-click and drag-and-drop gestures directly
control audio playback length and I show that sound length and intensity are
modulated by interactions with a programmable 2D coordinate grid. Leveraging
the speed and ubiquity of browser-based audio and hardware acceleration in
Google’s TensorFlow.js we generate time-varying high-fidelity sounds with
real-time interactivity. SnakeSynth adaptively reproduces and interpolates
between sounds encountered during model training, notably without long
training times, and I briefly discuss possible futures for deep generative
models as an interactive paradigm for musical expression.

###### keywords:

audio synthesis, generative adversarial network, gestures, musical expression,
controller, 2D

††conference: NIME’23, 31 May–2 June, 2023, Mexico City, Mexico.

[500]Applied computing Sound and music computing [100]Applied computing
Performing arts [500]Computing methodologies Neural networks [300]Human-
centered computing Interaction techniques [300]Human-centered computing
Interaction paradigms

##  1 Background

Interaction paradigms for deep generative models (DGMs) have remained
relatively shallow in contrast to the diversity of interactions that are
possible with musical interfaces and most research interest in DGMs still
seems to revolve around generation of fixed-size images and audio in
correspondence to fixed-size training data [12, 13]. These models often work
by learning a low-dimensional set of inputs that resemble the statistics of a
training dataset enabling us to generate new samples through a small number of
controls significantly smaller than the size of training data. In experimental
contexts DGMs seem to be capable of generating novel outputs and controllably
interpolating between training data features [6]. Recent developments
including the success of WaveNet [10] and GANsynth [2] have revealed
possibilities for how DGMs might be developed to be more expressive in terms
of their outputs and this has consolidated some interest in using DGMs as
tools for musical expression. Possibly the largest unified effort to do this
might be Magenta (https://research.google/teams/brain/magenta/) at Google
Research which leverages DGMs as part of a larger effort to create music
synthetically using machine learning (ML) models.

Yet many projects featured for DGM-based music production and performance
still suffer from common structural limitations in DGMs and how they function.
Auto-regressive models [5] like WaveNet [10] inherently rely on sequential and
often somewhat random updates to inputs to produce appreciable changes in
outputs. In performance contexts changes in sound in response to new inputs
then need to computed in real time or otherwise delayed. Setting aside the
challenges of computing DGM outputs in real-time this breaks down an essential
auditory feedback loop between a performer and their instrument(s). Responses
on the part of the performer in response to an auto-regressive DGM then have
to be anticipated as inputs sequentially and must randomly evolve towards more
refined outputs.

This runs counter to how we think about musical instruments and related
interfaces. While we would not expect a string to resonate the same way every
time the same note is played, we do expect to hear the same note and for it to
resonate when we play it. Correspondingly there is some expectation in
performance being informed by anticipation about where and how to sound
instruments in a one-to-one way. This one-to-one-ness also ensures that
instruments play the same way today as they do tomorrow. In turn we think that
regressive DGMs and randomness alone cannot produce usable and moreover re-
usable ML-based digital music tools.

Luckily not all deep generative models are regressive and some are capable of
producing inputs and outputs in a one-to-one way. Generative Adversarial
Networks (GANs) [4] and variational auto-encoders [9] amongst other DGMs
require only a single forward pass (“single-pass”) from input to output making
them better candidates for musical interfaces by enabling performers to learn
relationships between how to play digital instruments and what will be sounded
when they play them. The development of audio-based GANs by Donahue et al. [1]
and Engel et al. [2] have shown particular promise for generating novel sounds
and musical forms. Technically speaking (input, output) pairs can be
established and anticipated during performance.

In broader performance contexts DGM-based instruments should exhibit
compatible playing dynamics with respect to player expectation. The
application of more energy to the instrument, for example by “strumming” or
“bowing” with greater intensity, should correspondingly produce more albeit
possibly cacophonous sound. Moving or scanning to selectively “pluck” strings
should not produce unwanted sound. Continuous “bowed” sounds should correspond
to continuous movements, particularly mechanical “driving” and resonance.
Reversing the direction of movement should “reverse” the sound in some way; on
a string this might correspond to differences in “down-picked” and “up-picked”
sounds. These are difficult to express with DGMs and even neural networks
broadly speaking due to the fixed length of their outputs and so there is an
opportunity here for new designs. The key problem is making DGMs expressive in
ways beyond their capacity to yield different outputs.

Part of this is a matter of producing and controlling continuous variable-
length sounds with DGMs. Discrete trigger-based controls for musical DGMs
resembling MIDI inputs are common but offer little to no control over the
length of output audio. This puts the burden of controlling audio length on
the underlying DGM(s). Previous work has done little to address the generation
of variable-length audio with DGMs despite the apparent utility of producing
variable-length sounds in music contexts. We can concatenate sounds to produce
longer streams of audio but results are often cacophonous (see algorithmic
music from Dadabots, https://dadabots.com). Thinking in terms of a 2D image
DGM architecture this is equivalent to generating variable-width images by
conjoining images sequentially and is a patch for the problem at best. This
makes the problem of generating variable-length audio with DGMs an interesting
gap in current work and a means to explore DGM expressivity in creative
settings as a performance tool.

_SnakeSynth_ (Figure 1) is an ML-driven music performance tool and interactive
controller that bridges the gap from discrete trigger-based DGM controls to
continuous variable-length controls to enable new forms of musical expression
and performance dynamics with DGMs. Deriving its name and interactive paradigm
from the “Snake” video game genre, SnakeSynth uses real-time 2D point-and-
click and drag-and-drop gestures to directly control DGM audio playback length
to generate variable-length audio in creative contexts. Interactions with a
programmable sound “grid” determine audio length relieving DGMs of un-needed
extraneous design constraints and giving more control to performers as the
primary creative agent. We forego concatenation-based approaches and model the
variability of audio length as an external interactive control over what is
otherwise a fixed-length DGM. In turn SnakeSynth enables “plucked,”
“strummed,” and “bowed” playing gestures by triggering different fixed-length
DGM-generated sounds and blending them through interaction to form longer
variable-length sounds.

Figure 1: The cursor-based _SnakeSynth_ web interface. Interactions with the
grid (left) are also touch-compatible supporting tap-to-click and touch-and-
drag gestures in correspondence to point-and-click and drag-and-drop cursor
gestures. Mel spectra (right) and time series (bottom) for individual sound
samples are displayed and updated in real time during playback.

##  2 Design

###  2.1 Model

####  2.1.1 Generative Adversarial Network

We set up a GAN made of two networks, a _generator_ and a _discriminator_ ,
configured as adversaries such that the generator network learns to generate
“fake” but convincingly real outputs that “fool” classifications by the
discriminator. As they train on new samples the generator improves its weights
by back-propagation to produce more “realistic” outputs that resemble the
statistics of training data. In turn the discriminator improves its weights to
discern fake samples from dataset samples. Any losses of the generator are
theoretically gains of the discriminator and vice versa, and both networks
improve with training.

We use a modified Deep Convolutional GAN (DCGAN) architecture and simplify the
DCGAN generator [11] to three convolutional layers and remove the batch
normalization layer succeeding the fully-connected layer. DCGANs have the
advantage of using local convolutional layers in place of exclusively fully-
connected layers [4] significantly reducing the total number of trainable
parameters and in turn reducing total training time. The generator consists of
a fully-connected layer and a sequence of three filter layers each containing
a convolutional layer, a batch normalization layer, and activated with a leaky
rectified linear unit (leaky ReLU) layer. Batch normalization layers
regularize training samples to increase training stability [11, 7]. The final
convolutional layer is activated with a $\tanh$ function layer.

We also use a Convolutional Neural Network (CNN)-based discriminator with two
convolutional layers, no dropout layers, and no batch normalization. Again
this significantly reduces the total number of trainable parameters. The
discriminator consists of two convolutional layers activated with leaky ReLU
layers and a fully-connected layer with no activation layer. Without an
activation layer the discriminator outputs values outside of $[-1,1]$ and we
compute cross-entropy loss directly from _logits_ from the fully-connected
layer. Unlike the DCGAN authors we do not change any network initialization
weights before training and we use _reshape_ and _flatten_ layers to transform
square images to and from layers expecting flat inputs. Both networks are
summarized in Figure 2 for a latent space of size two in correspondence with a
2D cursor or touch-based control space. This amounts to a generator with just
over one million trainable parameters (approximately 245 parameters per
pixel).

Figure 2: Simplified Deep Convolutional GAN (DCGAN) [11] network architecture
used to generate SnakeSynth sounds. The generator inputs two values, and
outputs 64x64 pixel images activated with a $\tanh$ function layer. The
discriminator inputs these 64x64 pixel images, and outputs one value to
classify “real” dataset samples versus “fake” generator samples. The
discriminator foregoes an activation function to output values outside of
$[-1,1]$.

####  2.1.2 Dataset: 2D Spectral Images

Our DCGAN training data comprises of square 64x64 pixel images made from Mel-
scaled spectral coefficients for a small collection of human voice samples,
which is only one of many possible training sets. This is in accordance with
observations by Engel et al. [2] that GANs are capable of producing high-
fidelity audio from limited spectral information and are significantly faster
to train. Notably Mel-scaled coefficients are an effective compression of
spectral audio information in 2D that simultaneously accounts for human audio
perception. Compared to 1D time series-based approaches like WaveGAN [1] using
images of spectral coefficients in place of audio significantly reduces the
overall dimensionality and memory requirements for training and our use of GAN
models in SnakeSynth.

Sounds in SnakeSynth are computed by inverting generated 2D spectral images
into 1D time series through Griffin-Lim inversion. Each sound is windowed
using a cosine window or similar to remove edge audio artifacts. We currently
choose the number of sounds in the SnakeSynth interaction grid so this
inversion is automated as part of data post-processing. Faster inversions
could be realized in real-time interactive settings.

####  2.1.3 GAN Training

Generator outputs are initialized as Gaussian noise and we train the DCGAN
generator and discriminator in lockstep, training the generator first to
produce new outputs and then the discriminator second on outputs produced by
the generator and against real samples from the training dataset. This
training strategy is equivalent to a zero-sum competitive game between two
players (or networks in this case) where losses of the generator amount to
gains of the discriminator and vice versa. Goodfellow et al. [4] represents
this with the objective function

$L(G,D)=\mathbb{E}_{x\sim\mu_{ref}}[\ln
D(x)]+\mathbb{E}_{z\sim\mu_{Z}}[\ln(1-D(G(z))]$

for a generator $G$ and discriminator $D$ where $\mathbb{E}$ shows expectation
values and $\mu_{ref}$ is the set of possible outcomes in the sample
distribution and $\mu_{Z}$ is the set of possible generated outcomes in a
Gaussian (normal) distribution. Correspondingly $\mathbb{E}_{x\sim\mu_{ref}}$
shows the expectation value that real samples ($x$) from the training dataset
or generated samples ($z$) are part of the sample space or the generated space
respectively. Together these incur a generator and discriminator “loss” during
training that we use to back-propagate updates to generator and discriminator
weights.

The training dataset is shuffled and separated into batch sizes of one so that
every image is seen during training and we train for 300 epochs using the same
objective function defined by Goodfellow et al. [4]. Notably increasing the
batch size can reduce training time. We find that a small 64x64 pixel DCGAN
model with a two-dimensional generator latent space can train hundreds of
epochs within a few minutes on a standard MacBook Pro (2020). Re-training on
new samples correspondingly takes at most a few minutes longer and trained
models and sounds can be stored and loaded for later use without further
training.

Because the generator input dimension is only two we are able to directly
access and visualize the space of possible generator outputs by passing 2D
quantile values as coordinate inputs to the generator latent space. This
enables us to produce samples from most of the generator output distribution
using a 2D and particularly _finite_ grid-based interactive controller. To do
this quantile values up to 95th percentile outcomes are computed from the
inverse cumulative density function (inverse CDF) for a 2D Gaussian (normal)
distribution with zero mean ($\mu=0$) and unit variance ($\sigma^{2}=1$).
Plotting images generated from these quantile values produces a visualization
of nearly all possible generator outputs to arbitrary precision from a
restriction of the entire generator output distribution (Figure 3, right)
helping us design controllers with consideration for underlying generator
statistics.

###  2.2 Interactions

Figure 3: SnakeSynth generates images through maps of Gaussian (normal)
quantile values sampled (in red) from two standard deviations in a 2D
coordinate grid to a 2D generator latent space. This produces sample outputs
from a restriction of the entire possible generator output distribution up to
95th percentile outcomes for different arbitrary grid sizes.

SnakeSynth affords a number of different interaction types naturally through
interactions with a two-dimensional $N\times N$ coordinate grid (Figure 3,
left):

  1. 1.

Click (or touch) gestures produce fixed-length audio (resembles “plucking”).

  2. 2.

Linear or near-linear gestures produce variable-length audio (resembles
“strumming”) (Figure 5). Gesture distance determines sound length.

  3. 3.

Suddenly changing movement direction creates sudden audio changes and
corresponding audio attack (resembles a “finite bow”) (Figure 6).

  4. 4.

Continuous gestures create continuous audio (resembles an “infinite bow”)
(Figure 7). Particularly circular or near-circular gestures produce continuous
_rhythmic_ audio.

  5. 5.

Chaotic gestures with many directional changes to linear and/or circular
movements create cacophonous audio (resembles “brushing”) (Figure 8).

Interactions 2-5 are showcased in Appendix A.

###  2.3 Synthesis

Instead of directly concatenating audio clips we trigger equal-length clips
asynchronously and sum them over time to produce variable-length audio in
response to interaction. Each sound is windowed in data post-processing so
this is functionally similar to the overlap-add method. Simulating three
equally-spaced interactions with the SnakeSynth grid in Figure 3 we see three
generated sample sounds and their windowing functions sum to produce a single
variable-length sound. We also see the amplitude of the resulting sound
increase with greater sample overlap. This is chosen to produce interaction
analogies to mechanical “driving” and resonance (as mentioned before) and
other ways to blend overlapping audio could be explored.

Figure 4: Three time-triggered sound samples (rows) generated and summed over
time to produce a single variable-length sound (bottom). Increasing amplitude
by summing overlapping sounds enables virtual mechanical “driving” through the
SnakeSynth controller to produce resonance-like effects. The amplitude of the
resulting sound increases with greater sample overlap.

##  3 Discussion

By foregoing concatenation-based approaches and modelling the variability of
audio length in terms of interaction we lose the precision of a triggered
fixed-size model and we have to choose how to blend sounds in context.
However, we think that what we gain in flexibility in terms of modularity and
greater choice over DGMs should not be understated as it keeps the abundance
of fixed-length GAN models and ongoing research available to us as design
options. Non-generative models of the same dimension would even suffice. Going
further this flexibility enables us to create novel controllers for audio DGMs
capable of generating variable-length audio. This does not seem to be widely
recognized as a design interest and surprisingly we have seen little
discussion about it in previous work.

SnakeSynth offers one way around the problem by treating audio length as a
parameter of interactive control outside of the generative model. This bridges
the gap from fixed-length audio DGMs to controller-driven variable-length DGMs
and even to DGM-based music performance by recognizing that asynchronously
triggering audio clips over time is congruent to mapping user interactions
over time. Given the ubiquity of cursor movements and touch in 2D digital
coordinate spaces these seem to be an appropriate starting point for
discussion on and exploration of user interaction as a means of DGM control
and particularly DGM control for musical expression.

Choices on how to map from the SnakeSynth coordinate grid to generator latent
space(s) raise interesting questions about both the shape of the DGM latent
spaces themselves but also how to construct novel and/or non-trivial maps
between them and the SnakeSynth grid. We are not required to use quantile
values as inputs either and interactions could be readily extended to any
interface that produces at least two values in real time.

This is somewhere in the design of latent space-based control that existing
human-computer interface principles like Fitts’ law [3] could be applied such
as knowledge about distance to target, target size, cognitive load, etc.
Similarly the design of real-time sound blending beyond slowed attacks and
windowing, especially for asynchronously triggered sounds, deserves further
consideration. Semantically some of these design choices would reflect
different views of the audio space or context at hand so as to be recognizable
and learn-able by performers and reproducible in performance settings.

DGM research continues to evolve at quick pace and we are still finding new
ways to train high-fidelity GANs quickly enough, for example by progressively
adding layers during training [8], that it may soon be feasible to train small
GAN models in real time. This would enable SnakeSynth and derived tools to
“evolve” new auditory spaces in response to real-time interactions and/or new
data. This takes us away from the mindset of fixed-length audio models and re-
frames digital musical instruments as things capable of evolving over time to
adapt to context and changing how we might think about digital music tools for
performance.

##  4 Conclusion

I showed how _SnakeSynth_ , demoed as a web-based audio synthesizer, combines
DGM audio and real-time continuous 2D input to create and control variable-
length generative sounds through various interaction gestures. Interaction
gestures have analogies to strummed, bowed, and plucked musical instrument
controls. I showed that sound length and intensity are modulated by
interactive control with a 2D programmable sound grid, and briefly discussed
possible futures for DGMs as an interactive paradigm for musical expression.

##  5 Acknowledgments

I thank Robert for his guidance and for pointing to the novelty of real-time
GAN synthesis in the browser.

##  6 Ethical Standards

The author is self-funded and reports no conflicts of interest. No living
subjects were studied in this work.

## References

  * [1] C. Donahue, J. McAuley, and M. Puckette.  Adversarial Audio Synthesis, Feb. 2019.  arXiv:1802.04208 [cs].
  * [2] J. Engel, K. K. Agrawal, S. Chen, I. Gulrajani, C. Donahue, and A. Roberts.  GANSynth: Adversarial Neural Audio Synthesis, Apr. 2019.  arXiv:1902.08710 [cs, eess, stat].
  * [3] P. M. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement.  Journal of Experimental Psychology, 47(6):381–391, June 1954.
  * [4] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio.  Generative Adversarial Networks, June 2014.  arXiv:1406.2661 [cs, stat].
  * [5] K. Gregor, I. Danihelka, A. Mnih, C. Blundell, and D. Wierstra.  Deep AutoRegressive Networks, May 2014.  arXiv:1310.8499 [cs, stat].
  * [6] E. Härkönen, A. Hertzmann, J. Lehtinen, and S. Paris.  GANSpace: Discovering Interpretable GAN Controls, Dec. 2020\.  arXiv:2004.02546 [cs].
  * [7] S. Ioffe and C. Szegedy.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.  In Proceedings of the 32nd International Conference on Machine Learning, pages 448–456. PMLR, June 2015.
  * [8] T. Karras, T. Aila, S. Laine, and J. Lehtinen.  Progressive Growing of GANs for Improved Quality, Stability, and Variation, Feb. 2018.  arXiv:1710.10196 [cs, stat].
  * [9] D. P. Kingma and M. Welling.  Auto-Encoding Variational Bayes, May 2014.  arXiv:1312.6114 [cs, stat].
  * [10] A. v. d. Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu.  WaveNet: A Generative Model for Raw Audio, Sept. 2016.  arXiv:1609.03499 [cs].
  * [11] A. Radford, L. Metz, and S. Chintala.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, Jan. 2016.  arXiv:1511.06434 [cs].
  * [12] S. Shahriar.  GAN computers generate arts? A survey on visual arts, music, and literary text generation using generative adversarial network.  Displays, 73:102237, July 2022.
  * [13] H. Zhang, L. Xie, and K. Qi.  Implement Music Generation with GAN: A Systematic Review.  In 2021 International Conference on Computer Engineering and Application (ICCEA), pages 352–355, Kunming, China, June 2021. IEEE.

##  Appendix A Interactions

Figure 5: Linear or near-linear gestures produce variable-length audio
(resembles “strumming”). Gesture distance determines sound length. Figure 6:
Suddenly changing movement direction creates sudden audio changes and
corresponding audio attack (resembles a “finite bow”). Figure 7: Continuous
gestures create continuous audio (resembles an “infinite bow”). Particularly
circular or near-circular gestures produce continuous _rhythmic_ audio. Figure
8: Chaotic gestures with many directional changes to linear and/or circular
movements create cacophonous audio (resembles “brushing”).


# Stabilization of stochastic dynamical systems of a random structure with
Markov switches and Poisson perturbations ††thanks: Citation: Lukashiv, T.;
Litvinchuk, Y.; Malyk, I.V.; Golebiewska, A.; Nazarov, P.V. Stabilization of
stochastic dynamical systems of a random structure with Markov switches and
Poisson perturbations. Pages…. DOI:000000/11111.

Taras Lukashiv
Multiomics Data Science Research Group, NORLUX Neuro-Oncology Laboratory
Department of Cancer Research
Luxembourg Institute of Health, L-1210, Luxembourg, Luxembourg
Department of Mathematical Problems of Control and Cybernetics,
Yuriy Fedkovych Chernivtsi National University, 58000, Chernivtsi, Ukraine
<EMAIL_ADDRESS>
Yuliia Litvinchuk, Igor V. Malyk
Department of Mathematical Problems of Control and Cybernetics,
Yuriy Fedkovych Chernivtsi National University, 58000, Chernivtsi, Ukraine
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Anna Golebiewska
NORLUX Neuro-Oncology Laboratory, Department of Cancer Research
Luxembourg Institute of Health, L-1210, Luxembourg, Luxembourg
<EMAIL_ADDRESS>
Petr. V. Nazarov
Multiomics Data Science Research Group, Department of Cancer Research
Luxembourg Institute of Health, L-1445, Strassen, Luxembourg
<EMAIL_ADDRESS>

###### Abstract

An optimal control for a dynamical system optimizes a certain objective
function. Here we consider the construction of an optimal control for a
stochastic dynamical system with a random structure, Poisson perturbations and
random jumps, which makes the system stable in probability. Sufficient
conditions of the stability in probability are obtained, using the second
Lyapunov method, in which the construction of the corresponding functions
plays an important role. Here we provide a solution to the problem of optimal
stabilization in a general case. For a linear system with a quadratic quality
function, we give a method of synthesis of optimal control based on the
solution of Riccati equations. Finally, in an autonomous case, a system of
differential equations was constructed to obtain unknown matrices that are
used for the building of an optimal control. The method of a small parameter
is justified for the algorithmic search of an optimal control. This approach
brings a novel solution to the problem of optimal stabilization for a
stochastic dynamical system with a random structure, Markov switches and
Poisson perturbations.

_K_ eywords Optimal control $\cdot$ Lyapunov function $\cdot$ System of
stochastic differential equations $\cdot$ Markov switches $\cdot$ Poisson
perturbations

##  1 Introduction

The main problem considered in this paper is the problem of synthesis of
optimal control for a controlled dynamical system, described by a stochastic
differential equation (SDE) with Poisson perturbations and external random
jumps [2, 7, 24, 22, 25]. The importance of this problem is linked to the fact
that the dynamics of many real processes cannot be described by continuous
models such as ordinary differential equations or Ito’s stochastic
differential equations [17]. More complex systems include the presence of
jumps, and these jumps can occur at random $\tau_{k},k\geq 1$ or deterministic
time moments $t_{m},m\geq 1$. In the first case, the jump-like change can be
described by point processes [3, 6], or in the more specific case by
generalized Poisson processes, the dynamics of which is characterized only by
the intensity of the jumps. The jumps of the system at deterministic moments
of time $t_{m}$ can be described by the relation

$\Delta x(t_{m})=x(t_{m})-x(t_{m}-)=g(...),$ (1)

where $x(t),t\geq 0$, is a random process describing the dynamics of the
system. According to the works of Katz I.Ya. [7], Yasinsky V.K., Yurchenko
I.V. and Lukashiv T.O. [14, 15] the description of jumps at deterministic time
moments $t_{m}$ are quite accurately described using the relation (1). It
allows a relatively simple transfer of the basic properties of stochastic
systems of differential equations without jumps ($g\equiv 0$) to systems with
jumps. Such properties, as will be noted below, include the Markovian property
$x(t),t\geq 0$, concerning natural filtering, and the martingale properties
$\|x(t)\|^{2},t\geq 0$ [4, 10]. It should be noted that the description of
real dynamical systems is not limited to the Wiener process and point
processes (Poisson process). A more general approach is based on the use of
semimartingales [18]. The disadvantage of this approach is that it is
impossible to link it with the methods used for systems described by ordinary
differential equations or stochastic Ito’s differential equations. The second
approach to describe jump processes $x(t)$ is based on the use of semi-Markov
processes, considered in the works of Korolyuk V.S. [11] and Malyk I.V. [21,
22, 23]. In [25], the authors proposed the asymptotic behavior of dynamical
systems described by semi-Markov processes. Several works [21, 22, 23] are
devoted to the convergence of semi-Markov evolutions in the averaging scheme
and diffusion approximation. The results derived in these works together with
the results of the works on large deviations (e.g. [9]) can also be used to
investigate the considered problems.

Since we consider generalized classical differential equations, the approaches
will also be used classical. The basic research method is based on the
Lyapunov methods described in the paper by Katz I.Ya. [7], Lukashiv T.O.,
Malyk I.V. [13]. It should be noted that the application of this method makes
it possible to find the optimal control for linear systems with a quadratic
quality functional, which also corresponds to classical dynamical systems [1].

The structure of the paper is as follows. In section 2 we consider the
mathematical model of a dynamical system with jumps. It is described by a
system of stochastic differential equations with Poisson’s integral and
external jumps. Sufficient conditions for the existence and uniqueness of the
solution of this system are given there. In section 3 we investigate the
stability in probability of the solution $x(t),\geq 0$. In this section, we
consider the notion of the Lyapunov function and prove the sufficient
conditions for stability in probability (Theorem 1). The algorithm for
computing the quality functional $J_{u}(y,h,x_{0})$ from the known control
$u(t)$ is given in the section 4. Moreover, we further present sufficient
conditions for the existence of optimal control (Theorem 2), which are based
on the existence of a Lyapunov function for the given system. Section 5
considers constructing an optimal control for linear non-autonomous systems
via the coefficients of the system. The optimal control is found by solving
auxiliary Ricatti equations (Theorem 3). For the analysis of linear autonomous
systems, we consider the construction of the quadratic functional. Finally, we
formulate sufficient conditions of existence of an optimal control (Theorem
4), and present the explicit form such control in the case of a quadratic
quality functional.

##  2 Task definition

On the probability basis $(\Omega,\mathfrak{F},F,\mathbf{P})$ [4], consider a
stochastic dynamical system of a random structure given by Ito’s stochastic
differential equation (SDE) with Poisson perturbations:

$dx(t)=a(t-,\xi(t-),x(t-),u(t-))dt+$ $b(t-,\xi(t-),x(t-),u(t-))dw(t)+$

$\int\limits_{\mathbb{R}^{m}}(c(t-,\xi(t-),x(t-),u(t-),z))\widetilde{\nu}(dz,dt),~{}t\in\mathbb{R}_{+}\backslash
K,$ (2)

with Markov switches

$\Delta
x(t)\Big{|}_{t=t_{k}}=g(t_{k}-,\xi(t_{k}-),\eta_{k},x(t_{k}-)),~{}~{}~{}t_{k}\in
K=\\{t_{n}\Uparrow\\}$ (3)

for $\lim\limits_{n\rightarrow+\infty}t_{n}=+\infty$ and initial conditions

$x(0)=x_{0}\in\mathbb{R}^{m},~{}\xi(0)=y\in\mathbf{Y},\eta_{0}=h\in\mathbf{H}.$
(4)

Here $\xi(t),t\geq 0,$ is a homogeneous continuous Markov process with a
finite number of states $\mathbf{Y}:=\\{y_{1},...,y_{N}\\}$ and a generator
$Q$; $\\{\eta_{k},k\geq 0\\}$ is a Markov chain with values in the space
$\mathbf{H}$ and the transition probability matrix $\mathbb{P}_{H}$;
$x:[0,+\infty)\times\Omega\rightarrow\mathbb{R}^{m}$; $w(t)$ is an
$m$-dimensional standard Wiener process;
$\widetilde{\nu}(dz,dt)=\nu(dz,dt)-\mathbb{E}\nu(dz,dt)$ is a centered Poisson
measure; the processes $w,\nu,\xi$ and $\eta$ are independent [4], [17]. We
denote by

${\mathfrak{F}}_{t_{k}}=\sigma(\xi(s),w(s),\nu(s,*),\eta_{e},s\leq
t_{k},t_{e}\leq t_{k})$

a minimal $\sigma$-algebra with respect to which $\xi(t)$ is measurable for
all $t\in[t_{0},t_{k}]$ and $\eta_{n}$ for $n\leq k$.

The process $x(t),t\geq 0$ is $c\grave{a}dl\grave{a}g$; the control
$u(t):=u(t,x(t)):[0,T]\times\mathbb{R}^{m}\rightarrow\mathbb{R}^{m}$ is an
$m$-measure function from the class of admissible controls $U$ [1].

The following mappings are measurable by a set of variables
$a:\mathbb{R}_{+}\times\mathbf{Y}\times\mathbb{R}^{m}\times\mathbb{R}^{m}\rightarrow\mathbb{R}^{m}$,
$b:\mathbb{R}_{+}\times\mathbf{Y}\times\mathbb{R}^{m}\times\mathbb{R}^{m}\rightarrow\mathbb{R}^{m}$,
$c:\mathbb{R}_{+}\times\mathbf{Y}\times\mathbb{R}^{m}\times\mathbb{R}^{m}\times\mathbb{R}^{m}\rightarrow\mathbb{R}^{m}$
and function
$g:\mathbb{R}_{+}\times\mathbf{Y}\times\mathbf{H}\times\mathbb{R}^{m}\rightarrow\mathbb{R}^{m}$
satisfy the Lipschitz condition

$|a(t,y,x_{1},u)-a(t,y,x_{2},u)|+|b(t,y,x_{1},u)-b(t,y,x_{2},u)|+$
$+\int\limits_{\mathbb{R}^{m}}|c(t,y,x_{1},u,z)-c(t,y,x_{2},u,z)|\Pi(dz)+$

$+|g(t,y,h,x_{1})-g(t,y,h,x_{2})|\leq L|x_{1}-x_{2}|,$ (5)

where $\Pi(dz)$ is defined by $\mathbb{E}\nu(dz,dt)=\Pi(dz)dt$, $L>0$,
$x_{1},x_{2}\in\mathbb{R}^{m}$ for $\forall t\geq
0,y\in\mathbf{Y},h\in\mathbf{H}$, and the condition

$|a(t,y,0,u)|+|b(t,y,0,u)|+\int\limits_{\mathbb{R}^{m}}|c(t,y,0,u,z)|\Pi(dz)+$
$|g(t,y,h,0)|\leq C<\infty,$ (6)

The conditions defined above with respect to the mappings $a,b,c$ and $g$
guarantee the existence of a strong solution of the problem (2)–(4) with the
exact stochastic equivalence [13].

Let us denote

${\it\mathbf{P}}_{k}((y,h,x),\Gamma\times G\times\mathbf{C}):=$
$:=P(\xi(t_{k+1}),\eta_{k+1},x(t_{k+1})\in\Gamma\times
G\times\mathbf{C}|(\xi(t_{k}),\eta_{k},x(t_{k}))=(y,h,x))$

the transition probability of a Markov chain $(\xi(t_{k}),\eta_{k},x(t_{k}))$,
determining the solution to the problem (2)–(4) $x(t)$, at the $k$-th step.

##  3 Stability in probability

Definition 1. The discrete Lyapunov operator $(lv_{k})(y,h,x)$ on a sequence
of measurable scalar functions
$v_{k}(y,h,x){\it:\mathbf{Y}}\times{\it\mathbf{H}}\times{\it\mathbb{R}}^{m}\to{\it\mathbb{R}}^{1},k\in{\it\mathbb{N}}\cup\\{0\\}$
for SDE (2) with Markov switches (3) is defined by the equation

$(lv_{k})(y,h,x):=$
$:=\int\limits_{{\it\mathbf{Y}}\times{\it\mathbf{H}}\times{\it\mathbb{R}}^{m}}{\it\mathbf{P}}_{k}((y,h,x),du\times
dz\times dl)v_{k+1}(u,z,l)-v_{k}(y,h,x),k\geq 0.$ (7)

When applying the second Lyapunov method to the SDE (2) with Markov switches
(3), special sequences of the above mentioned functions
$v_{k}(y,h,x),k\in{\it\mathbb{N}}$ are required.

Definition 2. The Lyapunov function for the system of the random structure
(2)-(4) is a sequence of non-negative functions $\left\\{v_{k}(y,h,x),k\geq
0\right\\},$ for which

  1. 1.

for all $k\geq
0,y\in{\it\mathbf{Y}},h\in{\it\mathbf{H}},x\in{\it\mathbb{R}}^{m}$ the
discrete Lyapunov operator is defined $(lv_{k})(y,h,x)$ (7);

  2. 2.

for $r\to\infty$

$\bar{v}(r)\equiv\mathop{\inf
v_{k}}\limits_{\begin{array}[]{l}{k\in{\it\mathbb{N}},y\in{\it\mathbf{Y}},}\\\
{h\in{\it\mathbf{H}},\left|x\right|\geq r}\end{array}}(y,h,x)\to+\infty;$

  3. 3.

for $r\to 0$

$\underline{v}(r)\equiv\mathop{\sup
v_{k}}\limits_{\begin{array}[]{l}{k\in{\it\mathbb{N}},y\in{\it\mathbf{Y}},}\\\
{h\in{\it\mathbf{H}},\left|x\right|\leq r}\end{array}}(y,h,x)\to 0,$

and moreover $\bar{v}(r)$ and $\underline{v}(r)$ are continuous and strictly
monotonous.

Definition 3. Let us call a system of random structure (2)-(4) stable in
probability on the whole, if for $\forall\varepsilon_{1}>0,\varepsilon_{2}>0$
on can specify such $\delta>0$ that from the inequality
$\left|x\right|<\delta$ follows the inequality

${\it\mathbf{P}}\left\\{\mathop{\sup}\limits_{t\geq
0}\left|x(t)\right|>\varepsilon_{1}\right\\}<\varepsilon_{2}$ (8)

for all $x_{0}\in{\it\mathbb{R}}^{m}$,
$y\in{\it\mathbf{Y}},h\in{\it\mathbf{H}}$.

To solve the problem (2)-(4) on the intervals $[t_{k},t_{k+1})$ the following
estimate takes place.

Lemma 1. Let the coefficients of the equation (2) $a,b,c$ and function $g$
satisfy the Lipschitz condition (5) and the uniform boundedness condition (6).

Then for all $k\geq 0$ the inequality for the strong solution of the Cauchy
problem (2)-(4) holds

${\it\mathbf{E}}\left\\{\mathop{\sup}\limits_{t_{k}\leq
t<t_{k+1}}\left|x(t)\right|^{2}\right\\}\leq
7\left[\mathbb{E}\left|x(t_{k})\right|^{2}+3C^{2}(t_{k+1}-t_{k})\right]\times$
$\times\exp\left\\{7L^{2}((t_{k+1}-t_{k})+8)\right\\},t\in(t_{k},t_{k+1}).$
(9)

Proof of Lemma 1

Using the integral form of the strong solution of the equation (2) [10], for
all $t\in[t_{k},t_{k+1}),t_{k}\geq 0,$ the following inequality is true

${\left|x(t)\right|\leq\left|x(t_{k})\right|+\int_{t_{k}}^{t}\left|a(\tau,\xi(\tau),x(\tau),u(\tau))-a(\tau,\xi(\tau),0,u(\tau))\right|d\tau+}$
$+\int\limits_{t_{k}}^{t}\left|a(\tau,\xi(\tau),0,u(\tau))\right|d\tau+$
$+\int\limits_{t_{k}}^{t}\left|b(\tau,\xi(\tau),x(\tau),u(\tau))-b(\tau,\xi(\tau),0,u(\tau))\right|dw(\tau)+$
$+\int_{t_{k}}^{t}\left|b(\tau,\xi(\tau),0,u(\tau))\right|dw(\tau)+$
$+\int\limits_{t_{k}}^{t}\int\limits_{\mathbb{R}^{m}}\left|c(\tau,\xi(\tau),x(\tau),u(\tau),z)-c(\tau,\xi(\tau),0,u(\tau),z)\right|\widetilde{\nu}(dz,d\tau)+$
$+\int\limits_{t_{k}}^{t}\int\limits_{\mathbb{R}^{m}}\left|c(\tau,\xi(\tau),0,u(\tau),z)\right|\widetilde{\nu}(dz,d\tau)$

Given (5), (6) and the inequality $\left(\sum_{i=1}^{n}x_{i}\right)^{2}\leq
n\sum_{i=1}^{n}x_{i}^{2}$ we get:

$\sup\limits_{t_{k}\leq t<t_{k+1}}\left|x(t)\right|^{2}\leq
7\sup\limits_{t_{k}\leq t<t_{k+1}}\left[\left|x(t_{k})\right|^{2}+\right.$

$+\left|\int_{t_{k}}^{t}\left|a(\tau,\xi(\tau),x(\tau),u(\tau))-a(\tau,\xi(\tau),0,u(\tau))\right|d\tau\right|^{2}+$
$+\left|\int\limits_{t_{k}}^{t}\left|a(\tau,\xi(\tau),0,u(\tau))\right|d\tau\right|^{2}+$
$+\left|\int\limits_{t_{k}}^{t}\left|b(\tau,\xi(\tau),x(\tau),u(\tau))-b(\tau,\xi(\tau),0,u(\tau))\right|dw(\tau)\right|^{2}+$
$+\left|\int_{t_{k}}^{t}\left|b(\tau,\xi(\tau),0,u(\tau))\right|dw(\tau)\right|^{2}+$
$+\left|\int\limits_{t_{k}}^{t}\int\limits_{\mathbb{R}^{m}}\left|c(\tau,\xi(\tau),x(\tau),u(\tau),z)-c(\tau,\xi(\tau),0,u(\tau),z)\right|\widetilde{\nu}(dz,d\tau)\right|^{2}+$
$+\left.\left|\int\limits_{t_{k}}^{t}\int\limits_{\mathbb{R}^{m}}\left|c(\tau,\xi(\tau),0,u(\tau),z)\right|\widetilde{\nu}(dz,d\tau)\right|^{2}\right]\leq$
$\leq 7\left[\sup\limits_{t_{k}\leq
t<t_{k+1}}\left|x(t)\right|^{2}+\sup\limits_{t_{k}\leq
t<t_{k+1}}L^{2}\left|\int\limits_{t_{k}}^{t}\left|x(\tau)\right|d\tau\right|^{2}+\right.$
$+C^{2}(t_{k+1}-t_{k})+\sup\limits_{t_{k}\leq
t<t_{k+1}}L^{2}\left|\int\limits_{t_{k}}^{t}\left|x(\tau)\right|dw(\tau)\right|^{2}+C^{2}(t_{k+1}-t_{k})+$
$+\sup\limits_{t_{k}\leq
t<t_{k+1}}\left|\int\limits_{t_{k}}^{t}\int\limits_{\mathbb{R}^{m}}\left|c(\tau,\xi(\tau),x(\tau),u(\tau),z)-c(\tau,\xi(\tau),0,u(\tau),z)\right|\widetilde{\nu}(dz,d\tau)\right|^{2}+$
$+\left.\sup\limits_{t_{k}\leq
t<t_{k+1}}\left|\int\limits_{t_{k}}^{t}\int\limits_{\mathbb{R}^{m}}\left|c(\tau,\xi(\tau),0,u(\tau),z)\right|\widetilde{\nu}(dz,d\tau)\right|^{2}\right].$

Consider the designation

$y(t)=\mathbb{E}\left\\{\sup\limits_{t_{k}\leq
s<t}\left|x(s)\right|^{2}/\mathfrak{F}_{t_{k}}\right\\}.$

Then, according to the last inequality, $y(t)$ satisfies the ratio

$y(t)\leq
7\left[\mathbb{E}\left\\{\left|x(t)\right|^{2}/\mathfrak{F}_{t_{k}}\right\\}+3C^{2}(t_{k+1}-t_{k})+L^{2}((t_{k+1}-t_{k})+8)\cdot\int\limits_{t_{k}}^{t}y(\tau)d\tau.\right]$

Using the Gronwall inequality, we obtain an estimate of

$\mathbb{E}\left\\{\sup\limits_{t_{k}\leq
t<t_{k+1}}\left|x(t)\right|^{2}/\mathfrak{F}_{t_{k}}\right\\}\leq$ $\leq
7\left[\mathbb{E}\left|x(t_{k})\right|^{2}+3C^{2}(t_{k+1}-t_{k})\right]e^{7L^{2}\left((t_{k+1}-t_{k})+8\right)},$

as required to prove.

End of proof of Lemma 1.

Remark 1. We will consider the stability of the trivial solution $x\equiv 0$
of the system (2)-(4) that is, the fulfillment of (6) when $C=0$ [5], [19],
[20].

Theorem 1. Let:

1) interval lengths $[t_{k},t_{k+1})$ do not exceed $\Delta>0$, i.e.
$\left|t_{k+1}-t_{k}\right|\leq\Delta,k\geq 0;$

2) the Lipschitz condition is satisfied (5);

3) there exist Lyapunov functions $v_{k}(y,h,x),k\geq 0$ such that the
following inequality holds true

$(lv_{k})(y,h,x)\leq 0,k\geq 0.$ (10)

Then the system of random structure (2)-(4) is stable in probability on the
whole.

Remark 2. It should be noted that if condition 1 is not satisfied, the number
of jumps (3) is finite and the system (2)-(4) turns into a system without
jumps after $\max t_{k}$. In this case, we can use the results presented in
[7].

Proof of Theorem 1. The conditional expectation of the Lyapunov function is

${\it\mathbf{E}}\left\\{{v_{k+1}(\xi(t_{k+1}),\eta_{k+1},x(t_{k+1}))\mathord{\left/{\vphantom{v_{k+1}(\xi(t_{k+1}),\eta_{k+1},x(t_{k+1})){\mathfrak{F}}_{t_{k}}}}\right.\kern-1.2pt}{\mathfrak{F}}_{t_{k}}}\right\\}=$
$=\int_{{\it\mathbf{Y}}\times{\it\mathbf{H}}\times{\it\mathbb{R}}^{m}}{\it\mathbf{P}}_{k}((\xi(t_{k}),\eta_{k},x(t_{k}))(du\times
dz\times dl)v_{k+1}(u,z,l)).$ (11)

Then by the definition of the discrete Lyapunov operator $(lv_{k})(y,h,x)$
(see (7)) and from equation (11), taking into account (10), we can get the
following inequality

${\it\mathbf{E}}\left\\{{v_{k+1}(\xi(t_{k+1}),\eta_{k+1},x(t_{k+1}))\mathord{\left/{\vphantom{v_{k+1}(\xi(t_{k+1}),\eta_{k+1},x(t_{k+1}))\mathfrak{F}_{t_{k}}}}\right.\kern-1.2pt}\mathfrak{F}_{t_{k}}}\right\\}=v_{k}(\xi(t_{k}),\eta_{k},x(t_{k}))+$
$+(lv_{k})(\xi(t_{k}),\eta_{k},x(t_{k}))\leq\bar{v}(\left|x(t_{k})\right|).$
(12)

From Lemma 1 and the properties of the function $\bar{v}$, it follows that the
conditional expectation of the left part of the inequality (12) exists.

Using (11), (12), let us write the discrete Lyapunov operator
$(lv_{k})(y,h,x)$,defined on solutions (2)-(4):

$(lv_{k})(\xi(t_{k}),\eta_{k},x(t_{k}))={\it\mathbf{E}}\left\\{{v_{k+1}(\xi(t_{k+1}),\eta_{k+1},x(t_{k+1}))\mathord{\left/{\vphantom{v_{k+1}(\xi(t_{k+1}),\eta_{k+1},x(t_{k+1}))\mathfrak{F}_{t_{k}}}}\right.\kern-1.2pt}\mathfrak{F}_{t_{k}}}\right\\}-$
$-v_{k}(\xi(t_{k}),\eta_{k},x(t_{k}))\leq 0.$ (13)

Then when $k\geq 0$ the following inequality is satisfied

${\it\mathbf{E}}\left\\{{v_{k+1}(\xi(t_{k+1}),\eta_{k+1},x(t_{k+1}))\mathord{\left/{\vphantom{v_{k+1}(\xi(t_{k+1}),\eta_{k+1},x(t_{k+1}))\mathfrak{F}_{t_{k}}}}\right.\kern-1.2pt}\mathfrak{F}_{t_{k}}}\right\\}\leq
v_{k}(\xi(t_{k}),\eta_{k},x(t_{k})).$

This means that the sequence of random variables

$v_{k}(\xi(t_{k}),\eta_{k},x(t_{k}))$

is a supermartingale with respect to $\mathfrak{F}_{t_{k}}$ [6].

Thus, the following inequality holds:

${\it\mathbf{E}}\left\\{v_{N+1}(\xi(t_{N+1}),\eta_{N+1},x(t_{N+1}))\right\\}-{\it\mathbf{E}}\left\\{v_{n}(\xi(t_{n}),\eta_{n},x(t_{n}))\right\\}=$
$=\sum_{k=n}^{N}{\it\mathbf{E}}\left\\{(lv_{k})(\xi(t_{k}),\eta_{k},x(t_{k}))\right\\}\leq
0.$

Since the random variable $\mathop{\sup}\limits_{t_{k}\leq
t<t_{k+1}}\left|x(t)\right|^{2}$ is independent of events of $\sigma$\-
algebra $\mathfrak{F}_{t_{k}}$ [3], then

${\it\mathbf{E}}\left\\{{\mathop{\sup}\limits_{t_{k}\leq
t<t_{k+1}}\left|x(t)\right|^{2}\mathord{\left/{\vphantom{\mathop{\sup}\limits_{t_{k}\leq
t<t_{k+1}}\left|x(t)\right|^{2}\mathfrak{F}_{t_{k}}}}\right.\kern-1.2pt}\mathfrak{F}_{t_{k}}}\right\\}={E}\left\\{\mathop{\sup}\limits_{t_{k}\leq
t<t_{k+1}}\left|x(t)\right|^{2}\right\\},$

i.e., the inequality (9) also holds for the usual expectation

${\it\mathbf{E}}\left\\{\mathop{\sup}\limits_{t_{k}\leq
t<t_{k+1}}\left|x(t)\right|^{2}\right\\}\leq
7\left[\mathbb{E}\left|x\right|^{2}\right]e^{7L^{2}\left(\Delta+8\right)}$

at $C=0$, assuming that the stability of the trivial solution is investigated.

Then

${\it\mathbf{P}}\left\\{\mathop{\sup}\limits_{t\geq
0}\left|x(t)\right|>\varepsilon_{1}\right\\}=$
$={\it\mathbf{P}}\left\\{\mathop{\sup}\limits_{n\in\mathbb{N}}\mathop{\sup}\limits_{t_{n-1}\leq
t<t_{n}}\left|x(t)\right|>\varepsilon_{1}\right\\}\leq$
$\leq{\it\mathbf{P}}\left\\{\mathop{\sup}\limits_{n\in\mathbb{N}}7e^{7L^{2}\left(\Delta+8\right)}\left|x(t_{n-1})\right|>\varepsilon_{1}\right\\}\leq$
$\leq{\it\mathbf{P}}\left\\{\mathop{\sup}\limits_{n\in\mathbb{N}}\left|x(t_{n-1})\right|>\frac{\varepsilon_{1}}{7}e^{-7L^{2}\left(\Delta+8\right)}\right\\}\leq$
$\leq{\it\mathbf{P}}\left\\{\mathop{\sup}\limits_{n\in\mathbb{N}}v_{n-1}(\xi(t_{n-1}),\eta_{n-1},x(t_{n-1}))\geq\bar{v}(\frac{\varepsilon_{1}}{7}e^{-7L^{2}\left(\Delta+8\right)})\right\\}$
(14)

If $\sup\left|x(t_{k})\right|\geq r$, then based on the definition of the
Lyapunov function the inequality is fulfilled

$\mathop{\sup}\limits_{k\geq
0}v_{k}(\xi(t_{k}),\eta_{k},x(t_{k}))\geq\inf\limits_{k\geq
0,y\in{\it\mathbf{Y}},{h\in{\it\mathbf{H}},\left|x\right|\geq
r}}v_{k}(y,h,x)=\bar{v}(r).$ (15)

Using the inequality for non-negative supermartingales [4], [6], we obtain an
estimate of the right-hand side (14):

${\it\mathbf{P}}\left\\{\mathop{\sup}\limits_{n\in\mathbb{N}}v_{n-1}(\xi(t_{n-1}),\eta_{n-1},x(t_{n-1}))\geq\bar{v}(\frac{\varepsilon_{1}}{7}e^{-7L^{2}\left(\Delta+8\right)})\right\\}\leq$
$\leq\frac{1}{\bar{v}(\frac{\varepsilon_{1}}{7}e^{-7L^{2}\left(\Delta+8\right)})}v_{0}(y,h,x)\leq\frac{\bar{v}(\left|x\right|)}{\bar{v}(\frac{\varepsilon_{1}}{7}e^{-7L^{2}\left(\Delta+8\right)})}.$
(16)

Given the inequality (14), the inequality (16) makes it possible to guarantee
that the inequality (8) of stability in probability for the whole system
(2)-(4).

End of proof of Theorem 1.

##  4 Stabilization

The optimal stabilization problem is that for an SDE (2) with switches (3),
one should construct such a control $u(t,x(t))$ that the unperturbed motion
$x(t)\equiv 0$ of the system (2)-(4) is stable in probability on the whole.

It is assumed that the control $u$ will be determined by the full feedback
principle. In addition, the condition of continuity of $u(t)$ on $t$ in the
range

$t\geq 0,~{}x\in\mathbb{R}^{m},~{}y\in\mathbf{Y},~{}h\in\mathbf{H}.$ (17)

for every fixed $\xi(t)=y\in\mathbf{Y}$ and $\eta_{k}=h\in\mathbf{H}$.

It is also assumed that the structure in which the system is at time $t\geq
0$, which is independent of the Markov chain $\eta_{k}$ ($k\geq 0$ corresponds
to time $t_{k}\in K$), is known.

Obviously, there is an infinite set of controls. The only control should be
chosen from the requirement of the best quality of the process, which is
expressed in the form of the minimization condition of the functional

${\it I}_{u}(y,h,x_{0}):=$ $:=\sum_{k=0}^{\infty}\int_{t_{k}}^{\infty}{\it
E}\left\\{W(t,x(t),u(t))/\xi(0)=y,\eta_{0}=h,x(0)=x_{0}\right\\}dt,$ (18)

where $W(t,x,u)\geq 0$ – is a non-negative function defined in the region
$t\geq 0,x\in{\it\mathbb{R}}^{m}$, $u\in{\it\mathbb{R}}^{r}$.

The algorithm for calculating the (18) functional for a given control $u(t,x)$
is as follows.

A) Find the trajectory $x(t)$ with an SDE (2) at $u\equiv u(t,y,h,x)$, for
example by the Euler-Maruyama method [8].

B) Substitute $x(t)$, $\xi(t)$, $u(t)=u(t,x(t))$ into the functional (18).

C) Calculate the value of the function (18) by statistical modeling (Monte
Carlo).

D) The problem of choosing the functional $W(t,x,u)$, which determines the
estimate ${\it I}_{u}$ and the quality of the process $x(t)$ as a strong
solution of the SDE (2), is related to the specific features of the problem
and the following three conditions can be identified:

  1. 1.

the minimization conditions of the functional (18) must ensure that the strong
solution $x(t)$ of the SDE (2) fades fast enough on average with high
probability;

  2. 2.

the value of the integral should satisfactorily estimate the computation time
spent on generating the control $u(t)$;

  3. 3.

the value of the quality functional should satisfactorily estimate the
computation time spent on forming the control $u(t)$;

  4. 4.

The functional $W(t,x,u)$ must be such that the solution of the stabilization
problem can be constructed.

Remark 3. For linear SDE (2), in many cases the quadratic form with respect to
the variables $x,u$ is satisfactory

$W(t,x,u)=x^{T}M(t)x+u^{T}D(t)u,$ (19)

where $M(t)$ – is a symmetric non-negative matrix of size $m\times m$; $D(t)$
– is a positively determined matrix of the size $r\times r$ for all $t\geq 0$.

Remark 4. Note that according to the feedback principle, $M(t)$ and $D(t)$
depend indirectly on the values of $\xi(t)$ and $\eta_{k}$. Therefore, in the
examples below, we will calculate the values of $M(t)$ and $D(t)$ for fixed
$\xi(t)$ and $\eta_{k}$.

The value ${\it I}_{u}$ in the case of the quadratic form of the variables $x$
and $u$ evaluates the quality of the transition process quite well on average.
The presence of the term $u^{T}Du$ and the minimum condition simultaneously
limit the amount of the control action $u\in{\it\mathbb{R}}^{r}$.

Remark 5. If the jump condition of the phase trajectory is linear, then the
solution of the stabilization problem belongs to the class of linear on the
phase vector $x\in{\it\mathbb{R}}^{m}$ controls $u(t,x)$. Such problems are
called linear-quadratic stabilization problems.

Definition 4. The control $u^{0}(t)$, which satisfies the condition

${\it I}_{u^{0}}(y,h,x_{0})=\min{\it I}_{u}(y,h,x_{0}),$

where the minimum should be searched for all controls continuous variables $t$
and $x$ at each $\xi(0)=y\in{\it\mathbf{Y},}\eta_{0}=h\in{\it\mathbf{H}}$, let
us call it optimal in the sense of optimal stabilization of the strong
solution $x\in{\it\mathbb{R}}^{m}$ of the system (2)-(4).

Theorem 2. Let the system (2)-(4) have a scalar function $v^{0}(t_{k},y,h,x)$
and $r$-vector function $u^{0}(t,y,h,x)\in{\it\mathbb{R}}^{r}$ in the region
(17) and fulfill the conditions:

  1. 1.

the sequence of the functions $v_{k}^{0}(y,h,x)\equiv v^{0}(t_{k},y,h,x)$ is
the Lyapunov functional;

  2. 2.

the sequence of $r$-measured functions-control

$u_{k}^{0}(y,h,x)\equiv u^{0}(t_{k},y,h,x)\in\it\mathbb{R}^{r};$ (20)

is measurable in all arguments, where $0\leq t_{k}<t_{k+1},k\geq 0$;

  3. 3.

sequence of functions from the criterion (18) by $x\in{\it\mathbb{R}}^{m}$ is
positive definite, i.e. for $\forall t\in[t_{k},t_{k+1}),k\geq 0$,

$W(t,x,u_{k}^{0}(y,h,x))>0;$ (21)

  4. 4.

the sequence of infinitesimal operators
$(lv_{k}^{0})\left.\right|_{u_{k}^{0}}$, calculated for $u_{{}^{k}}^{0}\equiv
u^{0}(y,h,x)$, satisfies the condition for $\forall t\in[t_{k},t_{k+1})$

$(lv_{k}^{0})\left.\right|_{u_{k}^{0}}=-W(t,x,u_{k}^{0});$ (22)

  5. 5.

the value of $(lv_{k}^{0})+W(t,x,u)$ reaches a minimum at $u=u^{0},k\geq 0$,
i.e.

$(lv_{k}^{0})\left.\right|_{u_{k}^{0}}+W(t,x,u_{k}^{0})=$
$=\mathop{\min}\limits_{u\in{\it\mathbb{R}}^{r}}\\{(lv_{k}^{0})\left.\right|_{u}+W(t,x,u)\\}=0.$
(23)

  6. 6.

the series

$\sum_{k=0}^{\infty}\int_{t_{k}}^{\infty}{\it\mathbf{E}}\left\\{W(t,x(t),u(t))/x(t_{k-1})\right\\}dt<\infty$
(24)

converges.

Then the control $u_{{}^{k}}^{0}\equiv u^{0}(t_{k},y,h,x),k\geq 0$, stabilizes
the solution of the problem (2)-(4). In this case, is hold the equality

$v^{0}(y,h,x_{0})\equiv$
$\equiv\sum_{k=0}^{\infty}\int_{t_{k}}^{\infty}{\it\mathbf{E}}\left\\{W(t,x(t),u(t))/x(t_{k-1})\right\\}dt=$
$=\mathop{\min}\limits_{u\in{\it\mathbb{R}}^{r}}\sum_{k=0}^{\infty}\int_{t_{k}}^{\infty}{\it\mathbf{E}}\left\\{W(t,x(t),u(t))/x_{(}t_{k})\right\\}dt\equiv{\it
I}_{u^{0}}(y,h,x_{0}).$ (25)

Proof of Theorem 2.

I. Stability in probability in the whole of a dynamical system of a random
structure (2)-(4) for $u\equiv u^{0}(t_{k},x),k\geq 0$ immediately follows
from the Theorem 1, since the functionals $v^{0}(y,h,x)$ for anyone
$t\in[t_{k},t_{k+1}),k\geq 0$ satisfy the conditions of this theorem.

II. The equality (25) is obviously also a consequence of the Theorem 1.

III. Proof by contradiction that the stabilization of a strong solution of a
dynamical system of random structure (2)-(4) is controlled by
$u^{0}(t_{k},x),$ $t_{k}\leq t<t_{k+1},k\geq 0$.

Let exist the control $u^{*}(t_{k},x)\neq u^{0}(t_{k},x)$, which, when
substituted into the SDE (2), realizes a solution $x^{*}(t)$ with initial
conditions (3), (4) such that hold the equality

${\it I}_{u^{*}}(y,h,x_{0})\leq{\it I}_{u^{0}}(y,h,x_{0}).$ (26)

The fulfillment of conditions 1)-6) of the Theorem 2 will lead to an
inequality (see (27)) opposite to (26).

From the condition 5) (see (23)) follows the inequality

$(lv_{k}^{0})\left.\right|_{u^{*}}\geq-W(t,x,u^{*}(t,y,h,x)).$ (27)

Averaging (27) over random variables $\\{x^{*}(t),\xi(t),\eta_{k}\\}$ over
intervals $[t_{k},t_{k+1}),k\geq 0$ and integrating over $t$ from $0$ to $T$,
we obtain $n$ inequalities:

${\it\mathbf{E}}{\\{}v^{0}{(}{t_{1},\xi(t_{1}),\eta_{k_{1}},x^{*}(t_{1}))\mathord{\left/{\vphantom{t_{1},\xi(t_{1}),\eta_{k_{1}},x^{*}(t_{1}),)y,h,x_{0}}}\right.\kern-1.2pt}y_{1},\eta_{k_{1}},x^{*}(t_{1})}{\\}-}v^{0}(y,h,x_{0})\geq$
$\geq-\int_{t_{0}}^{t_{1}}{\it\mathbf{E}}\left\\{W(t,x^{*}(t),u^{*}(t))/x_{0}\right\\}dt,$
(28)
${\it\mathbf{E}}{\\{}v^{0}{(}{t_{2},\xi(t_{2}),\eta_{k_{2}},x^{*}(t_{2}))\mathord{\left/{\vphantom{t_{2},\xi(t_{2}),\eta_{k_{2}},x^{*}(t_{2}))y_{1},\eta_{k_{1}},x^{*}(t_{1})}}\right.\kern-1.2pt}y_{1},\eta_{k_{1}},x^{*}(t_{1})}{\\}-}$
${-\\{}v^{0}{(}{t_{1},\xi(t_{1}),\eta_{k_{1}},x^{*}(t_{1}))\mathord{\left/{\vphantom{t_{1},\xi(t_{1}),\eta_{k_{1}},x^{*}(t_{1}))y,h,x_{0}}}\right.\kern-1.2pt}y,h,x_{0}}{\\}}\geq$
$\geq-\int_{t_{1}}^{t_{2}}{\it\mathbf{E}}\left\\{W(t,x^{*}(t),u^{*}(t))/x^{*}(t_{1})\right\\}dt,$
(29)

…

${\it\mathbf{E}}{\\{}v^{0}{(}{t_{n},\xi(t_{n}),\eta_{k_{n}},x^{*}(t_{n}))\mathord{\left/{\vphantom{t_{n},\xi(t_{n}),\eta_{k_{n}},x^{*}(t_{n}))y_{n-1},x^{*}(t_{n-1}),\eta_{k_{n-1}}}}\right.\kern-1.2pt}y_{n-1},\eta_{k_{n-1}},x^{*}(t_{n-1})}{\\}-}$
${-\\{}v^{0}{(}{t_{n-1},\xi(t_{n-1}),\eta_{k_{n-1}},x^{*}(t_{n-1}))\mathord{\left/{\vphantom{t_{n-1},\xi(t_{n-1}),\eta_{k_{n-1}},x^{*}(t_{n-1}))y_{n-2},\eta_{k_{n-2}},x^{*}(t_{n-2})}}\right.\kern-1.2pt}y_{n-2},\eta_{k_{n-2}},x^{*}(t_{n-2})}{\\}}\geq$
$\geq-\int_{t_{n-1}}^{t_{n}}{\it\mathbf{E}}\left\\{W(t,x^{*}(t),u^{*}(t))/x^{*}(t_{n-1})\right\\}$
(30)

Taking into account the martingale property of the Lyapunov functions
$v^{0}(t,\xi(t),h,x^{*}(t))$ (see condition 1) of the theorem) due to the
system (2)-(4), i.e. by the definition of a martingale, we have $n$ equalities
with the probability of one:

${\it\mathbf{E}}\\{v^{0}({t_{k},\xi(t_{k}),\eta_{k},x^{*}(t_{k}))/y_{k-1},\eta_{k-1},x^{*}(t_{k-1})\\}}=$
$=v^{0}(t_{k-1},\xi(t_{k-1}),\eta_{k-1},x^{*}(t_{k-1})),k=\overline{1,n}.$
(31)

Substituting (31) into the inequalities (28)-(30), we obtain the inequality

${\it\mathbf{E}}{\\{}v^{0}(t_{n},\xi(t_{n}),\eta_{k_{n}},x^{*}(t_{n}))/{t_{n-1},\xi(t_{n-1}),\eta_{k_{n-1}},x^{*}(t_{n-1})\\}-}v^{0}{(}y,h,x_{0})\geq$
$\geq-\sum_{k=0}^{n}\int_{t_{k}}^{t_{k+1}}{\mathbf{E}}\left\\{W(t,x^{*}(t),u^{*}(t))/x^{*}(t_{k-1})\right\\}dt\geq$

$\geq-\sum_{k=0}^{\infty}\int_{t_{k}}^{\infty}{\it\mathbf{E}}\left\\{W(t,x^{*}(t),u^{*}(t))/x^{*}(t_{k-1})\right\\}dt.$
(32)

According to the assumption (26) it follows that for $t_{n}\to\infty$ the
integrals on the right-hand side (32) converge and, taking into account the
convergence of the series (24) (condition 6)), we have the inequality:

$v^{0}{(}y,h,x_{0})={\it I}_{u^{0}}(y,h,x_{0})\leq$
$\leq\sum_{k=0}^{\infty}\int_{t_{k}}^{\infty}{\it\mathbf{E}}\left\\{W(t,x^{*}(t),u^{*}(t))/x^{*}(t_{k-1})\right\\}dt=$
$={\it I}_{u^{*}}(y,h,x_{0}).$ (33)

Indeed, from the convergence of the series (32), under condition 6) it follows
that the integrands in (33) tend to zero as $t\to\infty$. In this way,
$\mathop{\lim}\limits_{n\to\infty}{\it\mathbf{E}}{\\{}v^{0}{(}t_{n},y_{n},\eta_{k_{n}},x^{*}(t_{n}){\\}}={0}$.

Note that it makes sense to consider natural cases when from the condition

${\it\mathbf{E}}{\\{}W{\\}}\mathop{\to}\limits_{{t}\to\infty}{0}$

it follows that
${\it\mathbf{E}}{\\{}v^{{0}}{\\}}\mathop{\to}\limits_{{t}\to\infty}{0}$.

Thus, the inequality (33) contradicts the inequality (26). This contradiction
proves the statement about the optimality of the control $u^{0}(t_{k},x),k\geq
0$.

End of proof of Theorem 2.

In case when the Markov process with a finite number of states $\xi(t_{k})$
admits a conditional expansion of the conditional transition probability

${\it\mathbf{P}}\\{\omega:\xi(t+\Delta t)=y_{j}/\xi(t)=y_{i},y_{i}\neq
y_{j}\\}=$ $=q_{ij}(t)\Delta t+o(\Delta t),i,j=\overline{1,N},$ (34)

we obtain an equation that must be satisfied by the optimal Lyapunov functions
$v_{k}^{0}(y,h,x)$ and the optimal control $u_{k}^{0}(t,x),\forall
t\in[t_{k},t_{k+1})$.

Note that according to [12], [14], [15] the weak infinitesimal operator (7)
has the form

$(lv_{k})(y,h,x)=\frac{\partial v_{k}(y,h,x)}{\partial t}+(\nabla
v_{k}(y,h,x),a(t,y,x,u))+$
$+\frac{1}{2}Sp(b^{T}(t,y,x,u)\cdot\nabla^{2}v_{k}(y,h,x)\cdot b(t,y,x,u))+$
$+\int\limits_{{\it\mathbb{R}}^{m}}[v_{k}(y,h,x+c(t,y,x,u,z))-v_{k}(y,h,x)-(\nabla
v_{k}(y,h,x))^{T}\cdot c(t,y,x,u,z)]\Pi(dz)+$ $+\sum\limits_{j\neq
i}^{N}[\int\limits_{{\it\mathbb{R}}^{m}}v_{j}(t,x)p_{ij}(t,z/x)dz-
v_{i}(t,x)]q_{ij},$ (35)

$(\cdot,\cdot)$ is a scalar product, $\nabla v_{k}=\left(\frac{\partial
v_{k}}{\partial x_{1}},...,\frac{\partial v_{k}}{\partial x_{m}}\right)^{T}$,
$\nabla^{2}v_{k}=\left[\frac{\partial^{2}v_{k}}{\partial x_{i}\partial
x_{j}}\right]_{i,j=1}^{m},k\geq 0$, ”$T$” stands for a transposition, $Sp$ is
a trace of matrix, $p_{ij}(t,z/x)$ is a conditional probability density

$P{x(\tau)\in[z,z+dz]/x(\tau-0)=x}=p_{ij}(\tau,z/x)dz+o(dz)$

assuming $\xi(\tau-0)=y_{i},~{}\xi(\tau)=y_{j}$.

Taking into account the formula (35), the first equation for $v^{0}$ can be
obtained by putting the expression for the averaged infinitesimal operator
$\left.(lv_{k}^{0})\right|_{u^{*}}$ [7] into the left side of (23).

Then the desired equation at the points $(t_{k},y_{j},\eta_{k},x)$ has the
form

$\frac{\partial v_{k}^{0}}{\partial t}+\left(\left(\frac{\partial
v_{k}^{0}}{\partial x}\right)^{T}\cdot
a(t,y,x,u)\right)+\frac{1}{2}Sp\left(\left(b^{T}(t,y_{i},x)\cdot\frac{\partial^{2}v_{k}^{0}}{\partial
x^{2}}\cdot b(t,y_{i},x)\right)\right)+$
$+\int\limits_{{\it\mathbb{R}}^{m}}[v_{k}^{0}(\cdot,\cdot,x+c(t,y,x,u,z))-v_{k}^{0}-(\frac{\partial
v_{k}^{0}}{\partial x})^{T}\cdot c(t,y,x,u,z)]\Pi(dz)+$ $+\sum_{j\neq
i}^{l}\left[\int_{-\infty}^{+\infty}v_{j}^{0}(y_{j},h,x_{j})p_{ij}\left(t,z/x\right)dz-
v_{i}^{0}(y_{i},h,x)\right]q_{ij}\left(t\right)dt+$ $+W(t,x,u)=0.$ (36)

The second equation for optimal control $u_{k}^{0}(t,y,h,x)$ we obtain from
(36) by differentiation with respect to the variable $u$, since $u=u^{0}$
delivers the minimum of the left side of (36)

$\left.\left[\left(\frac{\partial v^{0}}{\partial
x}\right)^{T}\cdot\left(\frac{\partial a}{\partial
u}\right)+\left(\frac{\partial W}{\partial
u}\right)^{T}\right]\right|_{u=u_{k}^{0}}=0,$ (37)

where $\frac{\partial a}{\partial u}$ – $m\times r$-matrix of Jacobi, stacked
with elements $\left\\{\frac{\partial a_{n}}{\partial
u_{s}},n=\overline{1,m},s=\overline{1,r}\right\\};$$\left(\frac{\partial
W}{\partial u}\right)\equiv\left(\frac{\partial W}{\partial
u_{1}},...,\frac{\partial W}{\partial u_{r}}\right),k\geq 0$.

Thus, the problem of optimal stabilization, according to the Theorem 2,
consists in solving of a complex nonlinear system of equations (23) with
partial derivatives to determine the unknown Lyapunov functions
$v_{ik}^{0}\equiv v_{k}^{0}(y,h,x),i=\overline{1,l},k\geq 0$.

Note that this system is obtained by eliminating the control
$u_{k}^{0}=u^{0}(t,y,h,x)$ from the equation (36), (37).

It is quite difficult to solve such a system, therefore, we will further
consider linear stochastic systems for which convenient solution schemes can
be constructed.

##  5 Stabilization of linear systems

Consider a controlled stochastic system defined by a linear Ito’s SDE with
Markov parameters and Poisson perturbations

$dx(t)=[A(t-,\xi(t-))x(t-)+B(t-,\xi(t-))u(t-)]dt+\sigma(t-,\xi(t-))x(t-)dw(t)+$
$\int\limits_{\mathbb{R}^{m}}c(t-,\xi(t-),u(t-),z)x(t-)\widetilde{\nu}(dz,dt),~{}t\in\mathbb{R}_{+}\backslash
K,$ (38)

with Markov switching

$\Delta
x(t)\Big{|}_{t=t_{k}}=g(t_{k}-,\xi(t_{k}-),\eta_{k},x(t_{k}-)),~{}~{}~{}t_{k}\in
K=\\{t_{n}\Uparrow\\}$ (39)

for $\lim\limits_{n\rightarrow+\infty}t_{n}=+\infty$ and initial conditions

$x(0)=x_{0}\in\mathbb{R}^{m},~{}\xi(0)=y\in\mathbf{Y},\eta_{0}=h\in\mathbf{H}.$
(40)

Here $A,B,\sigma,C$ are piecewise continuous integrable matrix functions of
appropriate dimensions.

Let us assume that the conditions for the jump of the phase vector
$x\in{\it\mathbb{R}}^{m}$ at the moment $t=t^{*}$ of the change in the
structure of the system due to the transition $\xi(t^{*}-)=y_{i}$ in
$\xi(t^{*})=y_{j}\neq y_{i}$ are linear and given in the form

$x(t^{*})=K_{ij}x(t^{*}-)+\sum_{s=1}^{N}\xi_{s}Q_{s}x(t^{*}-),$ (41)

where $\xi_{s}:=\xi_{s}(\omega)$ – independent random variables for which
${\it\mathbf{E}}\xi_{s}=0,{\it\mathbf{E}}\xi_{s}^{2}=1$, $K_{ij}$ and $Q_{s}$
– given $(m\times m)$-matrices.

Note that the equality (41) can replace the general jump conditions [14]:

\- the case of non-random jumps will be at $Q_{s}=0$, i.e.

$x(t^{*})=K_{ij}x(t^{*}-);$

\- continuous change of the phase vector means that $Q_{s}=0$,
$K_{ij}=A_{ij}=I$ (identity $(m\times m)$-matrix).

The quality of the transition process will be estimated by the quadratic
functional

${\it I}_{u}(y,h,x_{0}):=$ (42)
$:=\sum_{k=0}^{\infty}\int_{t_{k}}^{\infty}{\it\mathbf{E}}\left\\{x^{T}(t)M(t)x(t)+u^{T}(t)D(t)u(t)\right./\left.y,h,x_{0}\right\\}dt,$

where $M(t)\geq 0,D(t)>0$ – symmetric matrices of dimensions $(m\times m)$ and
$(r\times r)$ respectively.

According to the Theorem 2 we need to find optimal Lyapunov functions
$v_{k}^{0}(y,h,x)$ and a control $u_{k}^{0}(t,x)$ for $\forall
t\in[t_{k},t_{k+1}),t_{k}\in K,k=0,1,2,...$.

The optimal Lyapunov functions are sought in the form

$v_{k}^{0}(y,h,x)=x^{T}G(t,y,h)x,$ (43)

where $G(t,y,h)$ – positive-definite symmetric matrix of the size $(m\times
m)$.

Everywhere below, when $\xi(t)$ describes a Markov chain with a finite number
of states ${\it\mathbf{Y}}=\\{y_{1},y_{2},...,y_{l}\\}$, and $\eta_{k},k\geq
0$, – a Markov chain with values $h_{k}$ in metric space ${\it\mathbf{H}}$ and
with transition probability at the $k$-th step ${\it\mathbf{P}}_{k}(h,G)$, we
introduce the following notation:

$A_{i}(t):=A(t,y_{i}),~{}B_{i}(t):=B(t,y_{i}),~{}\sigma_{i}(t):=\sigma(t,y_{i}),~{}C_{i}(t,z):=C(t,y_{i},z),$
$G_{ik}(t):=G(t,y_{i},h_{k}),~{}~{}v_{ik}:=v(y_{i},h_{k},x).$

Let us substitute the functional (43) in equations (36) and (38) to find the
an optimal Lyapunov function $v_{k}^{0}(y,h,x)$ and an optimal control
$u_{k}^{0}(t,x)$ for $\forall t\in[t_{k},t_{k+1})$ and, given the form of a
weak infinitesimal operator (35), we obtain:

$x^{T}(t)\frac{dG_{ik}(t)}{dt}x(t)+2[A_{i}(t)x(t)+B_{i}(t)u(t)]G_{ik}(t)x(t)+$
$+Sp(x^{T}(t)\sigma_{i}^{T}(t)G_{ik}(t)\sigma_{i}(t)x(t))+\int\limits_{\mathbb{R}^{m}}x^{T}(t)C_{i}^{T}(t,z)G_{ik}(t)C_{i}(t,z)x(t)\Pi(dz)+$
$+x^{T}(t)\sum\limits_{j\neq
i}^{N}\left[K_{ij}^{T}G_{ik}(t)K_{ij}+\sum\limits_{s=0}^{l}Q_{s}^{T}G_{ik}(t)Q_{s}-G_{ik}(t)\right]q_{ij}x(t)+$
$+x^{T}(t)M_{ik}(t)x(t)+u^{T}(t)D_{ik}(t)u(t)=0,$ (44)

$2x^{T}(t)G_{ik}(t)B_{i}(t)+2u^{T}(t)D_{ik}(t)=0.$ (45)

Note that the partial derivative with respect to $u$ of the operator $(lv)$ is
equal to zero, which confirms the conjecture about constructing an optimal
control that does not depend on switching (39) for the system (40).

From (45) we find an optimal control for $\xi(t)=y_{i}$, when switching (39)
$\eta_{k}=h_{k},k\geq 0,$

$u_{ik}^{0}(t,x)=-D_{ik}^{-1}(t)B_{i}^{T}(t)G_{ik}(t)x(t).$ (46)

Given the matrix equality

$2x^{T}(t)G_{ik}(t)A_{i}(t)x=x^{T}(t)(G_{ik}(t)A_{i}(t)+A_{i}^{T}(t)G_{ik}(t))x(t),$

excluding $u_{ik}^{0}$ from (44) and equating to zero the resulting matrix of
a quadratic form, we can obtain a system of matrix differential equations of
Riccati type for finding the matrices $G_{ik}(t)$, where $i=1,2,...,l$, $k\geq
0$, corresponds to the interval $[t_{k},t_{k+1})$:

$\frac{dG_{ik}(t)}{dt}+G_{ik}(t)A_{i}(t)-B_{i}(t)D_{ik}^{-1}(t)B_{i}^{T}(t)G_{ik}(t))+$
$+Sp(\sigma_{i}^{T}(t)G_{ik}(t)\sigma_{i}(t))+\int\limits_{\mathbb{R}^{m}}C_{i}^{T}(t,z)G_{ik}(t)C_{i}(t,z)\Pi(dz)+$
$+\sum\limits_{j\neq
i}^{N}\left[K_{ij}^{T}G_{ik}(t)K_{ij}+\sum\limits_{s=0}^{l}Q_{s}^{T}G_{ik}(t)Q_{s}-G_{ik}(t)\right]q_{ij}+M_{ik}(t)=0,$
(47)

$\lim_{t\to\infty}G_{ik}(t)=0,i=\overline{1,N},k\geq 0.$ (48)

Thus, we have received the following statement.

Theorem 3. Let the system of matrix equations (47), (48) has positive-definite
solutions of the order $(m\times m)$

$G_{1k}(t)>0,G_{2k}(t)>0,...,G_{lk}(t)>0.$

Then the control (46) gives a solution to the problem of optimal stabilization
of the system (38)-(40) with jump condition (41) and the criterion of
optimality (42).

##  6 Stabilization of autonomous systems

Consider the case of an autonomous system that is given by the SDE

$dx=[A(\xi(t))x+B(\xi(t))u]dt+\sigma(\xi(t))xdw(t)+C(\xi(t))xdN(t),~{}t\in\mathbb{R}_{+}\backslash
K,$ (49)

with Markov switching (39) and initial conditions (40). Here
$x\in{\it\mathbb{R}}^{m}$, $u\in{\it\mathbb{R}}^{r}$, $A(y)$, $B(y)$,
$\sigma(y)$, $C(y)$ are known matrix functions defined on the set
${\it\mathbf{Y}}=\\{y_{1},y_{2},...,y_{k}\\}$ of possible values of the Markov
chain $\xi$. $N(t),t\geq 0$ – Poisson process with intensity $\lambda$ [3].

In the case of phase vector jumps (41) and the quadratic quality functional
(42) the system (47), (48) for finding unknown matrices
$G_{ik},i=\overline{1,N},k\geq 0,$ will take the form

$G_{ik}A_{i}+A_{i}^{T}G_{ik}-B_{i}D_{ik}^{-1}B_{i}^{T}G_{ik}+\sigma_{i}^{T}G_{ik}\sigma_{i}+$
$+\lambda C_{i}^{T}G_{ik}C_{i}+$ $+\sum\limits_{j\neq
i}^{N}\left[K_{ij}^{T}G_{ik}K_{ij}+\sum\limits_{s=0}^{l}Q_{s}^{T}G_{ik}Q_{s}-G_{ik}\right]q_{ij}+M_{ik}=0,i=\overline{1,N},k\geq
0.$ (50)

Remark 6. Note that any differential system written in the normal form (such
as the system (38), where the dependence of $x$ on $t$ is explicitly
indicated) can be reduced to an autonomous system by increasing the number of
unknown functions (coordinates) by one.

###  6.1 Small parameter method for solving the problem of the optimal
stabilization

The possibility of algorithmic solution of the problem of optimal
stabilization of a linear autonomous system of random structure (43), (39)
(40) is achieved by introducing a small parameter [7]. There are two ways to
introduce the small parameter:

Case I. Transition probabilities $y_{i}\to y_{j}$ of Markov chains $\xi$ are
small, i.e. the transition intensities $q_{ij}$ due to the small parameter
$\varepsilon>0$ can be represented as

$q_{ij}=\varepsilon r_{ij}.$ (51)

Case II. Small jumps of the phase vector $x(t)\in{\it R}^{m}$, i.e. matrices
$K_{ij}$ and $Q_{s}$ from (41), should be presented in the form

$K_{ij}=I+\varepsilon K_{ij};~{}Q_{s}=\varepsilon Q_{s}.$ (52)

In these cases, we will search the optimal Lyapunov function
$v_{k}^{0}(y,x,h),k\geq 0$, in the form of a convergent power series with a
base $\varepsilon>0$

$v_{k}^{0}(y,h,x)=x^{T}\sum_{r=0}^{\infty}\varepsilon^{r}G^{(r)}(y,h)x.$ (53)

According to (46), the optimal control $u^{0}$ should be sought in the form of
a convergent series

$u_{k}^{0}(y,h,x)=-[D^{-1}(y)B^{T}(y)\sum_{r=0}^{\infty}\varepsilon^{r}G^{(r)}(y,h)]x.$
(54)

Case I.  Let us substitute the series (53), (54), taking into account (51),
into (44):

$G_{ik}A_{i}+(A_{i})^{T}G_{ik}-B_{i}D_{ik}^{-1}B_{i}^{T}G_{ik}+\sigma_{i}^{T}G_{ik}\sigma_{i}+$
$+\lambda C_{i}^{T}G_{ik}C_{i}+$ $+\sum\limits_{j\neq
i}^{l}K_{ij}^{T}G_{ik}K_{ij}+\sum_{s=1}^{N}Q_{s}^{T}(G_{ik}Q_{s}-G_{ik})\varepsilon
r_{ij}+C_{ik}=0;i=\overline{1,l},k\geq 0.$

Equating the coefficients at the same powers of $\varepsilon>0$, we get:

$A_{i}^{T}G_{i}^{(0)}+G_{ik}^{(0)}A_{i}-B_{i}D_{ik}^{-1}B_{ik}^{T}G_{ik}^{(0)}+$
$+\sigma_{ik}^{T}G_{ik}^{(0)}\sigma_{ik}+\lambda
C_{i}^{T}G_{ik}^{(0)}C_{i}+M_{ik}=0,i=\overline{1,l},k\geq 0,$ (55)
$\begin{array}[]{l}{\tilde{A}_{ik}^{T}G_{ik}^{(r)}+G_{ik}^{(r)}\tilde{A}_{ik}+\sigma_{i}^{T}G_{ik}^{(r)}\sigma_{i}+\lambda
C_{i}^{T}G_{ik}^{(0)}C_{i}=}\\\ {=-\sum_{j\neq
i}^{l}(K_{ik}^{T}G_{ik}^{(r-1)}K_{ij}+\sum_{s=1}^{N}Q_{s}^{T}G_{ik}^{(r-1)}Q_{s}-G_{ik}^{(r-1)})r_{ij}+}\end{array}$
$+\sum_{q=1}^{r-1}B_{i}D_{ik}^{-1}B_{i}^{T}G_{ik}^{(r-q)},$ (56)
$r>1;\tilde{A}_{ik}\equiv
A_{i}-B_{i}D_{ik}^{-1}B_{i}^{T}G_{ik}^{(0)},i=\overline{1,l},k\geq 0.$

Note that the system (55) consists of independent matrix equations which, for
fixed $i=1,2,...,l$, give a solution to the problem of optimal stabilization
of the system

$dx(t)=(A_{i}x(t)+B_{i}u(t))dt+\sigma_{i}x(t)dw(t)+C_{i}x(t)dN(t),$ (57)

with the quality criterion

${\it I}_{u}(y,h,x_{0})=\sum_{k=0}^{\infty}\int_{t_{k}}^{\infty}{\it
E}\\{x^{T}(t)M_{ik}x(t)+u^{T}(t)D_{ik}u(t)/x_{0}\\}dt,$
$i=\overline{1,l},k\geq 0,M_{ik}>0,D_{ik}>0.$ (58)

A necessary and sufficient condition for the solvability of the system (55) is
the existence of linear admissible control in the system (57), which provides
exponential stability in the mean square of the unperturbed motion of this
system [5].

Let us assume that the system of matrix quadratic equations (55) has a unique
positive definite solution $G_{ik}^{(0)}>0,i=\overline{1,l},k\geq 0$.

Equation (56) to find $G_{ik}^{(r)}>0,r\geq 1,k\geq 0$ are linear, so they
have a unique solution for fixed $i=\overline{1,l},k\geq 0,r\geq 1$ and any
matrices that are on the right side of (56).

Indeed, the system

$dx(t)=\tilde{A}_{ik}x(t)dt+\sigma_{i}x(t)dw(t)+C_{i}x(t)dN(t)$ (59)

obtained by closure the system (57) with the optimal control

$u_{k}^{0}=-D_{ik}^{-1}B_{ik}^{T}G_{ik}^{(0)}x(t),$

which provides exponential stability in the mean square. Then there is a
unique solution to the system (56). Note that in the linear case for
autonomous systems, the asymptotic stability is equivalent to the exponential
stability [22]. Consider a theorem, which originates from the results of this
work.

Theorem 4. If a strong solution $x(t)$ of the system (57) is exponentially
stable in the mean square, then there exist Lyapunov functions
$v_{k}(y,h,x),k\geq 0$, which satisfy the conditions:

$c_{1}\left\|x\right\|^{2}\leq v_{k}(y,h,x)\leq c_{1}\left\|x\right\|;$
$\frac{d{\it E}[v_{k}]}{dt}\leq-c_{3}\left\|x\right\|^{2}.$

Thus, the system of matrix equations (55), (56) allows us to find consistently
the coefficients $G_{ik}^{(r)}>0$ of the corresponding series (53), (54),
starting with a positive solution $G_{ik}^{(0)}>0,i=\overline{1,l},k\geq 0$ of
the system (55).

The next step is to prove the convergence of the series (53), (54). Without
the loss of generality, we simplify notations by fixing $k\geq 0$. Denote
$L_{r}:=\mathop{\max}\limits_{\begin{array}[]{l}{i=\overline{1,l},}\\\ {k\geq
0}\end{array}}\left\|G_{i}^{(r)}\right\|$. Then from (56) it follows that
there is a constant $c>0$, that for any $r>0$ the following estimate is
correct

$L_{r}\leq c\left[\sum_{q=1}^{r-1}L_{q}L_{r-q}+L_{r-1}\right].$ (60)

Next, we use the method of majorant series.

Consider the quadratic equation

$\rho^{2}+(a+\varepsilon)\rho+b=0,$ (61)

where the coefficients $a,b$ are chosen such that the power series expansion
of one of the roots of this equation is a majorant series for (53).

We have

$\rho_{1,2}=-\frac{a+\varepsilon}{2}\pm\sqrt{\frac{(a+\varepsilon)^{2}}{4}-b}=\sum_{r=0}^{\infty}\varepsilon^{r}\rho_{r}.$
(62)

Let us substitute (62) into (61), and equate coefficients at equal powers of
$\varepsilon$. Then we get an expression for $\rho_{r}$ through
$\rho_{0},...,\rho_{r-1}$:

$\rho_{r}=-\frac{1}{2\rho_{0}+a}\left[\sum_{q=1}^{r-1}\rho_{q}\rho_{r-q}+\rho_{r-1}\right],$
(63)

where $\rho_{0}$ should be found from the equation

$\rho_{0}^{2}+a\rho_{0}+b=0.$ (64)

Comparing (60) and (63), we see that the series (62) will be major for (53),
if we consider

$c=-\frac{1}{2\rho_{0}+a}>0;~{}\rho_{0}=L_{0}>0.$

Thus, the values of the coefficients $a,b$ in the equation (61) are

$\begin{array}[]{l}{a=-\left[\frac{1}{c}+2L_{0}\right]<0;}\\\
{b=\frac{L_{0}}{c}+L_{0}^{2}>0.}\end{array}$

Using the known $a$ and $b$ from (61) we find that the majorant series for
(53) will be the expansion one of the roots of (61). This root is such that
its values are determined by

$\rho_{0}=L_{0}=-\frac{a}{2}-\sqrt{\frac{a^{2}}{4}-b}.$

Convergence of the series (53) for $v_{k}^{0}(y,h,x)$ follows from the obvious
inequality

$\left\|\sum_{r=0}^{\infty}\varepsilon^{r}G^{(r)}(y,h)\right\|\leq\sum_{r=0}^{\infty}L_{r}\varepsilon^{r}.$

Thus, we have proved the assertion which is formalized below as the Theorem 5:

Theorem 5. Let:

  1. 1.

for $\forall i=\overline{1,l},k\geq 0$ the system (57) has a linear admissible
control;

  2. 2.

transition intensities $q_{ij}$ of a homogeneous Markov chain $\xi$ satisfy
the condition (51).

Then

  1. 1.

there is a unique solution to the problem of optimal stabilization of the
system (43), (39), (40) with the jump condition (41) of the phase vector
$x\in{\it R}^{m}$;

  2. 2.

the optimal Lyapunov function $v_{k}^{0}(y,h,x)$ and optimal control
$u_{k}^{0}(y,h,x)$ determined by convergent series (53), (54), whose
coefficients are found from the corresponding systems (55), (56).

Case II. Let us substitute the series
$G_{ik}=\sum_{r=0}^{\infty}\varepsilon^{r}G_{ik}^{(r)}$ in (44) and equate the
coefficients at the same powers $\varepsilon$. Then, taking into account (52),
we obtain the following equations:

$G_{ik}^{(0)}A_{i}+A_{i}^{T}G_{ik}^{(0)}+\sigma_{i}^{T}G_{ik}^{(0)}\sigma_{i}-B_{i}D_{ik}^{-1}B^{T}_{i}G_{ik}^{(0)}+$
$+\lambda C_{i}^{T}G_{ik}C_{i}+\sum_{j\neq
i}^{l}(G_{jk}^{(0)}-G_{ik}^{(0)})q_{ij}+M_{ik}=0,k\geq 0,$ (65)
$G_{ik}^{(r)}\tilde{A}_{ik}+\tilde{A}_{ik}^{T}G_{ik}^{(r)}+\sigma_{i}^{T}G_{ik}^{(r)}\sigma_{i}+\lambda
C_{i}^{T}G_{ik}C_{i}+\sum_{j\neq
i}^{l}(G_{jk}^{(r)}-G_{ik}^{(r)})q_{ij}={\Phi}_{ik}^{(r)},$ (66)

where $i=\overline{1,l}k\geq 0$,
$\tilde{A}_{ik}=A_{i}-B_{i}D_{ik}^{-1}B_{i}^{T}G_{ik}^{(0)},$

${\Phi}_{ik}^{(r)}=\sum_{q=1}^{r-1}B_{i}D_{ik}^{-1}B_{i}^{T}G_{ik}^{(r-q)}-$
$-\sum_{j\neq
i}^{l}(K_{ij}^{T}G_{jk}^{(r-1)}+G_{jk}^{(r-1)}K_{ij}+K_{ij}^{T}G_{jk}^{(r-2)}K_{ij}+\sum_{s=1}^{N}Q_{s}^{T}G_{jk}^{(r-2)}Q_{s})q_{ij}.$

Based on the above-mentioned, the following theorem is correct:

Theorem 6. Let

  1. 1.

the system of matrix equations (65) has a unique positive definite solution
$G_{ik}^{(0)}>0,i=\overline{1,l};k\geq 0$;

  2. 2.

jumps of the phase vector $x\in{\it R}^{m}$ satisfy the condition (52).

Then the linear-quadratic optimal stabilization problem (43), (39), (40) of
minimizing the functional (42) has a unique solution, which is given in the
form of convergent series (53), (54), and the matrices
$G_{ik}^{(r)},i=\overline{1,l};r\geq 1,k\geq 0$ is the only solution to the
linear matrix equations (66).

##  7 Discussion

In this work, we have obtained sufficient conditions for the existence of an
optimal solution for a stochastic dynamical system with jumps, which transform
the system to a stable one in probability. The second Lyapunov method was used
to investigate the existence of an optimal solution. This method is efficient
both for ordinary differential equations (ODE) and for stochastic differential
equations (SDE). As it can be seen from the proof of the Theorem 2, the
existence of finite bounds for jumps at non-random time moments $t_{m}$
($\lim_{m\to\infty}t_{m}=T^{*}<\infty$) does not impact the stability of the
solution. On the other hand, the fact that $|t_{m+}-t_{m}|>\delta,m\geq 1$ was
used for proving the existence of the optimal control (Theorem 3). This
restriction is also present in the works of other authors. Thus, the goal of
the subsequent work could be to construct an optimal control without the
assumption $|t_{m+}-t_{m}|>\delta,m\geq 1$, which will considerably expand the
scope of the second Lyapunov method.

##  8 Conclusions

In this work, we obtained sufficient conditions for the existence of a
solution to an optimal stabilization problem for dynamical systems with jumps.
We considered the case of a linear system with a quadratic quality functional.
We showed that the problem of designing an optimal control that stabilizes the
system to a stable one in probability, reduces to the problem of solving the
Riccati equations. Additionally, for a linear autonomous system, the method of
a small parameter is substantiated for solving the problem of optimal
stabilization. The obtained solutions can be used to describe a model of a
stock market in economics, biological systems including models of cancer
response to treatment and other complex dynamical systems.

## Acknowledgments

We would like to acknowledge the administrations of the Luxembourg Institute
of Health (LIH) and Luxembourg National Research Fund (FNR) for their support
in organizing scientific contacts between research groups in Luxembourg and
Ukraine.

## References

  * [1] Andreeva, E.A.; Kolmanovskii V.B.; Shaikhet L.E. Control of hereditary systems; Nauka: Moskow, Russia, 1992. (in Russian)
  * [2] Das, A.; Lukashiv, T.O.; Malyk, I.V. Optimal Control Synthesis for Stochastic Dynamical Systems of Random Structure with the Markovian Switchings. Journal of Automation and Information Sciences 2017, 49 (4), 37–47.
  * [3] Doob, J.L. Stochastic Processes; Wiley: New York, USA, 1953.
  * [4] Dynkin, E.B. Markov Processes; Academic Press: New York, USA, 1965.
  * [5] Hasminsky, R.Z. Stability of Systems of Differential Equations under Random Parameter Perturbations; Nauka: Moscow, Russia, 1969. (in Russian)
  * [6] Jacod, J.; Shiryaev, A.N. Limit Theorems for Stochastic Processes. Vols. 1 and 2; Fizmatlit: Moscow, Russia, 1994. (in Russian)
  * [7] Kats, I.Ya. Lyapunov Function Method in Problems of Stability and Stabilization of Random-Structure Systems; Izd. Uralsk. Gosakademii Putei Soobshcheniya: Yekaterinburg, Russia, 1998. (in Russian)
  * [8] Kloeden, P.E.; Platen, E. Numerical Solution of Stochastic Differential Equations. Springer: Berlin, Germany, 1992.
  * [9] Koroliuk, V.S.; Limnios, N. Stochastic systems in merging phase space; World Scientific Publishing Company, 2005.
  * [10] Korolyuk, V.S.; Tsarkov, E.F.; Yasinskii, V.K. Probability, Statistics, and Random Processes. Theory and Computer Practice, Vol. 3, Random Processes. Theory and Computer Practice; Zoloti Lytavry: Chernivtsi, Ukraine, 2009. (in Ukrainian)
  * [11] Koroliouk, V.S.; Samoilenko, I.V. Asymptotic expansion of a functional constructed from a semi-Markov random evolution in the scheme of diffusion approximation. Theory of Probability and Mathematical Statistics 2018, 96, 83–100.
  * [12] Lukashiv, T. One Form of Lyapunov Operator for Stochastic Dynamic System with Markov Parameters. Journal of Mathematics 2016.
  * [13] Lukashiv, T.O.; Malyk, I.V. Stability of controled stochastic dynamic systems of random structure with Markov switches and Poisson perturbations. Bukovinian Math. Journal 2022, 10 (1), 85-–99.
  * [14] Lukashiv, T.O.; Yurchenko, I.V.; Yasinskii, V.K. Lyapunov function method for investigation of stability of stochastic Ito random-structure systems with impulse Markov switchings. I. General theorems on the stability of stochastic impulse systems. Cybernetics and Systems Analysis 2009, 45 (2), 281-–290.
  * [15] Lukashiv, T.O.; Yurchenko, I.V.; Yasinskii, V.K. Lyapunov function method for investigation of stability of stochastic Ito random-structure systems with impulse Markov switchings. II. First-approximation stability of stochastic impulse systems with markov parameters. Cybernetics and Systems Analysis 2009, 45 (3), 464–476.
  * [16] Lyapunov, A.M. The General Problem of Stability of Motion; Gostekhizdat: Moscow, Russia, 1958. (in Russian)
  * [17] Oksendal, B. Stochastic Differential Equations; Springer: New York, USA, 2013.
  * [18] Protter, Philip E. Stochastic Integration and Differential Equations(2nd ed.); Springer: New York, USA, 2004.
  * [19] Sverdan, M.L.; Tsar’kov, E.F. Stability of Stochastic Impulse Systems; RTU: Riga, Latvia, 1994. (in Russian)
  * [20] Skorokhod, A.V. Asymptotic Methods in the Theory of Stochastic Differential Equations; Naukova Dumka: Kyiv, Ukraine, 1987. (in Russian)
  * [21] Tsarkov, Y.F.; Yasinsky, V.K.; Malyk, I.V. Stability in impulsive systems with Markov perturbations in averaging scheme. I. Averaging principle for impulsive Markov systems. Cybernetics and Systems Analysis 2010, 46 (6), 975 - 985.
  * [22] Tsarkov, Y.F.; Yasinsky, V.K.; Malyk, I.V. Stability in impulsive systems with Markov perturbations in averaging scheme. 2. Averaging principle for impulsive Markov systems and stability analysis based on averaged equations. Cybernetics and Systems Analysis 2011, 47 (1), 44–54.
  * [23] Tsarkov, Y.F.; Yasinsky, V.K.; Malyk, I.V. Stability in impulsive systems with Markov perturbations in averaging scheme. 3. Weak convergence of solutions of impulsive systems; Cybernetics and Systems Analysis 2011, 47 (3), 442–458.
  * [24] Samoilenko, I.V.; Malyk, I.V.Convergence of a semi-Markov process and an accompanying Markov process to a Markov process; Ukrainian Mathematical Journal 2010, 62 (5), 774–784.
  * [25] Yasinsky, V.K.; Malyk, I.V. Parametric continuity of solutions to stochastic functional differential equations with Poisson perturbations. Cybernetics and Systems Analysis 2012, 48 (6), 846–860.