# Effect of solar free oscillations on TianQin’s range acceleration noise

Kun Liu, Chengjian Luo, and Xuefeng Zhang  MOE Key Laboratory of TianQin
Mission, TianQin Research Center for Gravitational Physics & School of Physics
and Astronomy, Frontiers Science Center for TianQin, Gravitational Wave
Research Center of CNSA, Sun Yat-sen University (Zhuhai Campus), Zhuhai
519082, China<EMAIL_ADDRESS>
###### Abstract

TianQin is a proposed space-based gravitational-wave detector mission to be
deployed and operated in high Earth orbits. As a sequel to [Zhang et al. Phys.
Rev. D 103, 062001 (2021)], we investigate a type of “orbital noise” in
TianQin’s range acceleration that is caused by gravitational perturbation
associated with solar free oscillations. Frequencies of such oscillations are
typically within TianQin’s measurement band of 0.1 mHz–1 Hz, and the
disturbance level needs careful assessment. By using high-precision orbit
propagation and adding the Sun’s time-variable oblateness $J_{2}$ to detailed
gravity-field models, we examine the effect in the frequency domain and show
that the solar free oscillation noise is expected to be two orders of
magnitude lower than the noise requirement on single links and hence has
little impact on the mission.

##  1 Introduction

TianQin is designed to detect gravitational waves (GWs) in the millihertz
frequency range (0.1 mHz–1 Hz), with a constellation consisting of three
geocentric spacecrafts forming a nearly equilateral triangle set almost
vertically to the ecliptic plane (see figure 1 and basic mission parameters in
table 1) [1]. The detector uses laser interferometry to measure tiny distance
variations between free-floating test masses inside each satellite [2], which
can detect sources such as galactic ultra-compact binaries, stellar-mass black
hole binaries, massive black hole binaries, extreme mass ratio inspirals, and
stochastic GW background [1, 3].

Figure 1:  An illustration of the Sun, the Moon, and three TianQin satellites
in an Earth-centered reference frame (figure reproduced from [4]).

The concept faces challenges from various environmental factors in space,
including but not limited to gravity field, thermal environment, plasma,
micrometeoroids, etc., among which the space gravity-field environment is of
particular interest. The highly-sensitive inter-satellite ranging measurements
can be influenced by the nearby gravity field. At large scales, the arm
lengths of the constellation change with time, which requires orbit
optimization to meet the configuration stability requirements [5]. At small
scales, it affects the geodesic motion of the test masses and also may
interfere with GW signals in the detection frequency band.

A considerable number of interference sources from the Earth-Moon system have
been evaluated in our previous works. Most of the orbital noises are in the
frequency band below $10^{-4}$ Hz [6], and how the effects vary with different
choices of orbital orientations and radii has been discussed in [7].
Nevertheless, we also need to discuss other potential sources which may enter
the detection band. Here our work focuses on the solar free oscillation.

Table 1: TianQin basic mission parameters.

  * Parameter | Value
---|---
Number of spacecraft | $N=3$
Constellation | Equilateral triangle
Type of orbits | Geocentric
Arm length |  $\sim 1.7\times 10^{5}$ km
Position measurement noise | 1 pm/Hz1/2
Residual acceleration noise |  $1\times 10^{-15}$ m/s2/Hz1/2

Solar oscillations can generate long-term stable noise background overlapping
with the frequency band of white dwarf binaries [8, 9]. The work of [8] has
estimated the magnitudes of the external gravitational perturbations that
would be required for the solar oscillations to be observed by LISA [10]. The
estimates use the tidal acceleration formula with all-sky averaging and are
based on observational upper limits and theoretical prediction of mode
energies. To compare some published results, research on the proposed ASTROD
mission has also evaluated some typical solar modes [11], and the strain
amplitudes are estimated to be $10^{-25}$–$10^{-23}$, which are consistent
with the results for LISA [8]. The work shows that ASTROD may have the ability
to separate the solar oscillation signals from the GW background signals [9].
The references [12, 13, 14] further investigated the prospects of observing
combined signals of Newtonian gravity and GWs generated by solar quadrupole
oscillations. Similarly for TianQin in geocentric orbits, we also need to
evaluate the strength of these signals to see if these could affect GW
detection.

The paper aims to simulate and evaluate the range-acceleration amplitude
spectral density (ASD) generated by the solar free oscillation and compare
them with the noise requirement of TianQin. Unlike previous works for LISA, we
will use high precision numerical orbit simulation instead. The paper is
structured as follows. In section 2, we discuss the properties of the solar
free oscillations, as well as the Newtonian and GW contributions to the range
acceleration noise. In section 3, we use a high-precision orbit simulation
program with various force models to obtain the expected noise spectrum of
solar free oscillations, and make a comparison with TianQin’s requirement.
Finally, we present the conclusions in section 4.

##  2 Properties of the Solar Modes

The oscillation in the solar atmosphere was first discovered in the 1960s
[15]. Such photospheric periodic motions have a discrete frequency spectrum
and have been observed in large numbers in the millihertz range [16]. Large-
scale projects such as SOHO (Solar and Heliospheric Observatory) [17], GONG
(Global Oscillations Network Group) [18] and BiSON (The Birmingham Solar
Oscillations Network) [19] can obtain nearly continuous observations of the
solar free oscillation. Millions of modes have been identified so far, and
have helped us infer the acoustic and dynamical structure of the solar
interior and place constraints on the models of the solar evolution [14].
However, the degree 2, low-order p, f, and g modes have not been observed so
far with helioseismic velocity and intensity measurements, as its
theoretically predicted amplitudes are far smaller than the threshold of
current detectability [8, 13, 20, 21]. These modes are closer to the denser
part of the Sun’s center, which suggests that space-based GW detectors like
LISA may be more sensitive to these modes than conventional techniques [13,
22].

One can describe the solar modes with the spherical harmonic function
$Y_{l}^{m}(\theta,\phi)$ and three indices $n$, $l$, $m$, which may determine
the frequencies and lifetimes of solar oscillation modes [23]. Due to that the
lifetimes of low-$l$, low-$n$ modes with low frequencies are expected to last
at least several months or much longer [23], we regard the low-$l$ and low-$n$
solar oscillation noises as approximately stationary in one observation window
(3 months) of TianQin [1]. Among the millions of different solar oscillation
modes, we mostly care about the quadrupole modes ($l=2$) that can generate
largest perturbations to the gravitational potential [12, 13]. In addition,
lower-order modes excite significantly larger external gravitational
perturbations than higher-order modes given fixed mode energy [8]. Therefore,
we can determine that the low-$n$ quadrupole modes are more relevant to
TianQin.

These modes with low frequencies are expected to have energies about
${10^{28}}$ ergs [8, 24, 11]. The associated quadrupole moments $\alpha$ are
given in [8] (see table 2), with mode energies normalized to ${10^{28}}$ ergs.
Other sources on the values of $\alpha$ can be seen in table 3 and 4. The
gravitational perturbation is expressed as

$\displaystyle\begin{aligned}
\delta\Phi=\alpha\frac{G{M_{\odot}}}{R_{\odot}}{\left(\frac{{{R_{\odot}}}}{r}\right)^{l+1}}{Y_{l}^{m}(\theta,\phi)}{e^{i\omega
t}},\end{aligned}$ (1)

where $M_{\odot}$ is the Sun’s mass, $R_{\odot}$ is the Sun’s radius, $\omega$
is the oscillation angular frequency, and $r$ is the distance from the Sun to
the satellite [8]. Due to the factor $(1/r)^{l+1}$ and that the TianQin
satellites are located at a distance of about 1 AU from the Sun, the
gravitational perturbation decreases rapidly with increasing degree $l$. Based
on Eq. (1), the corresponding strain amplitudes can be calculated from the
tidal acceleration equation [8].

Table 2: Parameters of the solar modes used in the simulation [8]. The
corresponding strain amplitudes $h$ are also included in the table. For LISA,
the detectable strain amplitude is estimated to be ${10^{-21}}$–${10^{-20}}$
at a signal-to-noise ratio of 5 with one year’s observation [13].

  * Mode | Frequency ($\mu$Hz) | $\alpha$ | $h$
---|---|---|---
g7 | $101.2$ | $1.42\times{10^{-12}}$ | $1.10\times{10^{-23}}$
g6 | $134.0$ | $2.80\times{10^{-12}}$ | $1.24\times{10^{-23}}$
g5 | $168.8$ | $4.72\times{10^{-12}}$ | $1.32\times{10^{-23}}$
g4 | $192.2$ | $5.96\times{10^{-12}}$ | $1.29\times{10^{-23}}$
g3 | $220.4$ | $7.17\times{10^{-12}}$ | $1.18\times{10^{-23}}$
g2 | $254.0$ | $6.07\times{10^{-12}}$ | $7.50\times{10^{-24}}$
g1 | $293.6$ | $2.81\times{10^{-12}}$ | $2.60\times{10^{-24}}$
f | $350.9$ | $2.31\times{10^{-12}}$ | $1.50\times{10^{-24}}$
p1 | $381.6$ | $5.31\times{10^{-12}}$ | $2.91\times{10^{-24}}$
p2 | $514.4$ | $2.02\times{10^{-12}}$ | $6.07\times{10^{-25}}$
p3 | $663.6$ | $8.60\times{10^{-13}}$ | $1.56\times{10^{-25}}$

Table 3: The quadrupole moments estimated from limiting the velocity
amplitudes $V$, which are converted from [9, 11]. When the effective bulk
amplitudes $\bar{\xi}_{nlm}=1$ mm, the quadrupole moments are about
$3.61\times{10^{-12}}$.

  * Mode | Frequency ($\mu$ Hz) |  $\alpha$ ($V=1$ mm/s) | $h$
---|---|---|---
g8 | $121.1$ | $3.51\times{10^{-11}}$ | $9.71\times{10^{-23}}$
g7 | $134.0$ | $3.41\times{10^{-11}}$ | $9.43\times{10^{-23}}$
g6 | $149.8$ | $3.23\times{10^{-11}}$ | $8.93\times{10^{-23}}$
g5 | $168.8$ | $3.09\times{10^{-11}}$ | $8.55\times{10^{-23}}$
g4 | $192.2$ | $2.62\times{10^{-11}}$ | $7.25\times{10^{-23}}$
g3 | $220.4$ | $1.97\times{10^{-11}}$ | $5.46\times{10^{-23}}$
g2 | $254.0$ | $1.17\times{10^{-11}}$ | $3.25\times{10^{-23}}$
g1 | $293.6$ | $3.87\times{10^{-12}}$ | $1.07\times{10^{-23}}$
f | $350.9$ | $2.30\times{10^{-12}}$ | $6.37\times{10^{-24}}$
p1 | $381.6$ | $3.14\times{10^{-12}}$ | $8.70\times{10^{-24}}$
p2 | $514.4$ | $4.41\times{10^{-13}}$ | $1.22\times{10^{-24}}$
p3 | $663.6$ | $1.01\times{10^{-13}}$ | $2.80\times{10^{-25}}$

Table 4: Quadrupole moments with observational upper limits of 1 cm/s on solar
surface velocity amplitudes, which are converted from [13].

  * Mode | Frequency ($\mu$Hz) | $\alpha$ | Mode | Frequency ($\mu$Hz) | $\alpha$
---|---|---|---|---|---
g22 | 50.6 | $1.43\times{10^{-11}}$ | g7 | 136 | $1.46\times{10^{-10}}$
g21 | 52.9 | $1.60\times{10^{-11}}$ | g6 | 152 | $1.74\times{10^{-10}}$
g20 | 55.4 | $1.77\times{10^{-11}}$ | g5 | 171 | $2.00\times{10^{-10}}$
g19 | 58.1 | $2.00\times{10^{-11}}$ | g4 | 194 | $2.12\times{10^{-10}}$
g18 | 61.1 | $2.28\times{10^{-11}}$ | g3 | 222 | $2.00\times{10^{-10}}$
g17 | 64.4 | $2.62\times{10^{-11}}$ | g2 | 257 | $1.46\times{10^{-10}}$
g16 | 68.0 | $3.00\times{10^{-11}}$ | g1 | 297 | $7.19\times{10^{-11}}$
g15 | 72.1 | $3.46\times{10^{-11}}$ | f | 356 | $5.28\times{10^{-11}}$
g14 | 76.8 | $4.06\times{10^{-11}}$ | p1 | 384 | $7.19\times{10^{-11}}$
g13 | 82.0 | $4.81\times{10^{-11}}$ | p2 | 515 | $1.48\times{10^{-11}}$
g12 | 87.9 | $5.70\times{10^{-11}}$ | p3 | 664 | $4.52\times{10^{-12}}$
g11 | 94.7 | $6.81\times{10^{-11}}$ | p4 | 812 | $1.58\times{10^{-12}}$
g10 | 103 | $8.29\times{10^{-11}}$ | p5 | 960 | $6.24\times{10^{-13}}$
g9 | 112 | $1.00\times{10^{-10}}$ | p6 | 1110 | $2.71\times{10^{-13}}$
g8 | 123 | $1.22\times{10^{-10}}$ |  |  |

In order to conduct orbit simulation by modifying the gravity field
coefficients in the program we developed, we rewrite gravity potential change
$\delta\Phi$ at radius $r$, co-latitude $\theta$ and longitude $\phi$ in the
form of equation (2) in [25], which is often seen in physical geodesy:

$\displaystyle\delta\Phi=\frac{GM_{\odot}}{R_{\odot}}\sum\limits_{l=0}^{\infty}{\left(\frac{{{R_{\odot}}}}{r}\right)^{l+1}}\sum\limits_{m=0}^{l}(\delta
C_{lm}\cos(l\phi)+\delta S_{lm}\sin(l\phi))P_{lm}(\cos(\theta)),$ (2)

where $P_{lm}$ is the associated Legendre function of spherical harmonic
degree $l$ and order $m$. Gravity potential coefficients $\delta C_{lm}$ and
$\delta S_{lm}$ describe the dependence on the time-varying Sun’s internal
mass distribution and also represent changes of the solar gravity field. Since
$C_{20}=-J_{2}$, the coefficient $\delta C_{20}$ with corresponding
oscillation frequencies can be obtained from the quadrupole moments of the
solar modes in table 2.

The solar oscillations can also generate GWs. The effect dominates in the
frequency band above ${\nu_{r}}\equiv c/2\pi r\approx$ 0.3 mHz, and is larger
than the Newtonian counterpart by a factor of the order $(2\pi r\nu/c)^{4}$
[13, 14]. The contribution of GWs will also be presented in the next section.

##  3 Simulation setup and results

We adopt the range acceleration $\ddot{\rho}$ as the observable, which is the
second time derivative of the range $\rho$ between two satellites, and compare
the ASD with the residual acceleration noise requirements ($1.4\times
10^{-15}$ m/s2/Hz1/2). To meet the precision requirement, our team have
developed a package TQPOP (TianQin Quadruple Precision Orbit Propagator) and
extended the precision to 34 significant digits with quadrupole precision
arithmetic [6]. The detailed force models can be found in table 2 in [7] and
the subsequent simulation is to further consider the perturbations of the
Sun’s $J_{2}$ given in section 2 and analyze their impact on the TianQin’s
detection.

The initial orbital parameters from table 1 in [7] are adopted as input and
the integration lasts for one observation window of 3 months [1]. In order to
highlight the contribution of the solar free oscillation to the total effect,
we only retain some necessary force models in the initial simulations, such as
the point mass of the Earth and the Sun, and do not consider models like EOP
(Earth Orientation Parameters), so that the numerical errors of high-frequency
parts can be significantly reduced.

Figure 2: The range-acceleration ASD of two TianQin satellites after 90 days
with and without the solar free oscillations, using the parameters from table
2. Figure 3: The contributions of solar free oscillations to the range-
acceleration ASD from two different estimation methods for mode amplitudes
(see table 3).

In figure 2, we simulate the range acceleration of two satellites with and
without the solar free oscillations (only modes above $10^{-4}$ Hz are
considered). The splitting of the peaks is caused by coupling with the orbital
frequency ($3\times{10^{-6}}$ Hz), which corresponds to an orbital period of
about 3.64 days. In this test case, the peaks at $<10^{-4}$ Hz are due to the
point masses of the Earth and the Sun. As a comparison, we also present the
simulation using the parameters from table 3 [9] in figure 3, which also meets
the range acceleration noise requirement and is close to the magnitude in
figure 2.

Figure 4:  Quadrupole modes with observational upper limits of 1 cm/s on solar
surface velocity amplitudes (see table 4). Curve (a) represents Newtonian part
and (b) represents GW part.

Since the above parameters of modes are theoretical predictions, we also need
to give the maximum estimation based on the actual observational upper limit.
If the upper limit of the surface velocity amplitude 1 cm/s is considered [13,
26], the worst case can be simulated as shown by the red curve in figure 4,
which is 1–2 orders higher than the previous results, but still lower than the
range acceleration noise requirement. The contribution of emitted GWs to the
noise is shown by the blue dotted line in figure 4. In the frequency band
above 0.3 mHz, the contribution of GW part is higher than the Newtonian part,
but both are expected to decrease at the higher frequency band.

After adding other force models [7] of the Earth-Moon system to the simulation
of figure 2, we can see the total effects including the solar free
oscillations in figure 5. The result still meets the noise requirement of the
range acceleration and compliments our previous result shown in figure 2 of
[6].

Figure 5:  The total effects combining the solar free oscillations and the
Earth-Moon system.

##  4 Conclusion

In this work, we have evaluated the disturbance caused by solar free
oscillations and compared it with TianQin’s noise requirement. Unlike previous
works that have been done for other missions [8, 9, 13], we have directly
calculated their effects on inter-satellite range acceleration via high-
precision orbit simulation. Given the current observation and theoretical
estimates on the mode amplitudes, the solar oscillation noise is expected to
be well below TianQin’s total noise requirement by at least 1 order of
magnitude. Therefore, we conclude that the solar free oscillation will not
affect TianQin’s GW detection.

Nevertheless, the work on evaluating gravity field disturbance does not end
here for TianQin. Free oscillation frequencies of the Earth can also enter the
target band of TianQin. These are currently being investigated and will be
reported in the future.

## Acknowledgment

The authors thank Bobing Ye, Yuzhou Fang, Junwei Zhao, Cong Yu, Jianwen Ou,
Jianwei Mei, Yiming Hu and Jiandong Zhang for their helpful discussions. X. Z.
is supported by the National Key R&D Program of China (Grant No.
2020YFC2201202).

## References

## References

  * [1] Jun Luo, Li-Sheng Chen, Hui-Zong Duan, Yungui Gong, et al.  TianQin: a space-borne gravitational wave detector.  Class. Quantum Grav., 33:035010, 2016.
  * [2] Laser interferometer space antenna: A cornerstone mission for the observation of gravitational waves, 2000.
  * [3] Jianwei Mei, Yan-Zheng Bai, Jiahui Bao, et al.  The tianqin project: Current progress on science and technology.  Progress of Theoretical and Experimental Physics, 2021(5), 08 2020\.
  * [4] Bobing Ye, Xuefeng Zhang, Yanwei Ding, and Yunhe Meng.  Eclipse avoidance in TianQin orbit selection.  Phys. Rev. D, 103:042007, 2021.
  * [5] Bobing Ye, Xuefeng Zhang, Mingyue Zhou, Yan Wang, Huimin Yuan, Defeng Gu, Yanwei Ding, Jinxiu Zhang, Jianwei Mei, and Jun Luo.  Optimizing orbits for tianqin.  International Journal of Modern Physics D, 28(09):1950121, 2019\.
  * [6] Xuefeng Zhang, Chengjian Luo, Lei Jiao, Bobing Ye, Huimin Yuan, Lin Cai, Defeng Gu, Jianwei Mei, and Jun Luo.  Effect of earth-moon’s gravity on tianqin’s range acceleration noise.  Phys. Rev. D, 103:062001, 2021.
  * [7] Chengjian Luo and Xuefeng Zhang.  Effect of earth-moon’s gravity on tianqin’s range acceleration noise. ii. impact of orbit selection.  Phys. Rev. D, 105:102007, May 2022.
  * [8] Curt Cutler and Lee Lindblom.  Gravitational helioseismology?  Phys. Rev. D, 54:1287–1290, 1996.
  * [9] Weitou Ni and Xiaohui Xu.  Separation of the gravitational-wave signals and the solar oscillation signals.  AIP Conference Proceedings, 523:459–460, 2000.
  * [10] Pau Amaro-Seoane, Heather Audley, Stanislav Babak, et al.  Laser interferometer space antenna, 2017.
  * [11] Weitou Ni.  Astrod–an overview.  International Journal of Modern Physics D, 11(07):947–962, 2002\.
  * [12] G. Giampieri, A. Polnarev, I. Roxburgh, and S. Vorontsov.  The effect of solar oscillations on space gravitational wave experiments.  Astrophys. Space Sci., 261:35–36, 1999.
  * [13] A. G. Polnarev, I. W. Roxburgh, and D. Baskaran.  Response of a spaceborne gravitational wave antenna to solar oscillations.  Phys. Rev. D, 79:082001, Apr 2009.
  * [14] A. G. Polnarev, D. Baskaran, and I. W. Roxburgh.  Gravitational waves generated by solar oscillations.  AIP Conference Proceedings, 1168(1):1101–1104, 2009.
  * [15] Robert B. Leighton, Robert W. Noyes, and George W. Simon.  Velocity Fields in the Solar Atmosphere. I. Preliminary Report.  APJ, 135:474, mar 1962.
  * [16] Michael Stix.  Oscillations. In: The Sun., pages 181–235.  Springer, Berlin, Heidelberg, 2002.
  * [17] A.H. Gabriel and the GOLF Team.  Global oscillations at low frequency from the soho mission (golf).  Advances in Space Research, 11(4):103–112, 1991.
  * [18] J.W. Harvey, F. Hill, J.R. Kennedy, J.W. Leibacher, and W.C. Livingston.  The global oscillation network group (gong), 1988.
  * [19] Y. Elsworth, R. Howe, G. R. Isaak, C. P. McLeod, and R. New.  Low-l p-mode solar eigenfrequency measurements from the birmingham network.  Monthly Notices of the Royal Astronomical Society, 251(1):7P–9P, 07 1991.
  * [20] D. O. Gough, J. W. Leibacher, P. H. Scherrer, and J. Toomre.  Perspectives in helioseismology.  Science, 272:1281–1283, 1996.
  * [21] Pawan Kumar, Eliot J. Quataert, and John N. Bahcall.  Observational searches for solar g-modes: Some theoretical considerations.  APJL, 458:L83, feb 1996.
  * [22] J. Christensen-Dalsgaard, Werner Dappen, S. Ayukov, Edwin Anderson, H. Antia, et al.  The current state of solar modeling.  Science (New York, N.Y.), 272:1287–92, 1996.
  * [23] J. W. Harvey, F. Hill, R. P. Hubbard, J. R. Kennedy, J. W. Leibacher, et al.  The global oscillation network group (gong) project.  Science, 272(5266):1284–1286, 1996.
  * [24] K. G. Libbrecht and M. F. Woodard.  Advances in helioseismology.  Science, 253(5016):152–157, 1991.
  * [25] Oliver Montenbruck and Eberhard Gill.  Satellite Orbits: Models, Methods and Applications, pages 57–58.  Springer, 2000.
  * [26] T. Appourchaux, K. Belkacem, Broomhall, et al.  The quest for the solar g modes.  AAPR, 18(1-2):197–277, Feb 2010.


# Uncertain AoI in Stochastic Optimal Control of Networked and Constrained LTI
Systems

Jannik Hahn1 and Olaf Stursberg1 1Control and System Theory, Dept. of
Electrical Engineering and Computer Science, University of Kassel, 34121
Kassel, Germany. Email<EMAIL_ADDRESS>
###### Abstract

This paper addresses finite-time horizon optimal control of single-loop
networked control systems with stochastically modeled communication channel
and disturbances. To cope with the uncertainties, an optimization-based
control scheme is proposed which uses a disturbance feedback and the age of
information as central aspects. The disturbance feedback is an extension of
the control law used for balanced stochastic optimal control previously
proposed for control systems without network. Balanced optimality is
understood as a compromise between minimizing of expected deviations from the
reference and minimization of the uncertainty of future states. Time-varying
state constraints as well as time-invariant input constraints are considered,
and the controllers are synthesized by semi-definite programs.

##  I INTRODUCTION

The increasing importance of networked control systems (NCS) is due to two
reasons: First, wireless sensors have become widely available and simpler to
integrate in control systems, enabling the control of non-stationary processes
in single loop, see e.g. [1]. Secondly, the need for controlling large systems
with multiple agents has promoted the development of distributed control
strategies. On the one hand, these reduce the computational complexity of
solving the overall control problem (thus alleviating to meet real-time
requirements), while principles of optimality can be maintained [2]. On the
other hand, they also require information flow between the agents. From the
perspective of a single agent, the reception of information from a neighbor
can be seen as obtaining sensor information through wireless communication. In
consequence, many methods provided for single-loop loops consisting of plant,
sensor, communication unit, and controller can be transferred to larger
distributed networks – thus, the motivation of this paper is to propose an
optimal control scheme for a single-loop structure, however, with the
perspective to be used for more general system structures and for receding
horizon control in upcoming work.

Any wireless communication network with possible imperfections (such as packet
loss and/or latencies) can be interpreted and modeled as stochastic process.
Consequently, the control loop is subject to this uncertainty, and the
imperfections of communication may endanger the stability of the controlled
system, at least if the resulting delays are in the order of the dominant time
constants of the plant. Then, the consideration of the dynamics of the
communication network within controller synthesis is mandatory, and the joint
design of the network operation and the controller appears as a promising
concept [3, 4].

In contrast to the latter two approaches, this paper addresses the finite-time
horizon stochastic optimal control of uncertain linear systems. Derived from
the methods in [5, 6] and [7], the recently presented scheme of _balanced
stochastic optimal control_ in [8] determines a compromise between rejecting
disturbances and meeting goals of optimal control. Thus, uncertainties arising
in predicting the system behavior can be effectively minimized, while the
convergence of the expected states to the origin is still guaranteed. The
disturbance rejection is formulated with respect to an optimization over
feedback policies, hence the satisfaction of state and input constraints in a
probabilistic sense (also called _chance constraints_) can be considered as
well. However, the control scheme in [8] is not applicable to networked
control systems with uncertain availability of information. To handle this,
the work in [9, 10] uses a policy which switches between open-loop and closed-
loop control depending on to the availability of information. There, the
stability is guaranteed for certain assumptions on the communication network,
but the satisfaction of state and/or input constraints formulated for the
plant were not addressed. (This holds true for the vast majority of papers on
feedback control of networked control systems.)

In contrast, the present paper combines the two mentioned aspects: the
balanced stochastic optimal control of [8], which optimizes over a control
policy to consider constraints as well as the reduction over conservatism, and
the use of a switching control policy for the communication network similar to
[10]. The paper is organized as follows: Sec. 2 clarifies the notation used
throughout the paper, and Sec. 3 specifies the control problem. Sec. 4
explains how the _age of information_ can be used efficiently to encode the
effect of the communication network on the plant control, and the Sec. 5 as
main part shows how this measure can be used within controller synthesis.
Section 6 provides a numerical example, and Sec. 7 concludes the paper.

##  II PRELIMINARIES

This section introduces a part of the notation used in the sequel, and it
refers to some mathematical facts required in the upcoming sections.

Let $s(k)$ denote the discrete-time value of a vector $s(t)\in\mathbb{R}^{n}$
for time $t=t_{0}+k\cdot\Delta t$ with $k\in\mathbb{N}^{+}$ and a constant
time step $\Delta t\in\mathbb{R}^{+}$. If $s(k)$ is predicted in a previous
point of time $l$, the notation $s_{k|l}$ is used. However, for brevity of
notation, let $t_{0}=0$ be frequently used as the time instant of prediction,
and then $s_{k}:=s_{k|0}$ is used for short for the predicted value.

The symbol $\mathbb{N}_{H}$ denotes the set $\left\\{k\in\mathbb{N}_{\geq
0}\left|\begin{array}[]{l}k\leq H-1\end{array}\right.\right\\}$ with $H$
denoting the prediction horizon. Bold letters denote matrices collecting sub-
matrices of a signal predicted along the horizon, e.g.
$\textbf{s}=\small\begin{bmatrix}s_{0}^{T},s_{1}^{T},\dots,s_{H}^{T}\end{bmatrix}^{T}$,
and the operator $\operatorname{row}_{k}(\cdot)$ is used to select the $k$-th
row of such an matrix, e.g. $\operatorname{row}_{3}(\textbf{s})=s_{3}$.

For a vector $s\in\mathbb{R}^{n}$, a convex polytope:

$\displaystyle\mathbb{S}=\left\\{s\left|\begin{array}[]{l}C_{s}\cdot s\leq
b_{s}\end{array}\right.\right\\},$ (2)

is denoted by $(C_{s},b_{s})$, and $n_{\mathbb{S}}$ is the number of its
faces.

An ellipsoidal set is defined by a center point $\bar{s}$ and a shape matrix
$\mathcal{S}$. The affine transformation of an ellipsoidal set with matrix $M$
and vector $v$ is again an ellipsoidal set according to:

$\displaystyle
M\cdot\varepsilon\left(\bar{s},\mathcal{S}\right)+v=\varepsilon\left(M\bar{s}+v,M\mathcal{S}M^{T}\right).$
(3)

A multivariate normal distribution of an $n$-dimensional random vector $s$
with covariance matrix $\mathcal{S}\in\mathbb{R}^{n\times n}$, and mean value
$\bar{s}\in\mathbb{R}^{n}$ is written as
$s\sim\mathcal{N}\left(\bar{s},\mathcal{S}\right)$, and a Bernoulli
distribution of a random vector $s$ with probability matrix $p$ is referred to
by $s\sim\mathcal{B}\left(p\right)$. The expected value of a random value $s$
is denoted by $\operatorname{\mathbb{E}}\left[s\right]=\bar{s}$, and the
covariance by $\operatorname{cov}[s]=\mathcal{S}$. The expected value is
always based on the information available in $t_{0}$.

The sum of two normally distributed random variables
$s_{1}\sim\mathcal{N}\left(\bar{s}_{1},\mathcal{S}_{1}\right)$ and
$s_{2}\sim\mathcal{N}\left(\bar{s}_{2},\mathcal{S}_{2}\right)$ is again
normally distributed:

$\displaystyle
s_{1}+s_{2}\sim\mathcal{N}\left(\bar{s}_{1}+\bar{s}_{2},\mathcal{S}_{1}+\mathcal{S}_{2}\right).$
(4)

The level curves of a Gaussian probability density function are ellipsoidal.
Throughout the paper, the mean value of a normal distribution coincides with
the center point of the confidence ellipsoid, and the shape matrix is equal to
the covariance matrix of the distribution, thus the same notation for these
quantities is used.

The symbol $\left\lVert s\right\rVert_{Q}=s^{T}\cdot Q\cdot s$ denotes a
weighted 2-norm of a vector $s$ with symmetric and positive semi-definite
weight matrix $Q$.

##  III NETWORKED SYSTEM AND PROBLEM DEFINITION

The sturcture of the system class under consideration is shown in Fig. 1(a)
and consists of a discrete-time linear system with probabilistically modeled
additive disturbances, a controller, and a communication network. The systems
dynamics is modeled as:

$\displaystyle x(k+1)=Ax(k)+Bu(k)+Ew(k),$ (5) $\displaystyle
w(k)\sim\mathcal{N}\left(0,\mathcal{W}(k)\right),$ (6)

where $x\in\mathbb{R}^{n_{x}}$ is the state vector, $u\in\mathbb{R}^{n_{u}}$
the input vector, and $w\in\mathbb{R}^{n_{w}}$ the disturbance vector. The
additive disturbances are assumed to be i.i.d. and normally distributed with
zero-mean and covariance matrix $\mathcal{W}$.

(a) Overall system.

(b) Communication network

Figure 1: Structure of the considered type of networked control system and the
communication system.

######  Assumption 1.

The initial state $x_{0}:=x(t_{0})$ is known. The stochastic process of the
disturbance may vary over time, but is assumed to be known as
$\mathcal{W}_{k}:=\mathcal{W}(k)$.

The state $x(k)$ and input $u(k)$ have to satisfy polytopic chance-constraints
with a probabilities given by ${\delta_{x}}$, and ${\delta_{u}}$ respectively
(both typically chosen to be close to 1):

$\displaystyle\operatorname{\mathbb{P}}\big{(}x(k)\in\mathbb{X}_{k}\big{)}\geq{\delta_{x}},\;\;\mathbb{X}_{k}=\left\\{x\left|\begin{array}[]{l}C_{x_{k}}\cdot
x\leq b_{x_{k}}\end{array}\right.\right\\},\quad$ (8)
$\displaystyle\operatorname{\mathbb{P}}\big{(}u(k)\in\mathbb{U}\big{)}\geq{\delta_{u}},\quad\;\;\mathbb{U}=\left\\{u\left|\begin{array}[]{l}C_{u}\cdot
u\leq b_{u}\end{array}\right.\right\\}.$ (10)

The sets $\mathbb{X}_{k}$ and $\mathbb{U}$ are assumed to be convex and
compact, and contain the origin in their interior for all $k$.

The controller is connected to the plant in wired form, but the link between
the sensor and the controller is established by wireless communication. Assume
here that the communication network is a simple one-link network and consists
of a sender S, a receiver R, and the link-probability $p(k)$. The link
probability can be understood as an abstract representation of the network
characteristics (or protocol), as e.g. resending information in case of failed
transmission, or activation/deactivation of the channel by a network
controller, etc. The simplicity of the communication network is here chosen
for brevity of explanation, but the control synthesis described in the
following is not limited to this network structure. Assuming a time-varying
link-probability, the communication network is modeled by the Markov chain for
each time-step $k$, as shown in Fig. 1(b).

######  Assumption 2.

The link-probability $p(k)$ may vary over time, but is assumed to be known
(possibly according to a predictive network control scheme, see [11]),
implying that $p_{k}:=p(k)$ holds true. Further, possible information dropout
at time $k$ and $k+t$ are i.i.d. for all $k$ and $t$.

In time $k$, the current state $x(k)$ is available to S. The information is
broadcasted to R (and thus available to the controller) with probability
$p(k)$. If information is not received, the information remains with S, and
the Markov chain remains in the corresponding state S. Independently of
whether information is received, the controller has to apply a control input
$u(k)$ to the system, which is then simultaneously affected by the disturbance
$w(k)$.
In $k+1$, a new state $x(k+1)$ is measured, and if the previous transfer
failed, old and new state information is available to S.

######  Assumption 3.

The communication network uses an acknowledgment”=based protocol like TCP,
thus the S knows whether information is received by R, or not. In case the
information failed to be sent in time $k$, then old and new information are
transmitted together in $k+1$ with link probability $p(k+1)$.

######  Remark 4.

Assumption 3, which will be necessary to reconstruct disturbances affecting
the dynamics previously, may imply an arbitrarily large package size in
recursive execution – this, however, would only pose problems for the
unrealistic case of persistently low values of $p(k)$.

Similarly to the control scheme presented in [8], this paper uses time-varying
feedback control laws, aiming at the determination of an admissible control
sequence for the horizon $H$:

$\displaystyle\textbf{u}=\begin{bmatrix}u_{0}^{T}&u_{1}^{T}&\dots&u_{H-1}^{T}\end{bmatrix}^{T}.$
(11)

The following description distinguishes between an offline controller design
(where disturbances, communication links, states etc. are modeled
stochastically), and the online use of the control law (where specific values
for disturbances, communications, etc. are measured and processed). According
to Assumption 1, the initial state $x_{0}$ is known, but the disturbance is
normally distributed, i.e., deterministic values for future states are not
available when the control law is determined. Nevertheless, one can predict
the behavior of the state $x_{1}$ under the impact of the control input
$u_{0}$ and the disturbance
$w_{0}\sim\mathcal{N}\left(0,\mathcal{W}_{0}\right)$ according to (5). The
latter is again normally distributed with respect to (3) and (4):

$\displaystyle
x_{1}=Ax_{0}+Bu_{0}+Ew_{0}\sim\mathcal{N}\left(\bar{x}_{1},\mathcal{X}_{1}\right).$
(12)

Due to recursive computation, the following states $x_{k}$, $k>1$ are normally
distributed, too. The main control objective is to steer the mean value of the
state to the origin with acceptable costs for the input, while the state
uncertainty is minimized at the same time. The minimization of the uncertainty
is equivalent to the minimization of the volume of the confidence ellipsoids
for the states (see Sec. 2), and thus to minimize the set of states which are
reachable with a certain probability.

As the availability of information on the system state to the controller is a
central point in this paper, this aspect is described in more detail in the
upcoming section.

##  IV AGE OF INFORMATION

Similar to the previous work in [12], the following definition of AoI is
employed:

######  Definition 5.

The quantity $a(k)\in\mathbb{N}_{\geq 0}$ denotes the age of the newest state
information $x(l),\;l\in\mathbb{N}_{\leq k}$ available to the controller, and
it quantifies the difference between the time-instances of sending and using
$x(l)$.

Example: According to Assumption 1, the initial state $x_{0}$ is known to the
controller. If then the communication link fails for the next five time-steps,
i.e. for $k\in\\{1,\ldots,5\\}$, the newest information is still $x_{0}$, and
$a(5)=5$. If the next transmission is successful, the AoI in $k=6$ equals
zero, i.e. $a(6)=0$.

######  Remark 6.

The Age of Information is defined with respect to the information of the state
$x(k-a(k))$. In consequence of Assumption 3 and (5), the controller can
reconstruct each disturbance up to $w(k-a(k)-1)$ in time-step $k$. Hence with
$a(k)=0$, the newest disturbance possibly available to the controller in $k$
is $w(k-1)$.

Similar to the prediction of future states, there are no deterministic values
for the AoI available at the time-instance of planning, but the behavior of
the AoI is predictable according to the Markov chain shown in Fig. 2 with
states $\sigma_{j}\rightarrow a_{k}=j$, and the probability $q_{k}=1-p_{k}$
for incrementing the AoI.

Figure 2: Exemplary Markov chain to model the AoI for 3 time-steps.

This Markov chain corresponds to the communication network shown in Fig. 1(b),
and is exemplary for 3 time-steps, since the maximum AoI is three according to
$\sigma_{3}$. For more time-steps, the model is extended to the right and ends
with a self-loop on the last state. With time-variant communication link
probabilities, the transition matrix $P_{k}$ for this model is time-variant,
too:

$\displaystyle P_{k}=\begin{bmatrix}p_{k}&p_{k}&p_{k}&1\\\ q_{k}&0&0&0\\\
0&q_{k}&0&0\\\ 0&0&q_{k}&0\end{bmatrix}.$ (13)

Initialized to $\mu_{0}=[1\;\;0\;\;0\;\;0]^{T}$, the probability vector
$\mu_{k}$ for the AoI evolves according to:

$\displaystyle\mu_{k}=\left(\prod_{l=1}^{k}P_{l}\right)\cdot\mu_{0},$ (14)

and the probability for the age of information is given by:

$\displaystyle\operatorname{\mathbb{P}}\big{(}a(k)=j\big{)}=\mu_{k}[j]$ (15)

with $\mu_{k}[j]$ being the $(j+1)$-th entry of the vector $\mu_{k}$.

##  V CONTROLLER SYNTHESIS

This section describes the synthesis of suitable feedback policies to control
the system in presence of stochastic AoI, and it explains how the state
uncertainty can be formalized.

###  V-A Feedback policy

On one hand, if the state is available to the controller, the optimal control
strategy for deterministic and constrained linear systems is based on time-
varying state feedback control laws, see e.g. [13]. On the other hand, if
state information is not available, the optimal control law should be based on
the feedback of an estimated value (e.g. a previously predicted value), i.e. a
feedback of expected values is realized. In general with uncertain
availability of the state information to the controller, a switched control
policy of time-varying feedback laws can thus be proposed to combine the two
cases:

$\displaystyle
u(k)=\sum_{r=0}^{k}K_{k,r}\cdot\tilde{x}_{r},\quad\tilde{x}_{r}=\begin{cases}x(r)&\text{if
$x(r)$ is available \;}\\\ \bar{x}_{r|l}&\text{otherwise}\end{cases}$ (16)

Here, $l:=k-a(k)$ denotes for a moment the time-step of the last certain
information, while $\bar{x}_{r|l}$ is the state predicted in $l$ for a time
$r>l$ of the nominal system:

$\displaystyle\bar{x}_{r|l}=A^{r-l}x(l)+\sum_{r=1}^{a(k)}A^{a(k)-r}Bu{(l+r-1)}.$
(17)

In other words, $\bar{x}_{r|l}$ is the expected state for time $r$ predicted
or reconstructed from the last certainly known state $x(l)$.

######  Remark 7.

To keep the controller synthesis as simple as possible, the paper aims at
using the same feedback matrix $K_{k,r}$ in (16) regardless the availability
of the state $x(r)$. Hence, the offline synthesis aims at determining the
feedback matrices. When using these online, the controller decides whether the
measured and communicated state (if available) or a predicted state has to be
fed back according to (16) with feedback matrix $K_{k,r}$.

In general for systems without communication, a state feedback can be
reformulated into a feedback of initial state and disturbances (the interested
reader is referred to [14] and [8, Lemma 2]). Motivated by and similar to
these reformulations of control laws, the following is stated:

######  Lemma 8.

The state feedback (16) with (17) can be reformulated into a disturbance
feedback:

$\displaystyle u(k)=V_{k}\cdot x_{0}+\sum_{r=0}^{k-1}\mathds{1}_{k,r}\cdot
M_{k,r}\cdot w(r)$ (18)

with feedback matrices $V_{k}\in\mathbb{R}^{n_{u}\times n_{x}}$ and
$M_{k,r}\in\mathbb{R}^{n_{u}\times n_{w}}$ and with an indicator function
depending on the AoI:

$\displaystyle\mathds{1}_{k,r}=\begin{cases}1&\forall\;r<k-a(k)\\\
0&\forall\;r\geq k-a(k)\end{cases}.$ (19)

###### Proof.

see Appendix -C. ∎

Control law (18) with (19) implies the feedback of all disturbances
$w(k-a(k)-1)$, which are available to the controller in time-step $k$.
According to Assumption 3 the controller can reconstruct all disturbances
$w(k-a(k)-1)$, see Remark 6.

######  Remark 9.

A common disturbance feedback as used in ([15, 14, 8], etc.), and the proposed
disturbance feedback (18) differ only wrt. the indicator function
$\mathds{1}_{k,r}$, and the indicator function only depends on the AoI $a(k)$.
Thus, (ex)changing the communication network in the setting Fig. 1 results
only in an adaption of the Markov chain in Fig. 2 used to model the AoI.
Consequently, the prediction of the systems behavior and the controller
synthesis described in the following are decoupled from the exact model of the
communication network, but requires a prediction of the AoI.

###  V-B Prediction of the closed-loop behavior

To plan the systems behavior over a horizon of length $H$, the notation of
stacked vectors for the states, inputs, and disturbances is used:

$\displaystyle\textbf{x}=\small\begin{bmatrix}x_{0}\\\ x_{1}\\\ \vdots\\\
x_{H}\end{bmatrix},\quad\textbf{u}=\small\begin{bmatrix}u_{0}\\\ u_{1}\\\
\vdots\\\ u_{H-1}\end{bmatrix},\quad\textbf{w}=\small\begin{bmatrix}w_{0}\\\
w_{1}\\\ \vdots\\\ w_{H-1}\end{bmatrix}.$ (20)

With matrices A, B, and E of adequate size (see Appendix -A), the evolution of
the states over the horizon is encoded by:

x $\displaystyle=\textbf{A}x_{0}+\textbf{B}\textbf{u}+\textbf{E}\textbf{w}.$
(21)

Similarly as in [8], a stacked form of the disturbance feedback (18) is
introduced (with feedback matrices $(\textbf{V},\textbf{M})$ as in Appendix
-A) to predict the closed-loop behavior in the following. The indicator
function (19) is modeled as random variable with Bernoulli distribution to
model the AoI:

$\displaystyle\mathds{1}_{k,r}\sim\mathcal{B}\left(p_{k,r}\right),\quad
p_{k,r}=\sum_{l=0}^{k-r-1}\mu_{k}[l],$ (22)

see Appendix -B. Collecting all indicator functions over the horizon into an
indicator matrix $\textbf{P}\in\mathbb{R}^{H\cdot n_{u}\times H\cdot n_{w}}$
(which has the same size as the feedback matrix M):

$\displaystyle\textbf{P}=\small\begin{bmatrix}0&0&\dots&0\\\
\mathds{1}_{1,0}&0&\dots&0\\\ \vdots&\ddots&\ddots&\vdots\\\
\mathds{1}_{H-1,0}&\dots&\mathds{1}_{H-1,H-2}&0\\\
\end{bmatrix}\otimes\textbf{1}_{n_{u}\times n_{w}},$ (23)

the control trajectory results in:

$\displaystyle\textbf{u}=\textbf{V}\cdot
x_{0}+\left(\textbf{P}\circ\textbf{M}\right)\cdot\textbf{w}.$ (24)

With the Hadamard product $\circ$, the indicator matrix P selects the entries
of M with which all disturbances available to the controller can be fed back.
With (21) and (24), the closed-loop dynamics of the system follows to:

x
$\displaystyle=(\textbf{A}+\textbf{B}\textbf{V})x_{0}+\left(\textbf{E}+\textbf{B}\left(\textbf{P}\circ\textbf{M}\right)\right)\textbf{w}.$
(25)

The uncertainty caused by the communication network (introduced by P) only
affects the rejection of the disturbance w. With stochastically independent P
and w, the state prediction x is a mixed distribution, and a vector of
expected values $\operatorname{\mathbb{E}}\left[\textbf{x}\right]$ is defined.
For the Bernoulli distribution, all possible cases are considered, one of
which is:

$\displaystyle\textbf{P}^{(i)}\in\textbf{P}^{(\Omega)}=\\{\textbf{P}^{(1)},\dots,\textbf{P}^{(o)}\\}$
(26)

with $\textbf{P}^{(\Omega)}$ denoting the set of all possible realizations and
$o$ its cardinality. The state trajectory can be written as function depending
on the Bernoulli case:

$\displaystyle\textbf{x}=\begin{cases}\textbf{x}^{(1)}&\text{if }i=1\\\
\textbf{x}^{(2)}&\text{if }i=2\\\ \vdots\\\ \textbf{x}^{(o)}&\text{if
}i=o\end{cases}$ (27)

with:

$\displaystyle\textbf{x}^{(i)}$
$\displaystyle=(\textbf{A}+\textbf{B}\textbf{V})x_{0}+\left(\textbf{E}+\textbf{B}\left(\textbf{P}^{(i)}\circ\textbf{M}\right)\right)\textbf{w},$
(28) $\displaystyle=:\mathcal{A}x_{0}+\mathscr{E}^{(i)}\textbf{w},$ (29)

and closed-loop matrices $\mathcal{A}$ and $\mathscr{E}^{(i)}$. All
disturbances $w_{k}$ are assumed to be iid., i.e. w is a vector of normally
distributed disturbances. Thus, each state $\textbf{x}^{(i)}$ is normally
distributed for $k>0$, too, and can be referred to by expected values and
covariance matrices representing the state uncertainty.

######  Remark 10.

The minimization of the uncertainty of x could be realized by minimizing the
uncertainty of each state $\textbf{x}^{(i)}$ separately. But for increasing
$H$, this would lead to large complexity, since each realization
$\textbf{P}^{(i)}$ results in a different input trajectory $\textbf{u}^{(i)}$
according to (24), and the maximum number of possible realizations $o$ is
given by the Catalan number $o\leq C_{H}=\frac{(2H)!}{H!(H+1)!}$. Thus, e.g.,
the one-link communication network given in Fig. 1(b) leads to $o=2^{H-1}$. In
addition, the communication network will evaluate over time, thus even if all
possible cases are minimized (and a set of input trajectories
$\textbf{u}^{(i)}$ is obtained), the question of which $\textbf{u}^{(i)}$ the
optimal one is could not be answered till time reaches the end of the
prediction horizon.

Since neither the consideration of each possible case obtained by the
Bernoulli distribution nor the consideration of an averaged case is advisable,
the complexity is here reduced by considering one specific case of the
Bernoulli distribution. This case is denoted by the superscript $\beta$
(leading to $\textbf{x}^{(\beta)}$, $\textbf{u}^{(\beta)}$, and
$\textbf{P}^{(\beta)}$), an it is described in detail in the following
section.

$\displaystyle\mathcal{L}\mathfrak{u}_{k}^{[i]}=\begin{bmatrix}(b_{u}^{[i]}-C_{u}^{[i]}V_{k}x_{0})&C_{u}^{[i]}c_{u}^{\frac{1}{2}}(\textbf{P}^{(\beta)}_{k}\circ\textbf{M}_{k})\textbf{W}\\\
\star&(b_{u}^{[i]}-C_{u}^{[i]}V_{k}x_{0})\textbf{W}\end{bmatrix}\succcurlyeq
0,\quad\mathcal{L}\mathfrak{x}_{k+1}^{[j]}=\begin{bmatrix}(b_{x_{k+1}}^{[j]}-C_{x_{k+1}}^{[j]}\mathcal{A}_{k}x_{0})&C_{x_{k+1}}^{[j]}c_{x}^{\frac{1}{2}}\mathscr{E}_{k}^{(\beta)}\cdot\textbf{W}\\\
\star&(b_{x_{k+1}}^{[j]}-C_{x_{k+1}}^{[j]}\mathcal{A}_{k}\cdot
x_{0})\textbf{W}\end{bmatrix}\succcurlyeq 0\quad$ (30)

###  V-C State and Input constraints

This subsection first clarifies the consequences considering one predefined
case $\textbf{P}^{(\beta)}$, and afterwards, how this case is chosen.
The constraints (8) and (10) require that each state $x(k)$ and input $u(k)$
are contained in their admissible sets $\mathbb{X}$ and $\mathbb{U}$ at least
with likelihood ${\delta_{x}}$ and ${\delta_{u}}$. With choosing to optimize
over one predefined control sequence according to $\textbf{P}^{(\beta)}$,
there are two possibilities in each time-step $k$: i) All disturbances, which
were expected to be available by
$\operatorname{row}_{k}{\textbf{P}^{(\beta)}}$ in $k$, are available to the
controller, and the corresponding control law $u_{k}^{(\beta)}$ is applicable.
Even if more disturbances are available than used with $u_{k}^{(\beta)}$, the
control law can be chosen to $u(k):=u_{k}^{(\beta)}$. If it is chosen to
$u(k):=u_{k}^{(\beta)}$, then, the state evolves according to
$x_{k}^{(\beta)}$, and the complexity caused by the number of different state
behaviors collapses. ii) Otherwise, some required disturbances are missing and
$u_{k}^{(\beta)}$ is not applicable. Then only a tailored control law (so
$u(k)$ as in (18), and not $u_{k}^{(\beta)}$) is applicable, and the
satisfaction of state and input constraints cannot be guaranteed.

Now, the main idea is the following: In each time-step $k$, adapt the
likelihoods to satisfy the state and input constraints ${\delta_{x}}$ and
${\delta_{u}}$ according to:

$\displaystyle{\delta_{u}}=\gamma_{k}^{(u)}\cdot\alpha_{k},\quad{\delta_{x}}=\gamma_{k}^{(x)}\cdot{\prod_{t=1}^{k-1}\alpha_{t}},$
(31)

where $\alpha_{k}$ is the probability that the determined control law
$u_{k}^{(\beta)}$ is applicable (case i)), and where $\gamma_{k}^{(u)}$ and
$\gamma_{k}^{(x)}$ are tailored likelihoods. Note, that the probability for
occurrence of state $x_{k}^{(\beta)}$ is stochastically depended of all prior
$\alpha_{t},\;t<k$, thus the product is used. Then, state and input
constraints under the condition of control law $u_{k}^{(\beta)}$ are satisfied
at least with probability $\gamma_{k}^{(x)}$ and $\gamma_{k}^{(u)}$, and the
satisfaction of (8) and (10) are guaranteed.
Note that the tailored likelihoods in (31) depend on $\alpha_{k}$ and case
$\textbf{P}^{(\beta)}$, which are defined in the following statements: Let
$\textbf{P}^{(\beta)}$ collect all indicator functions according to:

$\displaystyle\mathds{1}_{k,r}^{(\beta)}=\begin{cases}1&\text{if}\quad
p_{k,r}\geq\alpha_{k}\\\ 0&\text{else.}\end{cases},$ (32)

and
$\textbf{W}=\operatorname{blkdiag}(\mathcal{W}_{0},\dots,\mathcal{W}_{H-1})$
all covariance matrices over the prediction horizon, then the following is
stated:

######  Proposition 11.

Let $(C_{u}^{[i]},b_{u}^{[i]})$ and $(C_{x_{k+1}}^{[j]},b_{x_{k+1}}^{[j]})$
denote the $i$-th half-space of the polytopic input set $\mathbb{U}$, and the
$j$-th half-space of the polytopic state set $\mathbb{X}_{k+1}$. Then, the
state constraints (8) and the input constraints (10) are satisfied, if the
LMIs in (30) hold when using:

$\displaystyle\textbf{M}_{k}=\operatorname{row}_{k}(\textbf{M}),\quad\textbf{P}_{k}^{(\beta)}=\operatorname{row}_{k}(\textbf{P}^{(\beta)}),$
(33)
$\displaystyle\mathcal{A}_{k}=\operatorname{row}_{k}(\mathcal{A}),\quad\mathscr{E}_{k}^{(\beta)}=\operatorname{row}_{k}(\mathscr{E}^{(\beta)}),$
(34)
$\displaystyle\\{c_{u},c_{x}\\}=(F_{\chi}^{2})^{-1}(\\{\gamma_{k}^{(u)},\gamma_{k}^{(x)}\\},\\{n_{u},n_{x}\\}),$
(35)

for each $i\in\\{1,\dots,n_{\mathbb{U}}\\}$ and
$j\in\\{1,\dots,n_{\mathbb{X}_{k+1}}\\}$, and if $\alpha_{k}$ is chosen to:

$\displaystyle\alpha_{k}$ $\displaystyle=\min_{r\in
R_{k}}\\{p_{k,r},1\\},\;R_{k}=\left\\{r\in\mathbb{N}_{\geq
0}\left|\begin{array}[]{l}p_{k,r}>{{\delta_{u}}}\end{array}\right.\right\\}.$
(37)
$\displaystyle{\text{s.t.:}\quad\prod_{t=1}^{k-1}\alpha_{t}\geq{\delta_{x}}}$
(38)

###### Proof.

See Appendix -D. ∎

Note that the above choice of determining $\alpha_{k}$ is not the only
possible one, and other options satisfying
$\alpha_{k}>\max\\{{\delta_{x}},{\delta_{u}}\\}$ are conceivable.

###  V-D Determination of feedback matrices by SDP

For minimizing the state covariance matrices, an auxiliary matrix
$\textbf{S}=\textbf{S}^{T}\succcurlyeq 0$ is introduced (motivated by the
scheme in [7]), which determines an upper bound for the cross-covariance of
the state trajectory $\textbf{x}^{(\beta)}$:

$\displaystyle\textbf{S}\succcurlyeq\operatorname{cov}\left[\textbf{x}^{(\beta)}\right]=\mathscr{E}^{(\beta)}\textbf{W}\mathscr{E}^{(\beta)^{T}}$
(40)

formulated by the following LMI:

$\displaystyle\mathcal{L}\mathfrak{s}=\begin{bmatrix}\quad\textbf{S}\quad\hbox{}&\mathscr{E}^{(\beta)}\textbf{W}\\\
\star&\textbf{W}\end{bmatrix}\succcurlyeq 0.$ (41)

An SDP problem with stacked weights $\mathcal{Q}$, and $\mathcal{R}$ is stated
as:

$\displaystyle\min_{\textbf{V},\textbf{M}}\left\lVert\operatorname{\mathbb{E}}\left[\textbf{x}\right]\right\rVert_{\mathcal{Q}}+\left\lVert\operatorname{\mathbb{E}}\left[\textbf{u}\right]\right\rVert_{\mathcal{R}}+\left\lVert\operatorname{trace}\left(\textbf{S}\right)\right\rVert_{\mathcal{S}}$
(42) s.t.:
$\displaystyle\operatorname{\mathbb{E}}\left[\textbf{x}\right]=(\textbf{A}+\textbf{B}\textbf{V})x_{0},\quad\operatorname{\mathbb{E}}\left[\textbf{u}\right]=\textbf{V}x_{0},$
(43) $\displaystyle\mathcal{L}\mathfrak{s}\succcurlyeq 0,$ (44)
$\displaystyle\mathcal{L}\mathfrak{x}^{[i]}_{k+1}\succcurlyeq
0\quad\forall\;i\in\\{1,\dots,n_{C_{x_{k+1}}}\\},\forall\;k\in\mathbb{N}_{H},$
(45) $\displaystyle\mathcal{L}\mathfrak{u}^{[i]}_{k}\succcurlyeq
0\quad\forall\;i\in\\{1,\dots,n_{C_{u}}\\},\forall\;k\in\mathbb{N}_{H},$ (46)
$\displaystyle\mathcal{Q}=\operatorname{blkdiag}(Q_{0},\dots,Q_{H}),$ (47)
$\displaystyle\mathcal{R}=\operatorname{blkdiag}(R_{0},\dots,R_{H-1}),$ (48)

with symmetric and positive semi definite weights $Q_{k}$, $R_{k}$,
$\mathcal{S}$. The optimal solution of this problem is denoted by
$(\textbf{V}^{\star},\textbf{M}^{\star})$, and an optimal state feedback
matrix $\textbf{K}^{\star}$ follows to specify the control law (16) (see [8,
Theorem 7]). In case, more information is available in $k$ than expected
according to $\textbf{P}^{(\beta)}$, the corresponding feedback matrices
$M_{k,r}$ are chosen to zero in the optimization, such that
$(\textbf{P}\circ\textbf{M})=(\textbf{P}^{(\beta)}\circ\textbf{M})$ holds,
thus guaranteeing the satisfaction of the constraints.

##  VI Numerical Example

To illustrate the proposed method, it is applied to a severely disturbed
example with the dynamics of type (5) using:

$\displaystyle A{\small=}\small\begin{bmatrix}1&0.3\\\
0&1\end{bmatrix},\,\quad B{\small=}\small\begin{bmatrix}0.045\\\
0.3\end{bmatrix},\,\quad E{\small=}I_{2},$ (49)

and the initial state and disturbance distributions:

$\displaystyle x(t_{0})=\small\begin{bmatrix}85\\\
-5\end{bmatrix},\;\mathcal{W}(k){\small=}\small\begin{bmatrix}0.1&0\\\
0&3\end{bmatrix}\,\forall k\in\mathbb{N}_{H}.$ (50)

The input constraint is defined by
$\mathbb{U}=\left\\{u\left|\begin{array}[]{l}\left\lVert u\right\rVert\leq
30\end{array}\right.\right\\}$, and the chance constraint for this region is
selected to:

$\displaystyle\operatorname{\mathbb{P}}\big{(}u(k)\in\mathbb{U}\big{)}\geq{\delta_{u}}=0.95.$
(51)

A time-invariant state constraint is defined to:

$\displaystyle\mathbb{X}_{k+1}=\left\\{x\left|\begin{array}[]{l}-20\leq
x_{1}(k)\leq 100,\\\ -40\leq x_{2}(k)\leq
30\end{array}\right.\right\\}\;\forall k\in\mathbb{N}_{H}$ (54)

with the probability
$\operatorname{\mathbb{P}}\big{(}x(k)\in\mathbb{X}_{k}\big{)}\geq{\delta_{x}}=0.9$.

Figure 3:  Simulation results for even values of $k$, and the two
parameterizations of the communication link: i) top , ii) bottom. The initial
states are shown as blue square and the admissible state space as gray box.
The ${\delta_{x}}$-confidence ellipsoids of the state are shown as blue
ellipsoids for the case of successful transmission of any information, and as
red ellipsoids for the case that communication fails in every time-step. The
$\gamma_{k}^{(x)}$-confidence ellipsoids used in the optimization problem for
$\alpha_{k}$ are marked as green ellipsoids. In addition, the simulated states
obtained for 1000 Monte Carlo simulations are shown as colored dots (different
colors for selected $k$).

To demonstrate the proposed methodology, the probability of the communication
link according to Fig. 1(b) is chosen with two different parameterizations, a
constant one and an alternating one:

  * i)

$p(k)=0.8\;\forall k\in\mathbb{N}_{H}$,

  * ii)

$\left[p(0)\dots p(H-1)\right]:=0.1\cdot\left[1\;\;9\;\;1\quad
1\;\;9\;\;1\quad\dots\right]$.

Note that these numbers are chosen, by intention, to model a poorly performing
communication network (in fact much worse than typically observed in practice)
– this serves to demonstrate that the proposed scheme nevertheless leads to
good control performance.

The control horizon is chosen to $H=12$, and the cost functional is
parameterized with time-invariant weights $Q_{k}=\operatorname{diag}(2,1)$,
$R_{k}=0.1$, and $\mathcal{S}=100$. The optimization problems were solved on a
$3.4\text{\,}\mathrm{GHz}$ Quad-Core CPU using Matlab2018a and the solver
Mosek, and the computations took an average time of around
$0.6\text{\,}\mathrm{s}$.

$\varnothing$ MCS | $\sum\left\lVert x(k)\right\rVert_{Q_{k}}$ | $\sum\left\lVert x(k)-\bar{x}(k)\right\rVert_{Q_{k}}$ | $\sum\left\lVert\bar{x}(k)\right\rVert_{Q_{k}}$
---|---|---|---
version i) | 3920 | 26.8 | 5.03
version ii) | 3945 | 74.5 | 5.03
Table I: Performance criteria for both versions.

The simulation results for the two parameterizations are shown in Fig. 3. It
is obvious that the $\gamma_{k}^{(x)}$-confidence ellipsoids are contained in
the admissible state space, and that the confidence ellipsoids and the
simulated states are reliably steered towards the origin. For an adequate
comparison, the same values for the disturbances underlie the two sets of
Monte Carlo simulations (MCS).

While for i) (top part of Fig. 3) the ellipsoids are obtained smaller than in
version ii) (bottom), the result of ii) is suitable, too, as the average
probability for successful transmission of information is only
$\bar{p}=0.3\bar{6}$, thus less than a half of the $\bar{p}$ in version i).
Note that the optimization problem considers the different performance of the
communication network, such that the blue and green ${\delta_{x}}$-confidence
ellipsoids differ in both versions.

In average over all simulated time-steps, the state $x(k)$ lies to $90.32\%$
within the $y_{k}^{(x)}$-confidence ellipsoid for version i), and the input
$u(k)$ with $96.508\%$ within the $y_{k}^{(u)}$-confidence ellipsoid. For
version ii), the probabilities are $90.48\%$ for the state, and $98.4\%$ for
the input. Thus, the state and input constraint are satisfied with only very
small conservatism.

For quantitative assessment, Table I shows three performance criteria for both
network parametrizations evaluated in Monte Carlo simulation: The first column
contains the weighted distance of all states to the origin (representing how
good the control goal of approaching the origin is met), which is similar for
both versions. The second column shows the weighted distance of all states to
the corresponding expected state, thus representing the state uncertainty. Due
to the poor network performance, a noticeable uncertainty of the states is
obtained, where a worse performing network (according to the average link-
probability $\bar{p}$) leads to an increase of the uncertainty of the
controlled system. The last column contains the weighted distances of the
expected states to the origin, which is similar for both versions.

##  VII CONCLUSIONS

This paper transfers a balanced stochastic optimal control scheme to a control
system with LTI dynamics and embedded stochastically modeled communication
link, where additive disturbances and uncertain communication has to be dealt
with. Uncertainties of the communication network are projected onto a tailored
probability for the satisfaction of state and input constraints. Furthermore,
balancing between the minimization of uncertainties and the minimization of
expected distances to the origin is realized. The simulation results show,
that the constraints are satisfied with nearly no conservatism, and that a
poorly performing communication network only increases the uncertainty and not
the performance of the control system (in the sense of state deviation from
the reference). In general, the presented optimization problem is independent
of the complexity of the communication network (such as the number of nodes or
links, and possible time-variance) and its imperfections (like delay, packet
loss etc.) by the use of a Markov chain modeling the Age of Information. In
consequence, the SDP for controller synthesis results with the same complexity
as in the case without networks.

This paper opens the field of balanced stochastic optimal control in the
considered understanding for larger structures of networked control systems,
which are subject of current and future work.

###  -A Vector Notation

The matrices $\textbf{A}\in\mathbb{R}^{(H+1)n_{x}\times n_{x}}$ and
$\textbf{C}\in\mathbb{R}^{(H+1)n_{x}\times H\cdot n_{x}}$ have the following
structure:

$\displaystyle\textbf{A}=\small\begin{bmatrix}I_{n_{x}}\\\ A\\\ \vdots\\\
A^{H}\end{bmatrix},\quad\textbf{C}=\small\begin{bmatrix}0&0&\cdots&0\\\
I_{n_{x}}&0&\cdots&0\\\ A&I_{n_{x}}&\cdots&0\\\ \vdots&\vdots&\ddots&0\\\
A^{H-1}&A^{H-2}&\cdots&I_{n_{x}}\end{bmatrix},$ (55)

while $\textbf{B}\in\mathbb{R}^{(H+1)n_{x}\times H\cdot n_{u}}$ and
$\textbf{E}\in\mathbb{R}^{(H+1)n_{x}\times H\cdot n_{w}}$ are chosen to
$\textbf{B}=\textbf{C}(I_{n_{x}}\otimes B)$, and
$\textbf{E}=\textbf{C}(I_{n_{x}}\otimes E)$ respectively.
The disturbance feedback matrices $\textbf{V}\in\mathbb{R}^{H\cdot n_{u}\times
n_{x}}$, and $\textbf{M}\in\mathbb{R}^{H\cdot n_{u}\times H\cdot n_{w}}$ are
defined to:

$\displaystyle\textbf{V}=\small\begin{bmatrix}V_{0}\\\ V_{1}\\\ \vdots\\\
V_{H-1}\end{bmatrix},\quad\textbf{M}=\small\begin{bmatrix}0&0&\cdots&0\\\
M_{1,0}&0&\cdots&0\\\ \vdots&\ddots&\ddots&\vdots\\\
M_{H-1,0}&\cdots&M_{H-1,H-2}&0\end{bmatrix}.$ (56)

###  -B Derivation of equation (22)

Each probability $p_{k,r}$ is given according to (19) with respect to the
probability for the AoI (15):

$\displaystyle p_{k,r}$
$\displaystyle=\operatorname{\mathbb{E}}\left[\mathds{1}_{k,r}\right]=\operatorname{\mathbb{P}}\big{(}r<k-a(k)\big{)}=\operatorname{\mathbb{P}}\big{(}a(k)\leq
k-r-1\big{)}$ (57)
$\displaystyle=\operatorname{\mathbb{P}}\big{(}a(k)=0\big{)}+\ldots+\operatorname{\mathbb{P}}\big{(}a(k)=k-r-1\big{)}=\sum_{l=0}^{k-r-1}\mu_{k}[l].$
(58)

###  -C Proof of Lemma 8

First, consider the system dynamics (5) and the control law (16), for the case
of available state information:

$\displaystyle x_{k+1}=f(x_{k},u_{k},w_{k}),\quad
u_{k}=\kappa_{k}(x_{0},\ldots,x_{k}),$ (59)

with general functions:

$\displaystyle f$
$\displaystyle:\mathbb{R}^{n_{x}}\times\mathbb{R}^{n_{u}}\times\mathbb{R}^{n_{w}}\rightarrow\mathbb{R}^{n_{x}},$
(60) $\displaystyle\kappa_{k}$
$\displaystyle:\mathbf{1}_{1,k+1}\otimes\mathbb{R}^{n_{x}}\rightarrow\mathbb{R}^{n_{u}}.$
(61)

The recursive state equation can be reformulated to a series of functions
$f_{k}$ and $\kappa_{k}$, where control laws are rewritten to
$\tilde{\kappa}_{k}$, e.g. for $k\in\\{0,1\\}$:

$\displaystyle u_{0}$ $\displaystyle=\kappa_{0}(x_{0}),$ (62) $\displaystyle
x_{1}$
$\displaystyle=f(x_{0},u_{0},w_{0})=f(x_{0},\kappa_{0}(x_{0}),w_{0})=:f_{1}(x_{0},w_{0}),$
(63) $\displaystyle u_{1}$
$\displaystyle=\kappa_{1}(x_{0},x_{1})=\kappa_{1}(x_{0},f_{1}(x_{0},w_{0}))=:\tilde{\kappa}_{1}(x_{0},w_{0}),$
(64)

or for arbitrary $k\in\mathbb{N}\geq 0$:

$\displaystyle u_{k}$
$\displaystyle=\tilde{\kappa}_{k}(x_{0},w_{0},\dots,w_{k-1}),$ (65)
$\displaystyle x_{k}$ $\displaystyle=f_{k}(x_{0},w_{0},\dots,w_{k-1}),$ (66)

or for linear functions:

$\displaystyle\tilde{\kappa}_{k}$ $\displaystyle=a^{(x)}_{0}\cdot
x_{0}+a^{(w)}_{0}\cdot w_{0}+\ldots+a^{(w)}_{k-1}\cdot w_{k-1},$ (67)
$\displaystyle f_{k}$ $\displaystyle=b^{(x)}_{0}\cdot x_{0}+b^{(w)}_{0}\cdot
w_{0}+\ldots+b^{(w)}_{k-1}\cdot w_{k-1},$ (68)

with $a_{i}$ and $b_{i}$ [8].

Secondly, if $a_{k}\geq 0$, (16) is used in combination with (17). Thus, with
the last certain information $x_{l}\rightarrow l:=k-a_{k}$, it holds that all
states $x_{l+1}$ up to $x_{k}$ are not available to the controller:

$\displaystyle\underbrace{x_{0},\dots,x_{l},}_{\text{available}}\underbrace{x_{l+1},\ldots,x_{k}}_{\text{not
available}}.$ (69)

With (16) and (17), it holds that:

$\displaystyle u_{k}$
$\displaystyle=\kappa_{k}(x_{0},\ldots,x_{l},x_{l+1|l},\ldots,x_{k|l}),$ (70)
$\displaystyle x_{k|l}$ $\displaystyle=f_{k,l}(x_{l},u_{l},\ldots,u_{k-1}),$
(71)

with functions $\kappa_{k}$ defined in (61) and:

$\displaystyle
f_{k,l}:\mathbb{R}^{n_{x}}\times\left(\textbf{1}_{1,a_{k}}\otimes\mathbb{R}^{n_{u}}\right)\rightarrow\mathbb{R}^{n_{x}}.$
(72)

In (70), the states with index up to $l$ can be expressed by the functions
given in (66). For the first of the remaining states/inputs in the function
arguments, it follows with (66) that:

$\displaystyle x_{l+1|l}$
$\displaystyle=f_{l+1,l}(x_{l},u_{l})=:\tilde{f}_{l+1,l}(x_{0},w_{0},\dots,w_{l-1}),$
(73) $\displaystyle u_{l+1}$
$\displaystyle=\kappa_{l+1}(x_{0},x_{l},x_{l+1|l})=:\tilde{\kappa}_{l+1}(x_{0},w_{0},\dots,w_{l-1}).$
(74)

Recursively for $k>l$, the states:

$\displaystyle x_{k|l}=\tilde{f}_{k,l}(x_{0},w_{0},\dots,w_{l-1}),$ (75)

and the inputs:

$\displaystyle u_{k}=\tilde{\kappa}_{k}(x_{0},w_{0},\dots,w_{l-1})$ (76)

are obtained. Again, with linear functions $f_{k,l}$ and $\kappa_{k}$, the
control law $\tilde{\kappa}_{k}$ in (76) is a linear function of its
arguments, thus (again with a set of parameters $a_{i}$) one can write:

$\displaystyle u_{k}=a_{0}^{(x)}\cdot x_{0}+a_{0}^{(w)}\cdot
w_{0}+\ldots+a_{l-1}^{(w)}\cdot w_{l-1}.$ (77)

Eventually, (77) feeds back all disturbances $w_{r}$ with $r\leq
l-1=k-a_{k}-1\Leftrightarrow r<k-a_{k}$. With $V_{k}:=a_{0}^{(x)}$ and
$M_{k,r}:=a_{r}^{(w)}$, (77) equals the disturbance feedback in Lemma 8 for
time $k$. ∎

###  -D Proof of Proposition 11

According to Proposition 11 the co-domain of $\alpha_{k}$ is given by:

$\displaystyle\alpha_{k}\in\;\left]\max({\delta_{u}},{\delta_{x}}),\;\;1\right],$
(78)

such that:

$\displaystyle\gamma_{k}^{(u)}=\frac{{\delta_{u}}}{\alpha_{k}}\in\left[{\delta_{u}},\;\;1\right],\quad\gamma_{k}^{(x)}{\geq}\frac{{\delta_{x}}}{\alpha_{k}}\in\left[{\delta_{x}},\;\;1\right],$
(79)

hold. Now recall (26), and let
$\textbf{P}^{(>\beta)}\subseteq\textbf{P}^{(\Omega)}$ denote a subset of
indicator matrices $\textbf{P}^{(i)}$, for which all entries satisfy the
inequality $\mathds{1}_{k,r}^{(i)}\geq\mathds{1}_{k,r}^{(\beta)}\;\forall
k,r\in\mathbb{N}_{H}$.

Then the following holds true with $\theta$ denoting the behavior of the
communication network according to
$\textbf{P}^{(\theta)}\in\textbf{P}^{(\Omega)}$:

$\displaystyle\operatorname{\mathbb{P}}\big{(}x_{k}\in\mathbb{X}_{k}\big{)}=\sum_{i=1}^{o}\operatorname{\mathbb{P}}\big{(}x_{k}^{(i)}\in\mathbb{X}_{k}|\theta=i\big{)}$
(80)
$\displaystyle=\sum_{i=1}^{o}\operatorname{\mathbb{P}}\left(\theta=i\right)\cdot\operatorname{\mathbb{P}}\big{(}x_{k}^{(i)}\in\mathbb{X}_{k}\big{)}\leq\prod_{t=1}^{k-1}\alpha_{t}\cdot\operatorname{\mathbb{P}}\big{(}x_{k}^{(\beta)}\in\mathbb{X}_{k}\big{)}\leq{\delta_{x}},$
(81)

if all control laws $u_{k}^{(i)}$, for which
$\textbf{P}^{(i)}\in\textbf{P}^{(>\beta)}$ holds, are truncated to
$u_{k}^{(\beta)}$. Therefore, the necessary condition results with (31) to:

$\displaystyle\operatorname{\mathbb{P}}\big{(}x_{k}^{(\beta)}\in\mathbb{X}_{k}\big{)}\leq\frac{{\delta_{x}}}{{\prod_{t=1}^{k-1}\alpha_{t}}}=\gamma_{k}^{(x)}.$
(82)

This is at least satisfied, if the $\gamma_{k}^{(x)}$-confidence ellipsoid of
the state $x_{k}^{(\beta)}$ lies within the admissible state set, i.e.:

$\displaystyle\mathcal{X}_{k}^{(\gamma,\beta)}\subseteq\mathbb{X}_{k}.$ (83)

Following the same steps of the proof to [8, Proposition 5], condition (83) is
satisfied if the LMI $\mathcal{L}\mathfrak{x}_{k+1}^{[j]}\succcurlyeq 0$ given
in (30) is satisfied with the tailored likelihood $\gamma_{k}^{(x)}$ for each
half-space $j\in\\{1,\dots,n_{\mathbb{X}_{k+1}}\\}$ of $\mathbb{X}_{k+1}$,
where $n_{\mathbb{X}_{k+1}}$ denotes the number of half-spaces.
The reasoning for the input follows analogously, but with $\alpha_{k}$ instead
of the product. ∎

## Acknowledgment

Partial financial support by the German Research Foundation (DFG) within the
research priority program SPP 1914: Cyberphysical Networking is gratefully
acknowledged.

## References

  * [1] J. P. Hespanha, P. Naghshtabrizi, and Y. Xu, “A survey of recent results in networked control systems,” _Proceedings of the IEEE_ , vol. 95, no. 1, pp. 138–162, 2007.
  * [2] G. Antonelli, “Interconnected dynamic systems: An overview on distributed control,” _IEEE Control Systems Magazine_ , vol. 33, no. 1, pp. 76–88, 2013\.
  * [3] M. Klugel, M. Mamduhi, O. Ayan, M. Vilgelm, K. H. Johansson, S. Hirche, and W. Kellerer, “Joint cross-layer optimization in real-time networked control systems,” _IEEE TCNS_ , pp. 1–1, 2020.
  * [4] J. Hahn, R. Schoeffauer, G. Wunder, and O. Stursberg, “Distributed mpc with prediction of time-varying communication delay,” _IFAC-PapersOnLine_ , vol. 51, no. 23, pp. 224–229, 2018.
  * [5] D. H. van Hessem and O. H. Bosgra, “A conic reformulation of model predictive control including bounded and stochastic disturbances under state and input constraints,” in _41st CDC_ , vol. 4, 2002, pp. 4643–4648.
  * [6] F. Oldewurtel, C. N. Jones, and M. Morari, “A tractable approximation of chance constrained stochastic mpc based on affine disturbance feedback,” in _47th CDC_ , 2008, pp. 4731–4736.
  * [7] L. Asselborn and O. Stursberg, “Probabilistic control of uncertain linear systems using stochastic reachability,” _IFAC-PapersOnLine_ , vol. 48, no. 14, pp. 167–173, 2015.
  * [8] J. Hahn and O. Stursberg, “Balanced stochastic optimal control of uncertain linear systems with constraints,” _IFAC-PapersOnLine_ , vol. 53, no. 2, pp. 7172–7178, 2020, 21th IFAC World Congress.
  * [9] D. Wu, J. Wu, S. Chen, and J. Chu, “Stability of networked control systems with polytopic uncertainty and buffer constraint,” _TAC_ , vol. 55, no. 5, pp. 1202–1208, 2010.
  * [10] P. V. Zhivoglyadov and R. H. Middleton, “Networked control design for linear systems,” _Automatica_ , vol. 39, no. 4, pp. 743–750, 2003.
  * [11] R. Schoeffauer and G. Wunder, “Predictive network control and throughput sub-optimality of max weight,” in _European Conf. on Networks and Communications_ , 2018, pp. 1–6.
  * [12] J. Hahn and O. Stursberg, “Robust distributed mpc for disturbed affine systems using predictions of time-varying communication,” in _18th ECC_ , 2019, pp. 56–62.
  * [13] F. Borrelli, _Constrained optimal control of linear and hybrid systems_. Springer, 2003, vol. 290.
  * [14] P. J. Goulart, E. C. Kerrigan, and J. M. Maciejowski, “Optimization over state feedback policies for robust control with constraints,” _Automatica_ , vol. 42, no. 4, pp. 523–533, 2006.
  * [15] D. Groß and O. Stursberg, “Distributed predictive control of communicating and constrained systems,” _ZAMM-Journal of Applied Mathematics and Mechanics_ , vol. 94, no. 4, pp. 303–316, 2014.