# Distributed Feedback Optimization of Linear Multi-agent Systems via
Decentralized Online Gradient Descent

Amir Mehrnoosh and Gianluca Bianchin  The authors are with ICTEAM Institute
and the Department of Mathematical Engineering at the Université catholique de
Louvain, Belgium. {amir.mehrnoosh<EMAIL_ADDRESS>
# Distributed Feedback Optimization of Linear Multi-agent Systems

Amir Mehrnoosh and Gianluca Bianchin  The authors are with ICTEAM Institute
and the Department of Mathematical Engineering at the Université catholique de
Louvain, Belgium. {amir.mehrnoosh<EMAIL_ADDRESS>
###### Abstract

Feedback optimization is an increasingly popular control paradigm to optimize
dynamical systems, accounting for control objectives that concern the system’s
operation at steady-state. Existing feedback optimization techniques heavily
rely on centralized system and controller architectures, and thus suffer from
scalability and privacy issues when systems become large-scale. In this paper,
we propose and study a distributed architecture for feedback optimization, in
which each agent updates its local control state by combining the average of
its neighbors with a local negative-gradient step. Under convexity and
smoothness assumptions, we establish convergence of the control method to a
fixed point. By reinforcing the assumptions to restricted strong convexity of
the cost, we show that our algorithm converges linearly to a neighborhood of
the optimal point, where the size of the neighborhood depends on the choice of
the stepsize. Simulations corroborate the theoretical results.

Index Terms – Optimization algorithms, feedback optimization, distributed
control, multi-agent systems.

##  I Introduction

Optimal steady-state regulation is concerned with the problem of controlling a
dynamical systems to an optimal steady-state point, as characterized by a
mathematical optimization problem [1]. The classical approach to tackle this
goal relies on the principle of separation between planning and control,
whereby the optimization problem is solved beforehand (offline) to determine
optimal system states, which are then inputed as references to controllers
responsible for regulating the system to these states. Remarkably, a key
assumption in this approach is that disturbances are known beforehand and fed
to the optimization solver; this allows the optimization to be solved with
high precision to generate the required reference states. Unfortunately, in
most control applications, disturbances are unknown. In fact, often the main
objective of a control system is to ensure optimality in the face of
unmeasurable disturbances or imprecise system knowledge. Notably, because
disturbances may perturb optimal steady states, classical batch optimization
algorithms fail [2] when are approximately known or vary after the
optimization has been solved.

Recently, several authors have studied optimal steady-state regulation
problems in a centralized setting. A list of representative works on this
topic (necessarily incomplete) includes [3, 4, 5, 6, 7, 8, 9] – see also the
recent developments using zeroth order algorithms and data-driven approaches
[10, 11, 12]. Feedback optimization controllers have gained popularity thanks
to their capability to regulate physical systems to optimal steady-state
points while rejecting constant [4] or time-varying disturbances [7, 8] The
central idea consists of adapting numerical optimization algorithms to operate
as feedback controllers; this is achieved by using an inexact gradient
evaluated using real-time measurements to update control inputs without
requiring the exact plant model and disturbances. This feature endows feedback
optimization with the versatility to handle various scenarios. All of these
methods are designed to be implemented in a centralized architecture, and thus
suffer from scalability issues when systems become large-scale, as well as
privacy concerns when cost functions or feedback signals need to be maintained
private. This work departs from this existing literature by focusing on the
problem of optimal steady-state regulation for systems with a distributed (or
multi-agent) architecture. This connects our work with the body of literature
on distributed optimization. Distributed gradient descent was proposed in
[13], studied in [14], a diminishing stepsize was used in [15] to ensure exact
convergence; see also [16, 17, 18, 19]. Particularly related to our problem
are the works on distributed optimization of nonlinear systems [20], which
however relies on a separation between optimization and tracking control;
[21], which relies on an inexact gradient obtained by approximating the
sensitivity matrix by its diagonal elements (corresponding to a situation
where the coupling between different subsystems is ignored); [22], which
focuses on systems modeled as a static linear map.

This work features three main contributions. First, we propose a distributed
architecture for the optimal steady-state regulation problem, and we propose a
distributed control algorithm to address this problem. Our algorithm is
inspired by distributed optimization approaches and combines a gradient-
descent step with a consensus operation to simultaneously solve an
optimization and seek an agreement between the agents. Second, we present a
proof of convergence to a fixed point for the controller-system state. Our
technical arguments provide guidelines on how to choose the (sufficiently
small) controller stepsize to guarantee convergence of the controlled system.
Third, provide an explicit bound for the control error. Precisely, we show
that under restricted strong convexity assumptions, the controller state
converges linearly to a neighborhood of the optimal point. In line with the
existing literature [14], the size of such a neighborhood depends on the
choice of the controller stepsize. Intuitively, convergence cannot be exact
since distributed controllers need to average between moving in a direction
that decreases the gradient while maintaining an agreement with the other
controllers.

The rest of the article is organized as follows. Section II formalizes the
problem of distributed optimal steady-state regulation, Section III
illustrates the proposed controller, Section IV presents the two main
contributions of this work, being the convergence analysis and error bound for
the proposed controller. Section V validates the findings through numerical
simulations, and Section VI concludes the paper.

Notation. For a symmetric matrix $W$, we denote its eigenvalues by
$\lambda_{1}(W)\geq\lambda_{2}(W)\geq\cdots\geq\lambda_{n}(W).$ We assume that
the mixing matrix $W=[w_{ij}]$ is symmetric and doubly stochastic. The
eigenvalues of $W$ are real and sorted in a nonincreasing order
$1=\lambda_{1}(W)\geq\lambda_{2}(W)\geq\cdots\geq\lambda_{n}(W){>}-1.$ Let the
second-largest eigenvalue magnitude of W be:

$\displaystyle\beta:=\max\\{|\lambda_{2}(W)|,|\lambda_{n}(W)|\\}.$ (1)

##  II Problem Setting

Figure 1: Distributed system architecture considered in this work (cf. (2)).
Each local controller actuates the corresponding subsystem (green lines), by
using global feedback information (red lines), see (9).

###  II-A System structure and problem formulation

Consider a distributed system composed of $N$ subsystems
$\mathcal{V}_{x}=\\{1,\dots,N\\};$ we describe the physical couplings between
interacting subsystems using a directed graph (called system graph)
$\mathcal{G}_{x}=(\mathcal{V}_{x},\mathcal{E}_{x})$ where
$\mathcal{E}_{x}\subseteq\mathcal{V}_{x}\times\mathcal{V}_{x}.$ See Fig.
1-(System Layer). Each subsystem $i\in\mathcal{V}$ is described by a local
state $x_{i}^{k}\in{}^{n_{i}},$ which updates for $k\in\mathbb{Z}_{\geq 0}$
as:

$\displaystyle x_{i}^{k+1}$
$\displaystyle=\sum_{j\in\mathcal{N}_{i}}A_{ij}x_{j}^{k}+B_{i}u_{i}^{k}+E_{i}w_{i},$
(2)

where $\mathcal{N}_{i}$ denotes the set of in-neighbors of $i,$
$u_{i}^{k}\in{}^{m_{i}}$ is the local control decision, and
$A_{ij}\in{}^{n_{i}\times n_{j}},B\in{}^{n_{i}\times m_{i}}.$ In (2),
$w^{i}\in{}^{r_{i}}$ with $E_{i}\in{}^{n_{i}\times r_{i}},$ models a constant
unknown disturbance acting on subsystem $i$. In what follows, we let
$n:=\sum_{i}n_{i},$ $m:=\sum_{i}m_{i},$ and $r:=\sum_{i}r_{i}.$ We assume that
the local controllers have access to global output measurements of the system
state, of the form:

$\displaystyle y^{k}$
$\displaystyle=\sum_{i=1}^{n}C_{i}x_{i}^{k}+D_{i}u_{i}^{k},$ (3)

where $C_{i}\in{}^{p_{i}\times n_{i}}$ and $D_{i}\in{}^{p_{i}\times m_{i}};$
we let $p:=\sum_{i}p_{i}.$

In vector form, (2) reads as:

$\displaystyle x^{k+1}$ $\displaystyle=Ax^{k}+Bu^{k}+Ew,$ $\displaystyle
y^{k}$ $\displaystyle=Cx^{k}+Du^{k},$ (4)

where $x^{k}=(x_{1}^{k},\dots,x_{n}^{k})\in{}^{N}$ is the vector of states,
$u^{k}=(u_{1}^{k},\dots,u_{N}^{k})\in{}^{m}$ is the vector of inputs,
$w=(w_{1},\dots,w_{N})\in{}^{r}$ is the vector of disturbances, and
$A=[A_{ij}],$ $B=[B_{ij}],$ $C=[C_{1},\dots,C_{N}],$ $D=[D_{1},\dots,D_{N}],$
and $E=[E_{1},\dots,E_{N}]$ are block matrices.

In what follows, we use the notation

$\displaystyle G(z)$ $\displaystyle=C(zI-A)^{\scalebox{0.75}[0.9]{$-$}1}B+D,$
$\displaystyle H(z)$ $\displaystyle=C(zI-A)^{\scalebox{0.75}[0.9]{$-$}1}E,$
(5)

to denote the transfer functions from $u$ to $y$ and from $w$ to $y$,
respectively. The definition (II-A) is intended for the values of
$z\in\mathbb{C}$ for which the inverse is defined.

In what follows, we will adopt the compact notation

$\displaystyle G:=G(1),$ $\displaystyle H:=H(1).$

While we make no assumption on $\mathcal{G}_{x}$, we assume the following for
(2).

###### Assumption 1

(Stability and control properties of system) The system (2) is asymptotically
stable. Precisely, given $Q\succ 0,$ we let $P\succ 0$ be such that
$A^{\mathsf{T}}PA-P=-Q$.

Moreover, (II-A) is controllable and observable. $\square$

Controllability and observability are standard assumptions to guarantee that
control problems are well defined; when $A$ is not Schur stable, it can be
pre-stabilized using standard static state feedback techniques. Since the
disturbance $w$ is constant, at steady-state, $x,u,$ and $y$ have constant
values $x^{\text{ss}},u^{\text{ss}},$ and $y^{\text{ss}}$ (by Assumption 1),
with

$y^{\text{ss}}=Gu^{\text{ss}}+Hw.$

In this work, we study a control problem where the ensemble of controllers
seeks to collaboratively optimize the system at steady-state:

$\displaystyle\underset{u\in{}^{m}}{\text{minimize}}~{}~{}\sum_{i=1}^{n}\Phi_{i}(u,Gu+Hw),$
(6)

where $\Phi_{i}$ is assumed to be private and locally known only by agent $i.$
We remark that solutions to the optimization (6) cannot be computed explicitly
since $w$ is unknown.

We make the following assumptions on the optimization.

###### Assumption 2

(Lipschitz and Convexity of cost) For all $i$, $(u,y)\mapsto\Phi_{i}(u,y)$ is
proper closed convex, lower bounded, and Lipschitz differentiable with
constant $L_{\Phi_{i}}$. $\square$

Assumption 2 is standard in optimization. By defining
$\Pi^{\mathsf{T}}:=\begin{bmatrix}I_{m}&G^{\mathsf{T}}\end{bmatrix},$ this
assumption guarantees that there exists $L_{\Phi}$ such that

$\displaystyle\|\Pi^{\mathsf{T}}(\nabla\Phi(u,y)-\nabla\Phi(u^{\prime},y^{\prime}))\|\leq
L_{\Phi}\left\|\begin{bmatrix}u\\\ y\end{bmatrix}-\begin{bmatrix}u^{\prime}\\\
y^{\prime}\end{bmatrix}\right\|,$ (7)

for all $y,y^{\prime}\in{}^{n}$ and $u,u^{\prime}\in{}^{m}$. Further, in what
follows we denote the set of optimizers of (6) by:

$\displaystyle\mathcal{A}^{*}:=\\{(u^{*},x^{*})\;:\;(u^{*},x^{*})\text{ is a
first-order optimizer of~{}\eqref{eq:steady_state_optimization}}\\}.$

We assume that this set is nonempty and closed. We will further denote in
compact form:

$\Phi(u,y):=\sum_{i=1}^{n}\Phi_{i}(u,y).$

To describe the collaborative nature of the controllers, we will adopt an
undirected graph (hereafter called control graph)
$\mathcal{G}_{u}=(\mathcal{V}_{u},\mathcal{E}_{u})$ where
$\mathcal{V}_{u}=\mathcal{V}_{x}$ and
$\mathcal{E}_{u}\subseteq\mathcal{V}_{u}\times\mathcal{V}_{u}.$ See Fig.
1-(Control Layer). A pair of agents can collaboratively compute a control law
if and only if they are connected by means of a communication link in
$\mathcal{E}_{u}.$ Recall that a graph is connected if there exists a path
between any two nodes. We make the following assumption on the control graph.

###### Assumption 3

(Connectivity of the control graph) The graph $\mathcal{G}_{u}$ is connected.
$\square$

Under Assumption 3, there exists $W=[w_{ij}]\in{}^{n\times n},$ such that
$(i,j)\not\in\mathcal{E}_{u},$ implies $w_{ij}=0$ and that is symmetric and
doubly stochastic with $\beta<1$ (see (1)).

##  III Distributed controller design

A centralized version to solve the steady-state regulation problem (6) has
been studied in [12] and continuous-time counterparts [4, 8, 7]. In [12], the
authors propose a gradient-type controller:

$\displaystyle u^{k+1}=u^{k}-\eta\Pi^{\mathsf{T}}\nabla\Phi(u^{k},y^{k}),$ (8)

where $\eta>0$ denotes a scalar stepsize, being a design parameter. The
controller (8) implements a gradient-type iteration to solve the optimization
(6), modified by replacing the true gradient
$\Pi^{\mathsf{T}}\nabla\Phi(u^{k},Gu^{k}+Hw)$ with a measurements-based
version $\Pi^{\mathsf{T}}\nabla\Phi(u^{k},y^{k})$, which avoids the need to
measure $w$.

Unfortunately, implementing (8) requires a centralized knowledge of the
gradients $\\{\nabla\Phi_{i}\\}_{i\in\mathcal{V}_{u}},$ which is impractical
in our case since each $\Phi_{i}$ is known only locally. Departing from this,
we next propose a distributed version of (8) that can be implemented in our
control architecture (cf. Fig. 1). We propose an algorithm where each agent
$i\in\mathcal{V}_{u}$ holds a local copy $u^{k}_{(i)}\in{}^{m}$ of $u^{k}$,
and updates it as:

$\displaystyle
u_{(i)}^{k+1}=\sum_{j=1}^{N}w_{ij}u_{(j)}^{k}-\eta\Pi^{\mathsf{T}}\nabla\Phi_{i}(u^{k}_{(i)},y^{k}).$
(9)

In this control model, each agent $i$ updates its local state $u_{(i)}$ by
performing two steps: (i) it computes a weighted average of its neighbors’
states $\sum_{j=1}^{n}w_{ij}u_{(j)}^{k}$ to seek a consensus between the
agents, and (ii) it applies
$-\Pi^{\mathsf{T}}\nabla\Phi_{i}(u^{k}_{(i)},y^{k})$ to decrease
$\Phi_{i}(u^{k}_{(i)},y^{k}).$ We remark that this control law is distributed
in the sense that each agent $i$ requires only knowledge of the local
$\nabla\Phi_{i}$ (rather than of all gradients
$\\{\nabla\Phi_{i}\\}_{i\in\mathcal{V}_{u}}$). Notice also that each agent
requires measures of the global output feedback signal $y^{k}$; we leave a
generalization of the approach to the use of local output feedback signals as
the scope of future works. See Fig. 1 for an illustration of the control
architecture.

##  IV Convergence analysis and error bounds

In this section, we study the convergence of (9) when applied to control the
system (II-A). In the remainder, we employ the following notations of stacked
vectors:
$u^{k}_{(1:N)}:=(u_{(1)}^{k},u_{(2)}^{k},\dots,u^{k}_{(N)})\in{}^{mn}$ and

$\displaystyle\gamma(u^{k}_{(1:N)},y^{k}):=\begin{bmatrix}\Pi^{\mathsf{T}}(\nabla\Phi_{1}(u^{k}_{(1)},y^{k}))\\\
\vdots\\\
\Pi^{\mathsf{T}}\nabla\Phi_{n}(u^{k}_{(N)},y^{k}))\end{bmatrix}\in{}^{mn}.$

In vector form, the system (II-A) controlled by (9) reads as:

$\displaystyle x^{k+1}$ $\displaystyle=Ax^{k}+BSu^{k}_{(1:N)}+Ew,$ (10a)
$\displaystyle y^{k}$ $\displaystyle=Cx^{k}+DSu^{k}_{(1:N)},$ $\displaystyle
u_{(i)}^{k+1}$
$\displaystyle=\sum_{j\in\mathcal{N}_{i}}w_{ij}u_{(j)}^{k}-\eta\Pi^{\mathsf{T}}\nabla\Phi_{i}(u^{k}_{(i)},y^{k}),$
$\displaystyle i\in\mathcal{V}_{u},$ (10b)

where $S\in{}^{m\times mn}$ is given by:

$S=\text{diag}([I_{m_{1}},0,\dots],[0,I_{m_{2}},0,\dots],\dots[0,\dots,0,I_{m_{n}}]).$

###  IV-A Asymptotic convergence

The following result shows that, under a suitable choice of the stepsize
$\eta,$ the state of (10) converges asymptotically.

###### Theorem IV.1

(Convergence of the state sequences) Let Assumptions 1-2 hold, $W$ be such
that $\beta<1,$ and the stepsize
$\eta\leq\bar{\eta}:=\min\\{\eta_{1},\eta_{2},\eta_{3}\\},$ where

$\displaystyle\eta_{1}=\frac{1-2\mu+\lambda_{n}(W)}{L_{\Phi}},$
$\displaystyle\eta_{2}=\frac{\mu}{\lambda_{1}(P)L_{h}^{2}},$ (11)
$\eta_{3}=\frac{\mu\lambda_{n}(Q)}{\frac{L_{\Phi}^{2}}{4}+L_{h}^{2}(\|A^{\mathsf{T}}P\|^{2}+\lambda_{n}(Q)\lambda_{1}(P))+{L_{h}}L_{\Phi}\|A^{\mathsf{T}}P\|},$

with $\mu$ an arbitrary constant, $0<\mu\leq
1-\frac{{(1-\lambda_{n}(W))+\eta{L_{\Phi}}}}{2}$, and
$L_{h}=\|(I-A)^{\scalebox{0.75}[0.9]{$-$}1}BS\|$. Then, the sequences $x^{k},$
$u^{k}_{(i)}$ generated by (10) converges. $\square$

###### Proof:

We will prove this claim by using La Salle’s Invariance Principle [23, Thm
4.4]. For clarity of presentation, the proof is organized into four main
steps.

1) Change of variables and storage function. Let
$h(u)=(I-A)^{\scalebox{0.75}[0.9]{$-$}1}BSu{+}(I-A)^{\scalebox{0.75}[0.9]{$-$}1}Ew,$
and consider the new coordinate $\tilde{x}^{k}=x^{k}-h(u^{k}_{(1:N)}),$ which
shifts the equilibrium point of (10a) to the origin. Inspired by singular-
perturbation reasonings [23, Sec. 11], consider the storage function:

$\displaystyle
U(u_{(1:N)},\tilde{x}):=\frac{d}{\eta}V_{u}(u_{(1:N)})+(1-d)V_{x}(\tilde{x}),$
(12)

for each $\tilde{x}\in{}^{n},u_{(1:N)}\in{}^{n{m}},$ where $d\in(0,1)$ and

$\displaystyle V_{u}(u_{(1:N)})$
$\displaystyle=-\frac{1}{2}\sum_{i,j=1}^{N}w_{ij}u_{(i)}^{\mathsf{T}}{u_{(j)}}$
$\displaystyle\quad+\sum_{i=1}^{N}\left(\frac{1}{2}\|u_{(i)}\|^{2}+\eta\Phi_{i}(u_{(i)},Gu_{(i)}+Hw)\right),$
$\displaystyle V_{x}(\tilde{x})$
$\displaystyle=\tilde{x}^{\mathsf{T}}P\tilde{x}.$ (13)

Notice that $V_{u}$ is Lipschitz differentiable with constant
$L_{V_{u}}{\leq}(1-\lambda_{n}(W))+\eta{L_{\Phi}}$ and it is convex (since all
$\Phi_{i}$ are convex and
$\sum_{i=1}^{N}\|u_{(i)}\|^{2}-\sum_{i,j=1}^{N}w_{ij}u_{(i)}^{\mathsf{T}}u_{(j)}^{\mathsf{T}}$
is also convex due to $\lambda_{1}(W)=1$). Next, we introduce the quantity

$\displaystyle\tilde{V}_{u}(u_{(1:N)},\tilde{x})=-\frac{1}{2}\sum_{i,j=1}^{N}w_{ij}u_{(i)}^{\mathsf{T}}{u_{(j)}}$
(14)
$\displaystyle\quad+\sum_{i=1}^{N}\left(\frac{1}{2}\|u_{(i)}\|^{2}+\eta\Phi_{i}(u_{(i)},C\tilde{x}+GSu_{(i)}+Hw)\right),$

and

$\displaystyle
F_{c}(u_{(1:N)},\tilde{x}):=\begin{bmatrix}\nabla_{u_{1}}\tilde{V}_{u}(u_{(1:N)},\tilde{x})\\\
\vdots\\\ \nabla_{u_{n}}\tilde{V}_{u}(u_{(1:N)},\tilde{x})\end{bmatrix}.$

With this notation, (10b) and (14) can be re-expressed as:

$\displaystyle u^{k+1}_{(1:N)}$
$\displaystyle=u^{k}_{(1:N)}-F_{c}(u_{(1:N)}^{k},\tilde{x}^{k}),$
$\displaystyle V_{u}(u_{(1:N)})$ $\displaystyle=\tilde{V}_{u}(u_{(1:N)},0),$
(15)

by using $y^{k}=C(\tilde{x}^{k}+h(u_{(1:N)}^{k})+{DSu^{k}_{(1:N)}}.$

2) Bounding the variation of $V_{u}(\cdot).$ We have:

$\displaystyle V_{u}(u_{(1:N)}^{k+1})\leq V_{u}(u_{(1:N)}^{k})+\nabla
V_{u}(u_{(1:N)}^{k})^{\mathsf{T}}(u_{(1:N)}^{k+1}-u_{(1:N)}^{k})$
$\displaystyle\quad\quad\quad\quad\quad\quad+\frac{L_{V_{u}}}{2}\|u_{(1:N)}^{k+1}-u_{(1:N)}^{k}\|^{2}$
$\displaystyle=V_{u}(u_{(1:N)}^{k}){-}\nabla
V_{u}(u_{(1:N)}^{k})^{\mathsf{T}}F_{c}(u_{(1:N)}^{k},\tilde{x}^{k})$
$\displaystyle\quad\quad+\frac{L_{V_{u}}}{2}\|F_{c}(u_{(1:N)}^{k},\tilde{x}^{k})\|^{2}$
$\displaystyle\leq
V_{u}(u_{(1:N)}^{k})-\|F_{c}(u_{(1:N)}^{k},\tilde{x}^{k})\|^{2}$
$\displaystyle\quad\quad+\|F_{c}(u_{(1:N)}^{k},\tilde{x}^{k})\|\|\nabla
V_{u}(u_{(1:N)}^{k})^{\mathsf{T}}-F_{c}(u_{(1:N)}^{k},\tilde{x}^{k})\|$
$\displaystyle\quad\quad+\frac{L_{V_{u}}}{2}\|F_{c}(u_{(1:N)}^{k},\tilde{x}^{k})\|^{2}$
$\displaystyle\leq
V_{u}(u_{(1:N)}^{k})-\left(1-\frac{L_{V_{u}}}{2}\right)\|F_{c}(u_{(1:N)}^{k},\tilde{x}^{k})\|^{2}$
$\displaystyle\quad\quad+\|F_{c}(u_{(1:N)}^{k},\tilde{x}^{k})\|\|\nabla
V_{u}(u_{(1:N)}^{k})^{\mathsf{T}}-F_{c}(u_{(1:N)}^{k},x^{k})\|$
$\displaystyle\leq
V_{u}(u_{(1:N)}^{k})-{\mu}\|F_{c}(u_{(1:N)}^{k},x^{k})\|^{2}$
$\displaystyle\quad\quad+\eta
L_{\Phi}\|F_{c}(u_{(1:N)}^{k},x^{k})\|\|\tilde{x}^{k}\|$ (16)

where the first row follows by convexity and Lipschitz differentiability of
$V_{u},$ the second row follows by application of (IV-A), the third row
follows by adding and subtracting $F_{c}(u_{(1:N)}^{k},\tilde{x}^{k})$ to
$\nabla V_{u}(u_{(1:N)}^{k})$ and using the triangle inequality, and the last
row follows by Lipschitz differentiability of $\Phi$ and definition of $\mu.$

3) Bounding the variation of $V_{x}(\cdot).$ In the variables $\tilde{x}^{k},$
the plant dynamics (10a) read as:

$\displaystyle\tilde{x}^{k+1}=A\tilde{x}^{k}+h(u^{k}_{(1:N)})-h(u^{k+1}_{(1:N)}).$

We have:

$\displaystyle
V_{x}(\tilde{x}^{k+1})=(\tilde{x}^{k})^{\mathsf{T}}A^{\mathsf{T}}PA\tilde{x}^{k}$
$\displaystyle\quad\quad+2(\tilde{x}^{k})^{\mathsf{T}}A^{\mathsf{T}}P(h(u^{k}_{(1:N)})-h(u^{k+1}_{(1:N)}))$
$\displaystyle\quad\quad+(h(u^{k}_{(1:N)})-h(u^{k+1}_{(1:N)}))^{\mathsf{T}}P(h(u^{k}_{(1:N)})-h(u^{k+1}_{(1:N)}))$
$\displaystyle\leq
V_{x}(\tilde{x}^{k})-(\tilde{x}^{k})^{\mathsf{T}}Q\tilde{x}^{k}+\lambda_{1}(P)\|h(u^{k}_{(1:N)})-h(u^{k+1}_{(1:N)})\|^{2}$
$\displaystyle\quad\quad+2\|A^{\mathsf{T}}P\|\|\tilde{x}^{k}\|\|h(u^{k}_{(1:N)})-h(u^{k+1}_{(1:N)})\|$
$\displaystyle\leq
V_{x}(\tilde{x}^{k})-{\lambda_{n}(Q)}\|\tilde{x}^{k}\|^{2}+\lambda_{1}(P)L_{h}^{2}\|u^{k}_{(1:N)}-u^{k+1}_{(1:N)}\|^{2}$
$\displaystyle\quad\quad+2L_{h}\|A^{\mathsf{T}}P\|\|\tilde{x}^{k}\|\|u^{k}_{(1:N)}-u^{k+1}_{(1:N)}\|$
$\displaystyle=V_{x}(\tilde{x}^{k})-{\lambda_{n}(Q)}\|\tilde{x}^{k}\|^{2}+\lambda_{1}(P)L_{h}^{2}\|F_{c}(u_{(1:N)}^{k},x^{k})\|^{2}$
$\displaystyle\quad\quad+2L_{h}\|A^{\mathsf{T}}P\|\|\tilde{x}^{k}\|\|F_{c}(u_{(1:N)}^{k},x^{k})\|,$
(17)

where the second row follows from Assumption 1, the third row from Lipschitz
continuity of $h,$ and the last row by application of (IV-A).

4) Combining the bounds. By combining (IV-A) and (IV-A), we conclude
$U(u_{(1:N)}^{k+1},\tilde{x}^{k+1})-U(u_{(1:N)}^{k},\tilde{x}^{k})\leq-\xi(u_{(1:N)}^{k},\tilde{x}^{k})^{\mathsf{T}}\Lambda\xi(u_{(1:N)}^{k},\tilde{x}^{k}),$
where

$\xi(u_{(1:N)}^{k},\tilde{x}^{k})=\begin{bmatrix}\|F_{c}(u_{(1:N)}^{k},\tilde{x}^{k})\|\\\
\|\tilde{x}^{k}\|\end{bmatrix}$

and

$\displaystyle\Lambda=\begin{bmatrix}d\frac{\mu}{\eta}-(1-d)\lambda_{1}(P)L_{h}^{2}&\alpha\\\
\alpha&(1-d)\lambda_{n}(Q)\end{bmatrix},$

where $\alpha:=-\frac{1}{2}(dL_{\Phi}+2(1-d)L_{h}\|A^{\mathsf{T}}P\|).$ It is
guaranteed that $\Lambda\succ 0$ when
$\eta\in(0,\bar{\eta}),\bar{\eta}=\min\\{\bar{\eta}_{1},\bar{\eta}_{2}\\}$,
where

$\displaystyle\bar{\eta}_{2}$
$\displaystyle=\frac{d\mu}{(1-d)\lambda_{1}(P)L_{h}^{2}},$
$\displaystyle\bar{\eta}_{3}$
$\displaystyle=\frac{d(1-d)\mu\lambda_{n}(Q)}{\frac{d^{2}L_{\Phi}^{2}}{4}+(1-d)^{2}L_{h}^{2}\rho+d(1-d)L_{h}L_{\Phi}\|A^{\mathsf{T}}P\|},$

with $\rho:=\|A^{\mathsf{T}}P\|^{2}+\lambda_{n}(Q)\lambda_{1}(P).$ By making
the following choice for the free variable $d:$

$d=\frac{L_{h}^{2}(\|A^{\mathsf{T}}P\|^{2}+\lambda_{n}(Q)\lambda_{1}(P))-\frac{L_{\Phi}L_{h}}{2}\sqrt{\rho}}{L_{h}^{2}(\|A^{\mathsf{T}}P\|^{2}+\lambda_{n}(Q)\lambda_{1}(P))-\frac{L_{\Phi}^{2}}{4}},$

we obtain the largest value of $\bar{\eta}_{3}$ that guarantees decrease of
the storage function. For the sake of simplicity, we choose $d=0.5$, yielding
the choice $\eta_{2}$ and $\eta_{3}$ in (11). To conclude, since
$U(u_{(1:N)}^{k},\tilde{x}^{k})$ strongly decreases along the trajectories of
(10) and its minimum is the
point$(\|F_{c}(u_{(1:N)}^{k},\tilde{x}^{k})\|,\|\tilde{x}^{k}\|)=(0,0),$ by La
Salle’s invariance principle the claim follows. ∎

Theorem IV.1 shows that under a sufficiently small choice of the stepsize
$\eta,$ the sequences of states $x^{k},u^{k}_{(i)}$ of, respectively, the
system and controller states, converge asymptotically to a fixed point. Notice
that this convergence claim is not straightforward, since the proposed
controller (9) incorporates two simultaneous steps: a consensus one and a
gradient step. As such, this update may oscillate or fail to converge when
$\eta$ is chosen inadequately. The upper bounds $\eta_{1},\eta_{2},\eta_{3}$
depend on the various parameters of the system (II-A) and optimization problem
(6). Importantly, notice that the imposed bounds on $\mu$ guarantee that
$\bar{\eta}>0$ is well-defined.

Before proceeding, we present an instrumental result that will be used in the
remainder. Based on the definition of $u_{(1:N)}^{k}$ and
$\gamma(u^{k}_{(1:N)},y^{k})$, we rewrite (9) as:

$\displaystyle u_{(1:N)}^{k+1}=(W\otimes
I)u_{(1:N)}^{k}-\eta\,\gamma(u^{k}_{(1:N)},y^{k}),$ (18)

where $\otimes$ denotes the Kronecker product.

###### Proposition IV.2

(Bounded gradient) Let the assumptions of Theorem IV.1 hold. Moreover, assume
that the initial conditions of the controller satisfy $u^{0}_{(i)}=0,\ \forall
i\in\mathcal{V}.$ Then, for all $k\in\mathbb{Z}_{\geq 0},$ (18) satisfies:

$\displaystyle\|\gamma(u^{k}_{(1:N)},y^{k})\|\leq\sigma,$ (19)

where
$\sigma:=\sqrt{2L_{\Phi}\left(\sum_{i=1}^{N}\Phi_{i}(u^{0}_{(i)},y^{0})-{\Phi}^{\textup{opt}}\right)}$
Here, $\Phi^{\text{opt}}=\sum_{i=1}^{N}\Phi_{i}^{\text{opt}},$ with
$\Phi_{i}^{\text{opt}}=\Phi_{i}(u_{(i)}^{\text{opt}},y^{\text{opt}})$ and
$(u_{(i)}^{\text{opt}},y^{\text{opt}})=\arg\min_{u,y}\Phi_{i}(u,y)$.

###### Proof:

To prove (19), we rewrite

$\displaystyle\sum_{i=1}^{N}\Phi_{i}(u_{(i)}^{k},y^{k})$
$\displaystyle\leq\eta^{-1}\tilde{V}(u_{(1:N)}^{k},\tilde{x}^{k})$ (20)
$\displaystyle\leq\eta^{-1}\tilde{V}(u_{(1:N)}^{0},\tilde{x}^{0})$
$\displaystyle=\sum_{i=1}^{N}\Phi_{i}(u_{(i)}^{0},y^{0})$

where the first inequality follows from (14) and by using
$\sum_{i=1}^{N}\|u_{(i)}\|^{2}-\sum_{i,j=1}^{N}w_{ij}u_{(i)}^{\mathsf{T}}u_{(j)}^{\mathsf{T}}\geq
0$ (which holds since $\beta<1$), the second inequality holds from
$\tilde{V}_{u}(u_{(1:N)}^{k+1},\tilde{x}^{k+1})\leq\tilde{V}_{u}(u_{(1:N)}^{k},\tilde{x}^{k})$
(which follows by iterating the steps in (IV-A) applied to
$\tilde{V}_{u}(u_{(1:N)}^{k+1},\tilde{x}^{k+1})$ instead of
$V_{u}(u_{(1:N)}^{k+1})$), and the last identity follows from (IV-A) using
$u^{0}_{(1:N)}=0$.

Moreover, recall that for any differentiable convex function $g$ with
minimizer $u^{*},y^{*}$, and Lipschitz constant $L_{g}$, we have
$g(u_{a},y_{a})\geq g(u_{b},y_{b})+\nabla
g^{\mathsf{T}}(u_{b},y_{b})[(u_{a}-u_{b}),(y_{a}-y_{b})]^{\mathsf{T}}+\frac{1}{2L_{g}}\|\nabla
g(u_{a},y_{a})-\nabla g(u_{b},y_{b})\|^{2}$ and $\nabla g(u^{*},y^{*})=0$.
Then, $\|\nabla g(u,y)\|^{2}\leq 2L_{g}(g(u,y)-g^{*})$, where
$g^{*}:=g(u^{*},y^{*})$. Using this inequality and (20), we obtain

$\displaystyle\|\gamma(u^{k}_{(1:N)},y^{k})\|^{2}=\sum_{i=1}^{N}\|\Pi^{\mathsf{T}}\nabla\Phi_{i}(u_{(i)}^{k},y^{k})\|^{2}$
(21)
$\displaystyle\quad\quad\leq\sum_{i=1}^{N}2L_{\Phi}(\Phi_{i}(u_{(i)}^{k},y^{k})-\Phi_{i}^{o})$
$\displaystyle\quad\quad\leq
2L_{\Phi}\left(\sum_{i=1}^{N}\Phi_{i}(u^{0}_{(i)},y^{0})-{\Phi}^{\textup{opt}}\right),$

where $\Phi^{\text{opt}}=\sum_{i=1}^{N}\Phi_{i}^{\text{opt}},$ with
$\Phi_{i}^{\text{opt}}=\Phi_{i}(u_{(i)}^{\text{opt}},y^{\text{opt}})$ and
$(u_{(i)}^{\text{opt}},y^{\text{opt}})=\arg\min_{u,y}\Phi_{i}(u,y)$. Note that
$u_{(i)}^{\text{opt}},y^{\text{opt}}$ exist because of Assumption 2 and 3. ∎

Proposition IV.2 ensures that the sequence of gradients
$\gamma(u^{k}_{(1:N)},y^{k})$ is uniformly bounded. This result will be key in
the subsequent section when characterizing the control error. Interestingly,
unlike [16, 17, 18] that assume bounded gradient, in our analysis our choice
of stepsize ensures that gradient remains bounded.

We conclude by noting that when the initial conditions $u^{0}_{(1:N)}$ are
nonzero, a uniform bound of the form (19) can still be proven by adjusting the
step (21), but $\sigma$ needs to be modified to account for additional error
terms.

###  IV-B Control error bounds

While Theorem IV.1 certifies that the states sequences converge
asymptotically, it remains to quantify explicitly the controller performance.
This is the focus of this section.

In line with the existing literature [14], to establish a liner rate of
convergence, we will restrict our focus on cost functions that are restricted
strongly convex; recall that $f:\operatorname{dom}f\rightarrow\real$ is
restricted strongly convex [24] with modulus $\nu_{f}$ if

$\displaystyle(\nabla f(z)-\nabla
f\left(z^{*}\right))^{\mathsf{T}}(z-z^{*})\geq\nu_{f}\left\|z-z^{*}\right\|^{2},$
(22)

for all
$z\in\operatorname{dom}f,z^{*}=\operatorname{Proj}_{\mathcal{Z}^{*}}(z)$,
where $\operatorname{Proj}_{\mathcal{Z}^{*}}(z)$ is the projection of $z$ onto
the solution set $\mathcal{Z}^{*}$ such that $\nabla f\left(z^{*}\right)=0$.
The following result is instrumental.

###### Lemma IV.3

[24, Lemma 6] Suppose that $f$ is restricted strongly convex with modulus
$\nu_{f}$ and $\nabla f$ is Lipschitz continuous with constant $L_{f}$. Then,
we have

$\displaystyle(z-z^{*})^{\mathsf{T}}(\nabla f(z)-\nabla f(z^{*}))$ (23)
$\displaystyle\quad\quad\quad\quad\quad\geq c_{1}\|\nabla f(z)-\nabla
f(z^{*})\|^{2}+c_{2}\|z-z^{*}\|^{2},$

where $z^{*}$ is as in (22). Moreover, for any $\theta\in[0,1],$

$\displaystyle c_{1}=\frac{\theta}{L_{f}},$ $\displaystyle
c_{2}=(1-\theta)\nu_{f}.$ (24)

###### Remark 1

Notice that, if $f$ is strongly convex with modulus $\nu_{f},$ then it is also
restricted strong convexity with the same modulus [24]. In this case, (23)
holds with

$\displaystyle c_{1}=\frac{1}{\nu_{f}+L_{f}},$ $\displaystyle
c_{2}=\frac{\nu_{f}L_{f}}{\nu_{f}+L_{f}}.$ (25)

$\square$

The following is the second main result of this paper.

###### Theorem IV.4

(Control error bounds) Let the assumptions of Proposition (IV.2) hold. If
$(u,y)\mapsto\Phi(u,y)$ is restricted strongly convex with modulus
$\nu_{\Phi}$, then, for (10) it holds:

$\displaystyle\|u_{(i)}^{k}-u^{*k}\|\leq
c_{3}^{k}\|u_{(i)}^{0}-u^{*0}\|+\frac{c_{4}}{\sqrt{1-c_{3}^{2}}}+\frac{\eta\sigma}{1-\beta},$
(26)

where

$\displaystyle c_{3}^{2}=1-\eta c_{2}+\eta\delta-\eta^{2}\delta c_{2},$
$\displaystyle
c_{4}^{2}=\eta^{3}(\eta+\delta^{-1})\frac{{L_{\Phi}^{2}}\sigma^{2}}{(1-\beta)^{2}},$

$(u^{*k},x^{*k}):=\text{Proj}_{\mathcal{A}^{*}}(\bar{u}^{k},x^{k}),$ $\sigma$
is as in (19), $\delta>0$ is an arbitrary constant, and $c_{1}$ and $c_{2}$
are as in (24) with $\nu_{f}=\nu_{\Phi}/N.$ Moreover, if $\Phi(u,y)$ is
strongly convex, then $c_{1}$ and $c_{2}$ are as in (25). $\square$

###### Proof:

We begin by proving (26). It will be convenient to measure the control error
relative to the average controller state:
$\bar{u}^{k}:=\frac{1}{n}\sum_{i=1}^{n}u_{(i)}^{k}.$ We have:

$\displaystyle\|u_{(i)}^{k}-u^{*k}\|$
$\displaystyle\leq\|u_{(i)}^{k}-\bar{u}^{k}\|+\|\bar{u}^{k}-u^{*k}\|.$ (27)

For presentation purposes, the proof is organized into two main steps.

1) Bound for $\|u_{(i)}^{k}-\bar{u}^{k}\|$. By expanding (18) in time:

$\displaystyle u_{(1:N)}^{k}=-\eta\sum_{s=0}^{k-1}(W^{k-1-s}\otimes
I)\gamma(u^{s}_{(1:N)},y^{s}).$ (28)

Next, let
$\bar{u}_{(1:N)}^{k}=(\bar{u}^{k},\cdots,\bar{u}^{k})\in\mathbb{R}^{nm}$, and
notice that $\bar{u}_{(1:N)}^{k}=\frac{1}{n}((1_{n}1_{n}^{\mathsf{T}})\otimes
I)u_{(1:N)}^{k}.$ As a result,

$\displaystyle\|u_{(i)}^{k}$
$\displaystyle-\bar{u}^{k}\|\leq\|u_{(1:N)}^{k}-\bar{u}_{(1:N)}^{k}\|$
$\displaystyle=\|u_{(1:N)}^{k}-\frac{1}{n}((1_{n}1_{n}^{\mathsf{T}})\otimes
I)u_{(1:N)}^{k}\|$ $\displaystyle=\|-\eta\sum_{s=0}^{k-1}(W^{k-1-s}\otimes
I)\gamma(u^{s}_{(1:N)},y^{s})$
$\displaystyle+\eta\sum_{s=0}^{k-1}\frac{1}{n}((1_{n}1_{n}^{\mathsf{T}}W^{k-1-s})\otimes
I)\gamma(u^{s}_{(1:N)},y^{s})\|$
$\displaystyle=\|-\eta\sum_{s=0}^{k-1}(W^{k-1-s}\otimes
I)\gamma(u^{s}_{(1:N)},y^{s})$
$\displaystyle+\eta\sum_{s=0}^{k-1}\frac{1}{n}((1_{n}1_{n}^{\mathsf{T}})\otimes
I)\gamma(u^{s}_{(1:N)},y^{s})\|$
$\displaystyle=\eta\|\sum_{s=0}^{k-1}\left(\left(W^{k-1-s}-\frac{1}{n}1_{n}1_{n}^{\mathsf{T}}\right)\otimes
I\right)\gamma(u^{s}_{(1:N)},y^{s})\|$
$\displaystyle\leq\eta\sum_{s=0}^{k-1}\|W^{k-1-s}-\frac{1}{n}1_{n}1_{n}^{\mathsf{T}}\|\|\gamma(u^{s}_{(1:N)},y^{s})\|$
$\displaystyle=\eta\sum_{s=0}^{k-1}\beta^{k-1-s}\|\gamma(u^{s}_{(1:N)},y^{s})\|,$
(29)

where the fourth row holds because $W$ is doubly stochastic. From
$\|\gamma(u^{k}_{(1:N)},y^{k})\|\leq\sigma$ and $\beta<1$, it follows that

$\displaystyle\|u_{(i)}^{k}-\bar{u}^{k}\|$
$\displaystyle\leq\eta\sum_{s=0}^{k-1}\beta^{k-1-s}\|\gamma(u^{s}_{(1:N)},y^{s})\|\leq\sum_{s=0}^{k-1}\beta^{k-1-s}\sigma$
$\displaystyle\leq\frac{\eta\sigma}{1-\beta}.$ (30)

2) Bound for $\|\bar{u}^{k}-u^{*k}\|$. We will denote in compact form:

$\displaystyle\bar{e}^{k}:=\bar{u}^{k}-u^{*k}.$

To bound this term, let

$\displaystyle g(u_{(1:N)}^{k},y^{k})$
$\displaystyle=\frac{1}{n}\sum_{i=1}^{N}\Pi^{\mathsf{T}}\nabla\Phi_{i}(u_{(i)}^{k},y^{k}),$
$\displaystyle\bar{g}(u_{(1:N)}^{k},y^{k})$
$\displaystyle=\frac{1}{n}\sum_{i=1}^{N}\Pi^{\mathsf{T}}\nabla\Phi_{i}(\bar{u}^{k},y^{k}).$

We are interested in $g(u_{(1:N)}^{k},y^{k})$ because $-\eta
g(u_{(1:N)}^{k},y^{k})$ updates $\bar{u}^{k}$. To see this, by taking the
average of (9) over $i$ and noticing $W=[w_{ij}]$ is doubly stochastic, we
obtain

$\displaystyle\bar{u}^{k+1}$
$\displaystyle=\frac{1}{n}\sum_{i=1}^{N}u_{(i)}^{k+1}$
$\displaystyle=\frac{1}{n}\sum_{i,j=1}^{N}w_{ij}u_{(j)}^{{k}}-\frac{\eta}{n}\sum_{i=1}^{N}\Pi^{\mathsf{T}}\nabla\Phi_{i}(u_{(i)}^{k},y^{k})$
$\displaystyle=\bar{u}^{k}-\eta g(u_{(1:N)}^{k},y^{k}).$ (31)

Before proceeding notice that the following bound holds:

$\displaystyle\|\Pi^{\mathsf{T}}(\nabla\Phi_{i}(u_{(i)}^{k},y^{k})-\nabla\Phi_{i}(\bar{u}^{k},y^{k}))\|$
$\displaystyle\leq L_{\Phi}\|u_{(i)}^{k}-\bar{u}^{k}\|$
$\displaystyle\leq\frac{\eta\sigma L_{\Phi}}{1-\beta}$

by Assumptions 2, 3, and (7), and where the last inequality follows from
(IV-B). Moreover, we also have:

$\displaystyle\|g(u_{(1:N)}^{k},y^{k})-\bar{g}(u_{(1:N)}^{k},y^{k})\|$
$\displaystyle\quad\quad=\|\frac{1}{n}\sum_{i=1}^{N}\Pi^{\mathsf{T}}(\nabla\Phi_{i}(u_{(i)}^{k},y^{k})-\nabla\Phi_{i}(\bar{u}^{k},y^{k}))\|$
$\displaystyle\quad\quad\leq\frac{1}{n}L_{\Phi}\sum_{i=1}^{N}\|u_{(i)}^{k}-\bar{u}^{k}\|$
$\displaystyle\quad\quad\leq\frac{\eta\sigma L_{\Phi}}{1-\beta}.$ (32)

Recalling that
$(u^{*k+1},x^{*k+1})=\text{Proj}_{\mathcal{A^{*}}}(\bar{u}^{k+1},x^{k+1})$ and
$\bar{e}^{k+1}=\bar{u}^{k+1}-u^{*k+1}$, we have

$\displaystyle\|\bar{e}^{k+1}\|^{2}\leq\|\bar{u}^{k+1}-u^{*k}\|^{2}$
$\displaystyle=\|\bar{u}^{k}-u^{*k}-\eta g(u_{(1:N)}^{k},y^{k})\|^{2}$
$\displaystyle=\|\bar{e}^{k}-\eta\bar{g}(u_{(1:N)}^{k},y^{k})+\eta(\bar{g}(u_{(1:N)}^{k},y^{k})-g(u_{(1:N)}^{k},y^{k}))\|^{2}$
$\displaystyle=\|\bar{e}^{k}-\eta\bar{g}(u_{(1:N)}^{k},y^{k})\|^{2}+\eta^{2}\|\bar{g}(u_{(1:N)}^{k},y^{k})-g(u_{(1:N)}^{k},y^{k})\|^{2}$
$\displaystyle+2\eta(\bar{g}(u_{(1:N)}^{k},y^{k})-g(u_{(1:N)}^{k},y^{k}))^{\mathsf{T}}(\bar{e}^{k}-\eta\bar{g}(u_{(1:N)}^{k},y^{k}))$
$\displaystyle\leq(1+\eta\delta)\|\bar{e}^{k}-\eta\bar{g}(u_{(1:N)}^{k},y^{k})\|^{2}$
$\displaystyle\quad\quad+\eta(\eta+\delta^{-1})\|\bar{g}(u_{(1:N)}^{k},y^{k})-g(u_{(1:N)}^{k},y^{k})\|^{2}.$

The first inequality holds since $u^{*}_{k+1}$ is the projection of
$\bar{u}_{k+1}$ onto the optimality set, and thus for any other optimizer
$\hat{u}^{*}_{k+1}$ we have
$|\hat{u}^{*}_{k+1}-\bar{u}_{k+1}|\geq|u^{*}_{k+1}-\bar{u}_{k+1}|$. The last
inequality follows from $\pm
2a^{\mathsf{T}}b\leq\delta^{-1}\|a\|^{2}+\delta\|b\|^{2}$ for any $\sigma\geq
0$. Next, we shall bound
$\|\bar{e}^{k}-\eta\bar{g}(u_{(1:N)}^{k},y^{k})\|^{2}$. Applying Lemma (IV.3),
we have

$\displaystyle\|\bar{e}^{k}-\eta\bar{g}(u_{(1:N)}^{k},y^{k})\|^{2}=\|\bar{e}^{k}\|^{2}+\eta^{2}\|\bar{g}(u_{(1:N)}^{k},y^{k})\|^{2}$
$\displaystyle-2\eta\bar{e}^{k\mathsf{T}}\bar{g}(u_{(1:N)}^{k},y^{k})\leq\|\bar{e}^{k}\|^{2}+\eta^{2}\|\bar{g}(u_{(1:N)}^{k},y^{k})\|^{2}$
$\displaystyle-\eta c_{1}\|\bar{g}(u_{(1:N)}^{k},y^{k})\|^{2}-\eta
c_{2}\|\bar{e}^{k}\|^{2}$ $\displaystyle=(1-\eta
c_{2})\|\bar{e}^{k}\|^{2}+\eta(\eta-
c_{1})\|\bar{g}(u_{(1:N)}^{k},y^{k})\|^{2}.$

We shall pick $\eta\leq c_{1}$ so that $\eta(\eta-
c_{1})\|\bar{g}(u_{(1:N)}^{k},y^{k})\|^{2}\leq 0$. Then, from the last two
inequality arrays, we have

$\displaystyle\|\bar{e}^{k+1}\|^{2}$ $\displaystyle\leq(1+\eta\delta)(1-\eta
c_{2})\|\bar{e}^{k}\|^{2}$
$\displaystyle\quad\quad+\eta(\eta+\delta^{-1})\|\bar{g}(u_{(1:N)}^{k},y^{k})-g(u_{(1:N)}^{k},y^{k})\|^{2}$
$\displaystyle\leq(1-\eta c_{2}+\eta\delta-\eta^{2}\delta
c_{2})\|\bar{e}^{k}\|^{2}$
$\displaystyle+\eta^{3}(\eta+\delta^{-1})\frac{L_{\Phi}^{2}\sigma^{2}}{(1-\beta)^{2}}.$

Where the second inequality follows from (IV-B). Note that if $\Phi$ is
restricted strongly convex, then
$c_{1}c_{2}=\frac{\theta(1-\theta)\nu_{\Phi}}{L_{\Phi}}<1$ because
$\theta\in[0,1]$ and $\nu_{\Phi}<L_{\Phi}$; if $\Phi$ is strongly convex, then
$c_{1}c_{2}=\frac{\mu_{\Phi}L_{\Phi}}{(\mu_{\Phi}+L_{\Phi})^{2}}<1$.
Therefore, we have $c_{1}<1/c_{2}$. When $\eta<c_{1}$, $(1+\eta\delta)(1-\eta
c_{2})>0$.

Using

$\displaystyle\|\bar{e}^{k}\|^{2}\leq
c_{3}^{k}\|\bar{e}^{0}\|^{2}+\frac{1-c_{3}^{2k}}{1-c_{3}^{2}}c_{4}^{2}\leq
c_{3}^{k}\|\bar{e}^{0}\|^{2}\frac{c_{4}^{2}}{1-c_{3}^{2}},$

we get

$\displaystyle\|\bar{e}^{k}\|\leq
c_{3}^{k}\|\bar{e}^{0}\|+\frac{c_{4}}{\sqrt{1-c_{3}^{2}}}.$ (33)

The claim thus follows by combining (IV-B) and (33) ∎

Theorem IV.4 shows that the local agents states converge geometrically until
reaching a neighborhood of the optimal solution. The size of this neighborhood
depends on two quantities: $\frac{\eta\sigma}{1-\beta},$ which measures the
asymptotic error due to an inexact agreement (namely,
$\|u_{(i)}^{k}-\bar{u}^{k}\|$ where
$\bar{u}^{k}:=\frac{1}{n}\sum_{i=1}^{n}u_{(i)}^{k}$), and
$\frac{c_{4}}{\sqrt{1-c_{3}^{2}}},$ which quantifies the asymptotic error
between the average and the optimizer (namely, $\|\bar{u}^{k}-u^{*k}\|$). We
conclude with the following remark, which relates
$\frac{c_{4}}{\sqrt{1-c_{3}^{2}}}$ explicitly with $\eta$ and $\beta.$

###### Remark 2

(Refinement of bound (26)) In (26), by choosing $\delta=\frac{c_{2}}{2(1-\eta
c_{2})},$ we have $c_{3}=\sqrt{1-\frac{\eta c_{2}}{2}}\in(0,1)$ and

$\displaystyle\frac{c_{4}}{\sqrt{1-c_{3}^{2}}}$ $\displaystyle=\frac{\eta
L_{\Phi}\sigma}{1-\beta}\sqrt{\frac{\eta(\eta+\frac{2(1-\eta
c_{2})}{c_{2}})}{\frac{\eta c_{2}}{2}}}=\frac{\eta
L_{\Phi}\sigma}{1-\beta}\sqrt{\frac{4}{c_{2}^{2}}-\frac{2}{c_{2}}\eta}$
$\displaystyle\leq\frac{2\eta
L_{\Phi}\sigma}{c_{2}(1-\beta)}=\mathcal{O}\left(\frac{\eta}{1-\beta}\right).$

In this case, the local agent states converge geometrically to an
$\mathcal{O}\left(\frac{\eta}{1-\beta}+\frac{\eta\sigma}{1-\beta}\right)$
neighborhood of the solution set $\mathcal{A}^{*}$ $\square$

##  V Simulation results

In this section, we report our numerical results. Consider a system consisting
of three agents with the dynamics of $A$, $B$, and $C$ matrices randomly
chosen from the normal distribution. As for $A$, we consider generating a
network consisting of $N$ agents with $\frac{N(N-1)}{2}\kappa$ edges that are
uniformly randomly chosen, where $N=3$ and $\kappa=0.5$. We choose $n_{i}=1,\
\forall i,$ so that $n=N.$ The same setting is used for the control layer
network except that in this case, we choose the mixing matrix $W$so that it is
symmetric and doubly stochastic. Moreover, we make sure that $\mathcal{G}_{u}$
is connected. We apply (10) to the problem

$\displaystyle\underset{u\in{}^{m}}{\text{minimize}}~{}~{}\frac{1}{2}\left(\|u\|_{R}^{2}+\|Gu+Hw-y^{ref}\|_{Q}^{2}\right),$
(34)

where $Q=I_{m}$, $R=0.001I_{p}$, and $y^{ref}=0.5I_{p}$. Notice that the cost
function in (34) is strongly convex in this case.

(a) Comparison of the proposed decentralized algorithm for the problem (34)
with different fixed stepsizes.

(b) Outputs of the system with $\eta=2\times 10^{-3}$.

(c) Outputs of the system with $\eta=5\times 10^{-3}$.

Figure 2: Error $\bar{e}^{k}=\frac{1}{n}\sum_{i=1}^{n}u_{(i)}^{k}-u^{*k}$ and
outputs of the proposed decentralized algorithm with different stepsizes.

Fig. 2(a) illustrates the convergence of the error $\bar{e}^{k}$ related to
two different stepsizes. Note that $\bar{e}^{k}$ reduces linearly until
reaching an $\mathcal{O}(\eta)$-neighborhood, thus validating the conclusions
of Theorem IV.4. Moreover, it shows that a smaller $\eta$ causes the algorithm
to converge slower. Fig. 2(b) and (c), illustrate that the outputs of the
system reach the desired output.

##  VI Conclusions

We proposed a distributed controller to solve optimal steady-state regulation
problems while rejecting constant disturbances. The controller follows a
distributed architecture; as such, the approach scales well with the system
size and can be applied in cases where the individual cost functions need to
be maintained private. Under convexity and smoothness assumptions, we showed
that the controller state converges; under restricted strong convexity
assumptions, we showed that the controller converges geometrically to a
neighborhood of the optimal solution, in line with the existing literature on
distributed optimization [14]. Our work opens the opportunity for several
future works, including scenarios where the output feedback signals are also
local, an investigation of algorithms that can ensure exact convergence, the
study of constrained optimization objectives, and a generalization to
nonlinear systems.

## References

  * [1] A. Jokic, M. Lazar, and P. van den Bosch, “On constrained steady-state regulation: Dynamic KKT controllers,” vol. 54, no. 9, pp. 2250–2254, 2009.
  * [2] E. Dall’Anese, A. Simonetto, S. Becker, and L. Madden, “Optimization and learning with information streams: Time-varying algorithms and applications,” _IEEE Signal Processing Magazine_ , vol. 37, no. 3, pp. 71–83, 2020.
  * [3] M. Colombino, E. Dall’Anese, and A. Bernstein, “Online optimization as a feedback controller: Stability and tracking,” vol. 7, no. 1, pp. 422–432, 2020.
  * [4] A. Hauswirth, S. Bolognani, G. Hug, and F. Dörfler, “Timescale separation in autonomous optimization,” vol. 66, no. 2, pp. 611–624, 2021.
  * [5] S. Menta, A. Hauswirth, S. Bolognani, G. Hug, and F. Dörfler, “Stability of dynamic feedback optimization with applications to power systems,” in _Annual Conf. on Communication, Control, and Computing_ , Oct. 2018, pp. 136–143.
  * [6] L. S. P. Lawrence, J. W. Simpson-Porco, and E. Mallada, “Linear-convex optimal steady-state control,” 2018, to appear.
  * [7] G. Bianchin, J. Cortés, J. I. Poveda, and E. Dall’Anese, “Time-varying optimization of LTI systems via projected primal-dual gradient flows,” vol. 9, no. 1, pp. 474–486, Mar. 2022.
  * [8] G. Bianchin, J. I. Poveda, and E. Dall’Anese, “Online optimization of switched LTI systems using continuous-time and hybrid accelerated gradient flows,” vol. 146, p. 110579, Aug. 2022.
  * [9] F. Brunner, H.-B. Dürr, and C. Ebenbauer, “Feedback design for multi-agent systems: A saddle point approach,” 2012, pp. 3783–3789.
  * [10] Z. He, S. Bolognani, J. He, F. Dörfler, and X. Guan, “Model-free nonlinear feedback optimization,” 2023, in press.
  * [11] X. Chen, J. I. Poveda, and N. Li, “Model-free feedback constrained optimization via projected primal-dual zeroth-order dynamics,” _arXiv preprint_ , 2022, arXiv:2206.11123.
  * [12] G. Bianchin, M. Vaquero, J. Cortés, and E. Dall’Anese, “Online stochastic optimization for unknown linear systems: Data-driven synthesis and controller analysis,” 2024, early Access.
  * [13] A. Nedić and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” 2009, to appear.
  * [14] K. Yuan, Q. Ling, and W. Yin, “On the convergence of decentralized gradient descent,” vol. 26, no. 3, pp. 1835–1854, 2016.
  * [15] A. Nedić, A. Ozdaglar, and P. A. Parrilo, “Constrained consensus and optimization in multi-agent networks,” vol. 55, no. 4, pp. 922–938, 2010.
  * [16] J. C. Duchi, A. Agarwal, and M. J. Wainwright, “Dual averaging for distributed optimization: Convergence analysis and network scaling,” vol. 57, no. 3, pp. 592–606, 2011.
  * [17] D. Jakovetić, J. Xavier, and J. M. Moura, “Fast distributed gradient methods,” vol. 59, no. 5, pp. 1131–1146, 2014.
  * [18] A. Nedić and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” vol. 54, no. 1, pp. 48–61, 2009.
  * [19] C. Xi, V. S. Mai, R. Xin, E. H. Abed, and U. A. Khan, “Linear convergence in optimization over directed graphs with row-stochastic matrices,” vol. 63, no. 10, pp. 3558–3565, 2018.
  * [20] T. Liu, Z. Qin, Y. Hong, and Z.-P. Jiang, “Distributed optimization of nonlinear multiagent systems: A small-gain approach,” vol. 67, no. 2, pp. 676–691, 2021.
  * [21] W. Wang, Z. He, G. Belgioioso, S. Bolognani, and F. Dörfler, “Decentralized feedback optimization via sensitivity decoupling: Stability and sub-optimality,” _arXiv preprint_ , 2023, arXiv:2311.09408.
  * [22] C.-Y. Chang, M. Colombino, J. Corté, and E. Dall’Anese, “Saddle-flow dynamics for distributed feedback-based optimization,” vol. 3, no. 4, pp. 948–953, 2019.
  * [23] H. K. Khalil, _Nonlinear Systems_ , 2nd ed., 1995.
  * [24] M.-J. Lai and W. Yin, “Augmented $\ell_{1}$ and Nuclear-norm models with a globally linearly convergent algorithm,” _SIAM Journal on Imaging Sciences_ , vol. 6, no. 2, pp. 1059–1091, 2013.