$\displaystyle=\left[\pi_{(n-2,0)}(1-p)+\frac{\pi_{(1,0)}(1-q_{1})}{q_{1}}\left(\prod\nolimits_{l=1}^{n-2}[1-pg(l)]\right)q_{n-2}\right](1-p)+\frac{\pi_{(1,0)}(1-q_{1})}{q_{1}}\left(\prod\nolimits_{l=1}^{n-1}[1-pg(l)]\right)q_{n-1}$
$\displaystyle=\pi_{(n-2,0)}(1-p)^{2}+\frac{\pi_{(1,0)}(1-q_{1})}{q_{1}}\bigg{[}(1-p)\left(\prod\nolimits_{l=1}^{n-2}[1-pg(l)]\right)q_{n-2}+\left(\prod\nolimits_{l=1}^{n-1}[1-pg(l)]\right)q_{n-1}\bigg{]}$
$\displaystyle\qquad\qquad\qquad\qquad\quad\qquad\qquad\qquad\qquad\qquad\quad\qquad\vdots$
$\displaystyle=\pi_{(1,0)}(1-p)^{n-1}+\frac{\pi_{(1,0)}(1-q_{1})}{q_{1}}\bigg{[}\sum\nolimits_{j=0}^{n-2}(1-p)^{j}\left(\prod\nolimits_{l=1}^{n-1-j}[1-pg(l)]\right)q_{n-1-j}\bigg{]}$
(137)

This gives the results (6).

Now, we determine the first probability $\pi_{(1,0)}$. Since all the
stationary probabilities add up to 1, we have the relation

$\displaystyle 1$
$\displaystyle=\sum\nolimits_{m=0}^{\infty}\sum\nolimits_{n=m+1}^{\infty}\pi_{(n,m)}$
$\displaystyle=\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}+\sum\nolimits_{m=1}^{\infty}\sum\nolimits_{n=m+1}^{\infty}\pi_{(n,m)}$
$\displaystyle=\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}+\sum\nolimits_{m=1}^{\infty}\pi_{(m+1,m)}+\sum\nolimits_{m=1}^{\infty}\sum\nolimits_{n=m+2}^{\infty}\pi_{(n,m)}$
$\displaystyle=\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}+\sum\nolimits_{m=1}^{\infty}\pi_{(1,0)}p(1-q_{1})\left(\prod\nolimits_{l=1}^{m-1}[1-pg(l)]\right)\Pr\\{B>m-1\\}$
$\displaystyle\quad+\sum\nolimits_{m=1}^{\infty}\sum\nolimits_{n=m+2}^{\infty}\bigg{[}\pi_{(n-m,0)}p(1-q_{1})+\left(\sum\nolimits_{j=1}^{n-m-1}\pi_{(n-m,j)}pg(j)\right)(1-q_{1})\bigg{]}\left(\prod\nolimits_{l=1}^{m-1}[1-pg(l)]\right)\Pr\\{B>m-1\\}$
$\displaystyle=\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}+p(1-q_{1})\pi_{(1,0)}\left[\sum\nolimits_{m=1}^{\infty}\left(\prod\nolimits_{l=1}^{m-1}[1-pg(l)]\right)\Pr\\{B>m-1\\}\right]$
$\displaystyle\quad+p(1-q_{1})\left(\sum\nolimits_{n=m+2}^{\infty}\pi_{(n-m,0)}\right)\left[\sum\nolimits_{m=1}^{\infty}\left(\prod\nolimits_{l=1}^{m-1}[1-pg(l)]\right)\Pr\\{B>m-1\\}\right]$
$\displaystyle\quad+p(1-q_{1})\left(\sum\nolimits_{n=m+2}^{\infty}\sum\nolimits_{j=1}^{n-m-1}\pi_{(n-m,j)}g(j)\right)\left[\sum\nolimits_{m=1}^{\infty}\left(\prod\nolimits_{l=1}^{m-1}[1-pg(l)]\right)\Pr\\{B>m-1\\}\right]$
$\displaystyle=\left(\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}\right)+p(1-q_{1})\bigg{[}\left(\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}\right)+\left(\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{m=1}^{n-1}\pi_{(n,m)}g(m)\right)\bigg{]}$
$\displaystyle\quad\times\left[\sum\nolimits_{m=1}^{\infty}\left(\prod\nolimits_{l=1}^{m-1}[1-pg(l)]\right)\Pr\\{B>m-1\\}\right]$
$\displaystyle=\left(\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}\right)+\frac{\pi_{(1,0)}(1-q_{1})}{q_{1}}\left[\sum\nolimits_{m=1}^{\infty}\left(\prod\nolimits_{l=1}^{m-1}[1-pg(l)]\right)\left(\sum\nolimits_{l=m}^{\infty}q_{l}\right)\right]$
(138)

Using the equation (137), we calculate the first sum in (138).

$\displaystyle\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}$
$\displaystyle=\pi_{(1,0)}\sum\nolimits_{n=1}^{\infty}\bigg{\\{}(1-p)^{n-1}+\frac{1-q_{1}}{q_{1}}\bigg{[}\sum\nolimits_{j=0}^{n-2}(1-p)^{j}\left(\prod\nolimits_{l=1}^{n-1-j}[1-pg(l)]\right)q_{n-1-j}\bigg{]}\bigg{\\}}$
$\displaystyle=\pi_{(1,0)}\bigg{\\{}\frac{1}{p}+\frac{1-q_{1}}{q_{1}}\bigg{[}\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{j=0}^{n-2}(1-p)^{j}\left(\prod\nolimits_{l=1}^{n-1-j}[1-pg(l)]\right)q_{n-1-j}\bigg{]}\bigg{\\}}$
(139)

Substituting (139) into (138), finally we obtain that

$\displaystyle\frac{1}{\pi_{(1,0)}}$
$\displaystyle=\frac{1}{p}+\frac{1-q_{1}}{q_{1}}\bigg{[}\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{j=0}^{n-2}(1-p)^{j}\left(\prod\nolimits_{l=1}^{n-1-j}[1-pg(l)]\right)q_{n-1-j}\bigg{]}$
$\displaystyle\qquad\quad+\frac{1-q_{1}}{q_{1}}\left[\sum\nolimits_{m=1}^{\infty}\left(\prod\nolimits_{l=1}^{m-1}[1-pg(l)]\right)\left(\sum\nolimits_{l=m}^{\infty}q_{l}\right)\right]$
(140)

Thus, we prove the equation (7) in Theorem 1.

For $n>m\geq 2$, equation (131) shows that probabilities $\pi_{(n,m)}$ can be
obtained by

$\pi_{(n,m)}=\pi_{(n-m+1,1)}\left(\prod\nolimits_{l=1}^{m-1}[1-pg(l)]\right)\Pr\\{B>m-1\\}$

Therefore, we only need to find those probabilities $\pi_{(n,1)}$, $n\geq 3$.
The second line in equations (130) shows that

$\displaystyle\pi_{(n,1)}=\pi_{(n-1,0)}p(1-q_{1})+\left(\sum\nolimits_{j=1}^{n-2}\pi_{(n-1,j)}pg(j)\right)(1-q_{1})$
(141)

In the following, we rewrite (141) as a difference equation of probabilities
$\pi_{(n,1)}$, $n\geq 3$. The sum in above equation is dealt with as

$\displaystyle\sum\nolimits_{j=1}^{n-2}\pi_{(n-1,j)}pg(j)$
$\displaystyle=\pi_{(n-1,1)}pg(1)+\sum\nolimits_{j=2}^{n-2}\pi_{(n-1,j)}pg(j)$
$\displaystyle=\pi_{(n-1,1)}pg(1)+\sum\nolimits_{j=2}^{n-2}\pi_{(n-j,1)}\left(\prod\nolimits_{l=1}^{j-1}[1-pg(l)]\right)\Pr\\{B>j-1\\}pg(j)$
$\displaystyle=\sum\nolimits_{j=1}^{n-2}\pi_{(n-j,1)}\left(\prod\nolimits_{l=1}^{j-1}[1-pg(l)]\right)\Pr\\{B>j-1\\}pg(j)$

if we denote $\prod\nolimits_{l=1}^{0}[1-pg(l)]$ equals 1.

Define
$\alpha_{j}=\left(\prod\nolimits_{l=1}^{j-1}[1-pg(l)]\right)\Pr\\{B>j-1\\}pg(j)$,
the equation (141) is transformed to

$\pi_{(n,1)}=\pi_{(n-1,0)}p(1-q_{1})+(1-q_{1})\sum\nolimits_{j=1}^{n-2}\alpha_{j}\pi_{(n-j,1)}$
(142)

which forms a difference equation of probabilities $\pi_{(n,1)}$ for $n\geq
3$.

When the distribution of the packet service time $B$ and the preemption
probabilities $g(m)$, $m\geq 1$ are given, the probabilities $\pi_{(n,1)}$ can
be derived by solving the difference equation (142).

So far, we have obtained all the results in Theorem 2, thus the proof is
complete.

##  Appendix B Proof of Theorem 3

For the status updating system with Ber/Geo/1/1 queue, let the preemption
function is defined as (12), we solve the following stationary equations of
the process $AoI_{Ber/Geo/1/1/\tilde{g}(m)}$ and determine all the
probabilities $\pi_{(n,m)}$, $n>m\geq 0$.

$\begin{cases}\pi_{(n,m)}=\pi_{(n-1,m-1)}(1-p)\frac{N_{p}+m}{N_{p}+m-1}(1-\gamma)&(n>m\geq
2)\\\
\pi_{(n,1)}=\pi_{(n-1,0)}p(1-\gamma)+\left(\sum\nolimits_{j=1}^{n-2}\pi_{(n-1,j)}\left[1-(1-p)\frac{N_{p}+j+1}{N_{p}+j}\right]\right)(1-\gamma)&(n\geq
3)\\\ \pi_{(2,1)}=\pi_{(1,0)}p(1-\gamma)\\\
\pi_{(n,0)}=\pi_{(n-1,0)}(1-p)+\left(\sum\nolimits_{k=n}^{\infty}\pi_{(k,n-1)}\right)(1-p)\frac{N_{p}+n}{N_{p}+n-1}\gamma&(n\geq
2)\\\
\pi_{(1,0)}=\left(\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}\right)p\gamma+\left(\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{m=1}^{n-1}\pi_{(n,m)}\left[1-(1-p)\frac{N_{p}+m+1}{N_{p}+m}\right]\right)\gamma\end{cases}$
(143)

The theorem is proved directly using the general formulas (6)-(9) in Theorem
2.

First of all, for $n\geq 2$, equation (136) shows that

$\displaystyle\pi_{(n,0)}$
$\displaystyle=\pi_{(n-1,0)}(1-p)+\frac{\pi_{(1,0)}(1-q_{1})}{q_{1}}\left(\prod\nolimits_{l=1}^{n-1}[1-pg(l)]\right)q_{n-1}$
$\displaystyle=\pi_{(n-1,0)}(1-p)+\frac{\pi_{(1,0)}(1-\gamma)}{\gamma}\left(\prod\nolimits_{l=1}^{n-1}(1-p)\frac{N_{p}+l+1}{N_{p}+l}\right)(1-\gamma)^{n-2}\gamma$
$\displaystyle=\pi_{(n-1,0)}(1-p)+\frac{\pi_{(1,0)}}{N_{p}+1}\left(N_{p}+n\right)[(1-p)(1-\gamma)]^{n-1}$
(144)

Applying (144) iteratively, we can obtain that

$\displaystyle\pi_{(n,0)}$
$\displaystyle=\pi_{(1,0)}(1-p)^{n-1}+\frac{\pi_{(1,0)}(1-p)^{n-1}}{N_{p}+1}\left(\sum\nolimits_{j=0}^{n-2}(N_{p}+n-j)(1-\gamma)^{n-1-j}\right)$
$\displaystyle=\pi_{(1,0)}(1-p)^{n-1}+\frac{\pi_{(1,0)}(1-p)^{n-1}}{N_{p}+1}\left(\frac{N_{p}\gamma-(N_{p}+1)\gamma^{2}+1}{\gamma^{2}}-\frac{[1+(N_{p}+n)\gamma](1-\gamma)^{n}}{\gamma^{2}}\right)$
$\displaystyle=\frac{\pi_{(1,0)}}{(N_{p}+1)\gamma^{2}}\bigg{\\{}\left(N_{p}\gamma+1\right)(1-p)^{n-1}-\left[1+(N_{p}+n)\gamma\right](1-\gamma)[(1-p)(1-\gamma)]^{n-1}\bigg{\\}}$
(145)

Let $n=1$, (145) reduces to the equation $\pi_{(1,0)}=\pi_{(1,0)}$, thus we
verify that expression (145) is valid for all $n\geq 1$.

To determine the first probability $\pi_{(1,0)}$, we use equation (138), which
says that

$\displaystyle 1$
$\displaystyle=\left(\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}\right)+\frac{\pi_{(1,0)}(1-\gamma)}{\gamma}\left[\sum\nolimits_{m=1}^{\infty}\left(\prod\nolimits_{l=1}^{m-1}[1-p\tilde{g}(l)]\right)\left(\sum\nolimits_{l=m}^{\infty}q_{l}\right)\right]$
$\displaystyle=\left(\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}\right)+\frac{\pi_{(1,0)}(1-\gamma)}{\gamma}\sum\nolimits_{m=1}^{\infty}\left(\prod\nolimits_{l=1}^{m-1}(1-p)\frac{N_{p}+l+1}{N_{p}+l}\right)(1-\gamma)^{m-1}$
$\displaystyle=\left(\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}\right)+\frac{\pi_{(1,0)}(1-\gamma)}{(N_{p}+1)\gamma}\sum\nolimits_{m=1}^{\infty}\left(N_{p}+m\right)[(1-p)(1-\gamma)]^{m-1}$
(146)

Using equation (145), the sum in (146) is equal to

$\displaystyle\sum\nolimits_{n=1}^{\infty}\pi_{(n,0)}$
$\displaystyle=\frac{\pi_{(1,0)}}{(N_{p}+1)\gamma^{2}}\bigg{\\{}\frac{N_{p}\gamma+1}{p}-\frac{1-\gamma}{p+\gamma-p\gamma}-\gamma(1-\gamma)\sum\nolimits_{n=1}^{\infty}(N_{p}+n)[(1-p)(1-\gamma)]^{n-1}\bigg{\\}}$
$\displaystyle=\frac{\pi_{(1,0)}[N_{p}(p+\gamma-p\gamma)+1]}{(N_{p}+1)p\gamma(p+\gamma-p\gamma)}-\frac{\pi_{(1,0)}(1-\gamma)}{(N_{p}+1)\gamma}\sum\nolimits_{n=1}^{\infty}(N_{p}+n)[(1-p)(1-\gamma)]^{n-1}$
(147)

Combining (146) and (147), it was observed that two sums cancel out and we
have

$1=\frac{\pi_{(1,0)}[N_{p}(p+\gamma-p\gamma)+1]}{(N_{p}+1)p\gamma(p+\gamma-p\gamma)}$

from which $\pi_{(1,0)}$ is figured out as

$\pi_{(1,0)}=\frac{(N_{p}+1)p\gamma(p+\gamma-p\gamma)}{N_{p}(p+\gamma-p\gamma)+1}$
(148)

Substituting (148) into equation (145), the probabilities $\pi_{(n,0)}$,
$n\geq 1$ are determined finally. It shows that

$\displaystyle\pi_{(n,0)}$
$\displaystyle=\frac{p(p+\gamma-p\gamma)}{N_{p}\gamma(p+\gamma-p\gamma)+\gamma}\bigg{\\{}(N_{p}\gamma+1)(1-p)^{n-1}-[1+(N_{p}+n)\gamma](1-\gamma)[(1-p)(1-\gamma)]^{n-1}\bigg{\\}}$
$\displaystyle=\xi_{1}(1-p)^{n-1}-\xi_{2}[1+(N_{p}+n)\gamma][(1-p)(1-\gamma)]^{n-1}$
(149)

where the coefficients $\xi_{1}$ and $\xi_{2}$ are defined to be

$\xi_{1}=\frac{(N_{p}\gamma+1)p(p+\gamma-p\gamma)}{N_{p}\gamma(p+\gamma-p\gamma)+\gamma},\qquad\xi_{2}=\frac{p(1-\gamma)(p+\gamma-p\gamma)}{N_{p}\gamma(p+\gamma-p\gamma)+\gamma}$

Then, we consider the other probabilities $\pi_{(n,m)}$, $n>m\geq 1$. From the
equation (131), $\pi_{(n,m)}$ can be written as

$\displaystyle\pi_{(n,m)}$
$\displaystyle=\pi_{(n-m+1,1)}\left(\prod\nolimits_{l=1}^{m-1}[1-p\tilde{g}(l)]\right)\Pr\\{B>m-1\\}$
$\displaystyle=\pi_{(n-m+1,1)}\left(\prod\nolimits_{l=1}^{m-1}(1-p)\frac{N_{p}+l+1}{N_{p}+l}\right)\left(\sum\nolimits_{l=m}^{\infty}q_{l}\right)$
$\displaystyle=\pi_{(n-m+1,1)}[(1-p)(1-\gamma)]^{m-1}\frac{N_{p}+m}{N_{p}+1}$
(150)

Thus, in order to find the probabilities $\pi_{(n,m)}$, it suffices to
determine all the probabilities $\pi_{(n,1)}$, $n\geq 3$.

For the considered case, in the following we prove that the general equation
(9) can be reduced to an order three difference equation with constant
coefficients, such that the solution of the obtained difference equation can
be solved.

For $n\geq 3$, the second line of (143) shows that

$\pi_{(n,1)}=\pi_{(n-1,0)}p(1-\gamma)+\bigg{(}\sum\nolimits_{j=1}^{n-2}\pi_{(n-1,j)}\left[1-(1-p)\frac{N_{p}+j+1}{N_{p}+j}\right]\bigg{)}(1-\gamma)$
(151)

Using (150), the sum in equation (151) can be written as

$\displaystyle\sum\nolimits_{j=1}^{n-2}\pi_{(n-1,j)}\left[1-(1-p)\frac{N_{p}+j+1}{N_{p}+j}\right]$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{j=1}^{n-2}\left\\{\pi_{(n-j,1)}[(1-p)(1-\gamma)]^{j-1}\frac{N_{p}+j}{N_{p}+1}\right\\}\left[1-(1-p)\frac{N_{p}+j+1}{N_{p}+j}\right]$
$\displaystyle={}$
$\displaystyle\frac{1}{N_{p}+1}\sum\nolimits_{j=1}^{n-2}\pi_{(n-j,1)}[(1-p)(1-\gamma)]^{j-1}[(N_{p}+j)-(1-p)(N_{p}+j+1)]$
$\displaystyle={}$
$\displaystyle\frac{1}{N_{p}+1}\sum\nolimits_{j=1}^{n-2}\pi_{(n-j,1)}[p(N_{p}+j)-(1-p)][(1-p)(1-\gamma)]^{j-1}$
(152)

Therefore, we obtain that

$\pi_{(n,1)}=\pi_{(n-1,0)}p(1-\gamma)+\frac{1-\gamma}{N_{p}+1}\bigg{(}\sum\nolimits_{j=1}^{n-2}\pi_{(n-j,1)}[p(N_{p}+j)-(1-p)][(1-p)(1-\gamma)]^{j-1}\bigg{)}$
(153)

Do once iteration of equation (153) and multiplying $(1-p)(1-\gamma)$ in both
sides gives

$\displaystyle(1-p)(1-\gamma)\pi_{(n-1,1)}$ $\displaystyle={}$
$\displaystyle(1-p)(1-\gamma)\pi_{(n-2,0)}p(1-\gamma)+\frac{1-\gamma}{N_{p}+1}\bigg{(}\sum\nolimits_{j=1}^{n-3}\pi_{(n-1-j,1)}[p(N_{p}+j)-(1-p)][(1-p)(1-\gamma)]^{j}\bigg{)}$
(154)

Let $j=\tilde{j}-1$, the sum in equation (154) can be rewritten as

$\sum\nolimits_{\tilde{j}=2}^{n-2}\pi_{(n-\tilde{j},1)}[p(N_{p}+\tilde{j}-1)-(1-p)][(1-p)(1-\gamma)]^{\tilde{j}-1}\bigg{)}$

Subtracting (154) from equation (153) yields

$\displaystyle\pi_{(n,1)}-(1-p)(1-\gamma)\pi_{(n-1,1)}$ $\displaystyle={}$
$\displaystyle
h^{(1)}(n)+\frac{1-\gamma}{N_{p}+1}\bigg{(}\pi_{(n-1,1)}[p(N_{p}+1)-(1-p)]+\sum\nolimits_{j=2}^{n-2}\pi_{(n-j,1)}p[(1-p)(1-\gamma)]^{j-1}\bigg{)}$

Rearranging the terms, we obtain that

$\pi_{(n,1)}=h^{(1)}(n)+\frac{N_{p}(1-\gamma)}{N_{p}+1}\pi_{(n-1,1)}+\frac{p(1-\gamma)}{N_{p}+1}\sum\nolimits_{j=1}^{n-2}\pi_{(n-j,1)}[(1-p)(1-\gamma)]^{j-1}$
(155)

where $h^{(1)}(n)$ is defined as

$\displaystyle h^{(1)}(n)$
$\displaystyle=p(1-\gamma)\left[\pi_{(n-1,0)}-(1-p)(1-\gamma)\pi_{(n-2,0)}\right]$
$\displaystyle=p(1-\gamma)\bigg{\\{}\xi_{1}(1-p)^{n-2}-\xi_{2}[1+(N_{p}+n-1)\gamma][(1-p)(1-\gamma)]^{n-2}$
$\displaystyle\qquad-\xi_{1}(1-p)^{n-2}(1-\gamma)+\xi_{2}[1+(N_{p}+n-2)\gamma][(1-p)(1-\gamma)]^{n-2}\bigg{\\}}$
$\displaystyle=p\gamma(1-\gamma)\bigg{\\{}\xi_{1}(1-p)^{n-2}-\xi_{2}[(1-p)(1-\gamma)]^{n-2}\bigg{\\}}$

Repeat the same procedures to equation (155), we show that

$\displaystyle\pi_{(n,1)}-(1-p)(1-\gamma)\pi_{(n-1,1)}$ $\displaystyle={}$
$\displaystyle
h^{(2)}(n)+\frac{N_{p}(1-\gamma)}{N_{p}+1}\pi_{(n-1,1)}-\frac{N_{p}(1-p)(1-\gamma)^{2}}{N_{p}+1}\pi_{(n-2,1)}+\frac{p(1-\gamma)}{N_{p}+1}\pi_{(n-1,1)}$

which is equivalent to

$\pi_{(n,1)}=h^{(2)}(n)+\pi_{(n-1,1)}\left(1-\gamma+\frac{N_{p}(1-p)(1-\gamma)}{N_{p}+1}\right)-\pi_{(n-2,1)}\frac{N_{p}(1-p)(1-\gamma)^{2}}{N_{p}+1}$
(156)

The term $h^{(2)}(n)$ is equal to

$h^{(2)}(n)=h^{(1)}(n)-(1-p)(1-\gamma)h^{(1)}(n-1)=p\gamma^{2}(1-\gamma)\xi_{1}(1-p)^{n-2}$
(157)

Consider the iteration of (156) again, at this time, we compute

$\displaystyle\pi_{(n,1)}-(1-p)\pi_{(n-1,1)}$ $\displaystyle={}$
$\displaystyle\pi_{(n-1,1)}\left(1-\gamma+\frac{N_{p}(1-p)(1-\gamma)}{N_{p}+1}\right)-\pi_{(n-2,1)}\left((1-p)(1-\gamma)+\frac{N_{p}(1-p)^{2}(1-\gamma)}{N_{p}+1}\right)$
$\displaystyle\quad-\pi_{(n-2,1)}\frac{N_{p}(1-p)(1-\gamma)^{2}}{N_{p}+1}+\pi_{(n-3,1)}\frac{N_{p}[(1-p)(1-\gamma)]^{2}}{N_{p}+1}$

which can be transformed to

$\displaystyle\pi_{(n,1)}-\pi_{(n-1,1)}\left[(1-p)+(1-\gamma)+\frac{N_{p}(1-p)(1-\gamma)}{N_{p}+1}\right]+\pi_{(n-2,1)}\bigg{[}(1-p)(1-\gamma)$
$\displaystyle\qquad+\frac{N_{p}(1-p)^{2}(1-\gamma)}{N_{p}+1}+\frac{N_{p}(1-p)(1-\gamma)^{2}}{N_{p}+1}\bigg{]}-\pi_{(n-3,1)}\frac{N_{p}[(1-p)(1-\gamma)]^{2}}{N_{p}+1}=0$
(158)

Eventually, we obtain the constant coefficient three-order difference equation
(158), whose characterization equation is

$\displaystyle
r^{3}-\left[(1-p)+(1-\gamma)+\frac{N_{p}(1-p)(1-\gamma)}{N_{p}+1}\right]r^{2}$
$\displaystyle\qquad+\bigg{[}(1-p)(1-\gamma)+\frac{N_{p}(1-p)^{2}(1-\gamma)}{N_{p}+1}+\frac{N_{p}(1-p)(1-\gamma)^{2}}{N_{p}+1}\bigg{]}r-\frac{N_{p}[(1-p)(1-\gamma)]^{2}}{N_{p}+1}=0$
(159) $\displaystyle\Longleftrightarrow{}$
$\displaystyle\left[r-(1-p)\right]\left[r-(1-\gamma)\right]\left[r-\frac{N_{p}(1-p)(1-\gamma)}{N_{p}+1}\right]=0$

Thus, the three roots of (159) are determined as

$r_{1}=1-p,r_{2}=1-\gamma,\text{ and
}r_{3}=\frac{N_{p}(1-p)(1-\gamma)}{N_{p}+1}$

According to the difference equation theory, the solution of probabilities
$\pi_{(n,1)}$, $n\geq 3$ can be written as

$\pi_{(n,1)}=c_{1}r_{1}^{n}+c_{2}r_{2}^{n}+c_{3}r_{3}^{n}$ (160)

where $c_{1}$, $c_{2}$ and $c_{3}$ are undetermined coefficients, and can be
obtained by solving the following system of equations

$\pi_{(k,1)}=c_{1}(1-p)^{k}+c_{2}(1-\gamma)^{k}+c_{3}\left(\frac{N_{p}(1-p)(1-\gamma)}{N_{p}+1}\right)^{k}\quad(k=3,4,5)$
(161)

In form, we solve that

$\begin{cases}c_{1}=\frac{r_{2}r_{3}\pi_{(3,1)}-(r_{2}+r_{3})\pi_{(4,1)}+\pi_{(5,1)}}{(r_{2}-r_{1})(r_{3}-r_{1})r_{1}^{3}}\\\
c_{2}=\frac{r_{1}r_{3}\pi_{(3,1)}-(r_{1}+r_{3})\pi_{(4,1)}+\pi_{(5,1)}}{(r_{1}-r_{2})(r_{3}-r_{2})r_{2}^{3}}\\\
c_{3}=\frac{r_{1}r_{2}\pi_{(3,1)}-(r_{1}+r_{2})\pi_{(4,1)}+\pi_{(5,1)}}{(r_{1}-r_{3})(r_{2}-r_{3})r_{3}^{3}}\end{cases}$
(162)

The probabilities $\pi_{(k,1)}$, $k=3,4,5$ can be determined by applying the
second line of (143) directly. Substituting these probabilities, eventually
$c_{i}$, $i=1,2,3$ are represented by certain polynomials consisting of $p$,
$\gamma$, and $N_{p}$, but the exact expressions can be very complex. Thus, we
do not compute $c_{i}$ further, while determining their accurate values when
the numerical simulations of AoI-distribution are considered.

Collecting equations (149), (150), and (160), we obtain all the stationary
probabilities $\pi_{(n,m)}$ of the stochastic process
$AoI_{Ber/Geo/1/1/\tilde{g}(m)}$. This completes the proof of Theorem 3.

##  Appendix C Proof of Lemma 2

In this Appendix, we calculate following two sums

$\displaystyle M_{1}$
$\displaystyle=\sum\nolimits_{n=1}^{\infty}\pi_{(n,0,0)}=\frac{(1-p)\gamma^{2}}{(p+\gamma-2p\gamma)+p^{2}(1-\gamma)^{2}}$
$\displaystyle M_{2}$
$\displaystyle=\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{m=2}^{n-1}\pi_{(n,m,0)}=\frac{p\gamma(1-\gamma)}{(p+\gamma-2p\gamma)+p^{2}(1-\gamma)^{2}}$

In Lemma 1, we have obtained that for $n>m\geq 1$, the stationary
probabilities $\pi_{(n,m,0)}$ are determined by

$\displaystyle\pi_{(n,m,0)}$
$\displaystyle=\pi_{(n-m,0,0)}p(1-\gamma)\left[(1-p)(1-\gamma)\right]^{m-1}$
$\displaystyle\qquad+\frac{\gamma}{1-\gamma}\left(\sum\nolimits_{k=n-m+1}^{\infty}\pi_{(k,n-m,0)}\right)\left\\{(1-\gamma)^{m}-[(1-p)(1-\gamma)]^{m}\right\\}$
(163)

We add up all the probabilities $\\{\pi_{(n,m,0)},n>m\geq 1\\}$, showing that
an equation containing $M_{1}$ and $M_{2}$ can be deduced.

$\displaystyle M_{2}$
$\displaystyle=\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}$
$\displaystyle=\sum\nolimits_{n>m\geq
1}\bigg{[}\pi_{(n-m,0,0)}p(1-\gamma)\left[(1-p)(1-\gamma)\right]^{m-1}+\frac{\gamma}{1-\gamma}\left(\sum\nolimits_{k=n-m+1}^{\infty}\pi_{(k,n-m,0)}\right)\left\\{(1-\gamma)^{m}-[(1-p)(1-\gamma)]^{m}\right\\}\bigg{]}$
$\displaystyle=\sum\nolimits_{t=n-m=1}^{\infty}\sum\nolimits_{m=1}^{\infty}\bigg{(}\pi_{(t,0,0)}p(1-\gamma)\left[(1-p)(1-\gamma)\right]^{m-1}+\frac{\gamma}{1-\gamma}\left(\sum\nolimits_{k=t+1}^{\infty}\pi_{(k,t,0)}\right)\left\\{(1-\gamma)^{m}-[(1-p)(1-\gamma)]^{m}\right\\}\bigg{)}$
$\displaystyle=\sum\nolimits_{t=1}^{\infty}\bigg{\\{}\pi_{(t,0,0)}p(1-\gamma)\sum\nolimits_{m=1}^{\infty}[(1-\gamma)(1-p)]^{m-1}+\frac{\gamma}{1-\gamma}\left(\sum\nolimits_{k=t+1}^{\infty}\pi_{(k,t,0)}\right)\sum\nolimits_{m=1}^{\infty}\left\\{(1-\gamma)^{m}-[(1-p)(1-\gamma)]^{m}\right\\}\bigg{\\}}$
$\displaystyle=\sum\nolimits_{t=1}^{\infty}\bigg{(}\pi_{(t,0,0)}p(1-\gamma)\frac{1}{p+\gamma-p\gamma}+\frac{\gamma}{1-\gamma}\left(\sum\nolimits_{k=t+1}^{\infty}\pi_{(k,t,0)}\right)\frac{p(1-\gamma)}{\gamma(p+\gamma-p\gamma)}\bigg{)}$
$\displaystyle=\frac{p(1-\gamma)}{p+\gamma-p\gamma}\sum\nolimits_{t=1}^{\infty}\pi_{(t,0,0)}+\frac{p}{p+\gamma-p\gamma}\sum\nolimits_{t=1}^{\infty}\sum\nolimits_{k=t+1}^{\infty}\pi_{(k,t,0)}$
$\displaystyle=\frac{p(1-\gamma)}{p+\gamma-p\gamma}M_{1}+\frac{p}{p+\gamma-p\gamma}M_{2}$
(164)

To obtain equation (164), notice that

$\sum\nolimits_{t=1}^{\infty}\sum\nolimits_{k=t+1}^{\infty}\pi_{(k,t,0)}=\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}=M_{2}$

Therefore, we derive the first relation between the two sums $M_{1}$ and
$M_{2}$ as

$p(1-\gamma)M_{1}=(1-p)\gamma M_{2}$ (165)

Another relation can be found from the boundary condition which says that

$\displaystyle 1$
$\displaystyle=\sum\nolimits_{n=1}^{\infty}\pi_{(n,0,0)}+\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}+\sum\nolimits_{n=3}^{\infty}\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n,m,l)}$
$\displaystyle=M_{1}+M_{2}+\sum\nolimits_{n>m>l\geq 1}\pi_{(n,m,l)}$ (166)

where in (166) we need to compute only the last sum. It shows that

$\displaystyle\sum\nolimits_{n>m>l\geq 1}\pi_{(n,m,l)}$ $\displaystyle={}$
$\displaystyle\sum\nolimits_{n>m>l\geq 1}\pi_{(n-l,m-l,0)}p(1-\gamma)^{l}$
$\displaystyle={}$ $\displaystyle\sum\nolimits_{n>m>l\geq
1}\bigg{(}\pi_{(n-m,0,0)}p(1-\gamma)[(1-p)(1-\gamma)]^{m-l-1}$
$\displaystyle\qquad+\frac{\gamma}{1-\gamma}\left(\sum\nolimits_{k=n-m+1}^{\infty}\pi_{(k,n-m,0)}\right)\left\\{(1-\gamma)^{m-l}-[(1-p)(1-\gamma)]^{m-l}\right\\}\bigg{)}p(1-\gamma)^{l}$
$\displaystyle={}$ $\displaystyle\sum\nolimits_{n>m>l\geq
1}\bigg{(}\pi_{(n-m,0,0)}p^{2}(1-p)^{m-l-1}(1-\gamma)^{m}$
$\displaystyle\qquad+\frac{p\gamma}{1-\gamma}\left(\sum\nolimits_{k=n-m+1}^{\infty}\pi_{(k,n-m,0)}\right)\left\\{(1-\gamma)^{m}-(1-p)^{m-l}(1-\gamma)^{m}\right\\}\bigg{)}$
$\displaystyle={}$ $\displaystyle\sum\nolimits_{n>m\geq
2}\bigg{(}\pi_{(n-m,0,0)}p^{2}\frac{1-(1-p)^{m-1}}{p}(1-\gamma)^{m}$
$\displaystyle\qquad+\frac{p\gamma}{1-\gamma}\left(\sum\nolimits_{k=n-m+1}^{\infty}\pi_{(k,n-m,0)}\right)\bigg{[}(m-1)(1-\gamma)^{m}-\frac{(1-p)-(1-p)^{m}}{p}(1-\gamma)^{m}\bigg{]}\bigg{)}$
$\displaystyle={}$ $\displaystyle\sum\nolimits_{n>m\geq
2}\bigg{(}\pi_{(n-m,0,0)}p(1-\gamma)\bigg{\\{}(1-\gamma)^{m-1}-[(1-p)(1-\gamma)]^{m-1}\bigg{\\}}$
$\displaystyle\qquad+\frac{\gamma}{1-\gamma}\left(\sum\nolimits_{k=n-m+1}^{\infty}\pi_{(k,n-m,0)}\right)\bigg{\\{}(pm-1)(1-\gamma)^{m}+[(1-p)(1-\gamma)]^{m}\bigg{\\}}\bigg{)}$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{t=n-m=1}^{\infty}\sum\nolimits_{m=2}^{\infty}\bigg{(}\pi_{(t,0,0)}p(1-\gamma)\bigg{\\{}(1-\gamma)^{m-1}-[(1-p)(1-\gamma)]^{m-1}\bigg{\\}}$
$\displaystyle\qquad+\frac{\gamma}{1-\gamma}\left(\sum\nolimits_{k=t+1}^{\infty}\pi_{(k,t,0)}\right)\bigg{\\{}(pm-1)(1-\gamma)^{m}+[(1-p)(1-\gamma)]^{m}\bigg{\\}}\bigg{)}$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{t=1}^{\infty}\bigg{\\{}\pi_{(t,0,0)}p(1-\gamma)\frac{p(1-\gamma)}{\gamma(p+\gamma-p\gamma)}$
$\displaystyle\qquad+\frac{\gamma}{1-\gamma}\left(\sum\nolimits_{k=t+1}^{\infty}\pi_{(k,t,0)}\right)\bigg{[}\frac{(1-\gamma)^{2}[p-(1-p)\gamma]}{\gamma^{2}}+\frac{(1-p)^{2}(1-\gamma)^{2}}{p+\gamma-p\gamma}\bigg{]}\bigg{\\}}$
(167)

The coefficient of $\sum\nolimits_{k=t+1}^{\infty}\pi_{(k,t,0)}$ in (167) is
equal to

$\displaystyle\frac{\gamma}{1-\gamma}\left[\frac{(1-\gamma)^{2}[p-(1-p)\gamma]}{\gamma^{2}}+\frac{(1-p)^{2}(1-\gamma)^{2}}{p+\gamma-p\gamma}\right]$
$\displaystyle={}$
$\displaystyle\frac{(1-\gamma)[p-(1-p)\gamma]}{\gamma}+\frac{\gamma(1-p)^{2}(1-\gamma)}{p+\gamma-p\gamma}$
$\displaystyle={}$
$\displaystyle(1-\gamma)\left[\frac{p-(1-p)\gamma}{\gamma}+\frac{(1-p)^{2}\gamma}{p+\gamma-p\gamma}\right]$
$\displaystyle={}$
$\displaystyle(1-\gamma)\frac{[p-(1-p)\gamma][p+(1-p)\gamma]+(1-p)^{2}\gamma^{2}}{\gamma(p+\gamma-p\gamma)}$
$\displaystyle={}$
$\displaystyle\frac{p^{2}(1-\gamma)}{\gamma(p+\gamma-p\gamma)}$

Therefore, we obtain that

$\displaystyle\sum\nolimits_{n>m>l\geq 1}\pi_{(n,m,l)}$
$\displaystyle=\frac{p^{2}(1-\gamma)^{2}}{\gamma(p+\gamma-p\gamma)}\left(\sum\nolimits_{t=1}^{\infty}\pi_{(t,0,0)}\right)+\frac{p^{2}(1-\gamma)}{\gamma(p+\gamma-p\gamma)}\left(\sum\nolimits_{t=1}^{\infty}\sum\nolimits_{k=t+1}^{\infty}\pi_{(k,t,0)}\right)$
$\displaystyle=\frac{p^{2}(1-\gamma)^{2}}{\gamma(p+\gamma-p\gamma)}M_{1}+\frac{p^{2}(1-\gamma)}{\gamma(p+\gamma-p\gamma)}M_{2}$
(168)

Substituting (168) into (166) yields that

$1=\left[1+\frac{p^{2}(1-\gamma)^{2}}{\gamma(p+\gamma-p\gamma)}\right]M_{1}+\left[1+\frac{p^{2}(1-\gamma)}{\gamma(p+\gamma-p\gamma)}\right]M_{2}$
(169)

which gives another relationship between $M_{1}$ and $M_{2}$.

Combining equations (165) and (169), we can solve that

$M_{1}=\frac{(1-p)\gamma^{2}}{(p+\gamma-2p\gamma)\gamma+p^{2}(1-\gamma)^{2}},\qquad
M_{2}=\frac{p\gamma(1-\gamma)}{(p+\gamma-2p\gamma)\gamma+p^{2}(1-\gamma)^{2}}$

In addition, according to the last line of equations (39), the first
probability $\pi_{(1,0,0)}$ is determined as

$\pi_{(1,0,0)}=\left(\sum\nolimits_{n=1}^{\infty}\pi_{(n,0,0)}\right)p\gamma=p\gamma
M_{1}=\frac{p(1-p)\gamma^{3}}{(p+\gamma-2p\gamma)\gamma+p^{2}(1-\gamma)^{2}}$

So far, we complete the proof of Lemma 2.

##  Appendix D Proof of Lemma 3

In this part, we compute the infinite sum
$\sum\nolimits_{k=n}^{\infty}\pi_{(k,n-1,0)}$ and prove Lemma 3.

First of all, according to (39), probabilities $\pi_{(k,n-1,0)}$, $k\geq n\geq
3$ are determined by

$\displaystyle\pi_{(k,n-1,0)}$
$\displaystyle=\pi_{(k-1,n-2,0)}(1-p)(1-\gamma)+\left(\sum\nolimits_{j=k}^{\infty}\pi_{(j,k-1,n-2)}\right)\gamma$
(170)

Summing up (170) from $k=n$ to infinity, we have

$\displaystyle\sum\nolimits_{k=n}^{\infty}\pi_{(k,n-1,0)}$
$\displaystyle=\sum\nolimits_{k=n}^{\infty}\bigg{\\{}\pi_{(k-1,n-2,0)}(1-p)(1-\gamma)+\left(\sum\nolimits_{j=k}^{\infty}\pi_{(j,k-1,n-2)}\right)\gamma\bigg{\\}}$
$\displaystyle=\left(\sum\nolimits_{k=n-1}^{\infty}\pi_{(k,n-2,0)}\right)(1-p)(1-\gamma)+\left(\sum\nolimits_{k=n}^{\infty}\sum\nolimits_{j=k}^{\infty}\pi_{(j-n+2,k-n+1,0)}p(1-\gamma)^{n-2}\right)\gamma$
$\displaystyle=\left(\sum\nolimits_{k=n-1}^{\infty}\pi_{(k,n-2,0)}\right)(1-p)(1-\gamma)+p\gamma\left(\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{m=1}^{\infty}\pi_{(n,m,0)}\right)(1-\gamma)^{n-2}$
$\displaystyle=\left(\sum\nolimits_{k=n-1}^{\infty}\pi_{(k,n-2,0)}\right)(1-p)(1-\gamma)+p\gamma
M_{2}(1-\gamma)^{n-2}$ (171)

where $M_{2}$ is defined as

$M_{2}=\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}$

and has been obtained in Lemma 2.

Equation (171) gives a recursive relation for the infinite sums
$\sum\nolimits_{k=n}^{\infty}\pi_{(k,n-1,0)}$, $n\geq 2$, from which the
general formula of the sums can be derived. We have that

$\displaystyle\sum\nolimits_{k=n}^{\infty}\pi_{(k,n-1,0)}$
$\displaystyle=\left(\sum\nolimits_{k=n-1}^{\infty}\pi_{(k,n-2,0)}\right)(1-p)(1-\gamma)+p\gamma
M_{2}(1-\gamma)^{n-2}$
$\displaystyle=\bigg{[}\left(\sum\nolimits_{k=n-2}^{\infty}\pi_{(k,n-3,0)}\right)(1-p)(1-\gamma)+p\gamma
M_{2}(1-\gamma)^{n-3}\bigg{]}(1-p)(1-\gamma)+p\gamma M_{2}(1-\gamma)^{n-2}$
$\displaystyle=\left(\sum\nolimits_{k=n-2}^{\infty}\pi_{(k,n-3,0)}\right)\left[(1-p)(1-\gamma)\right]^{2}+p\gamma
M_{2}\left\\{[(1-p)(1-\gamma)](1-\gamma)^{n-3}+(1-\gamma)^{n-2}\right\\}$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\vdots$
$\displaystyle=\left(\sum\nolimits_{k=2}^{\infty}\pi_{(k,1,0)}\right)\left[(1-p)(1-\gamma)\right]^{n-2}+p\gamma
M_{2}\left(\sum\nolimits_{j=0}^{n-3}[(1-p)(1-\gamma)]^{j}(1-\gamma)^{n-2-j}\right)$
(172)

In (172), the sum $\sum\nolimits_{k=2}^{\infty}\pi_{(k,1,0)}$ is calculated as

$\displaystyle\sum\nolimits_{k=2}^{\infty}\pi_{(k,1,0)}$
$\displaystyle=\sum\nolimits_{k=2}^{\infty}\bigg{\\{}\pi_{(k-1,0,0)}p(1-\gamma)+\left(\sum\nolimits_{j=k}^{\infty}\pi_{(j,k-1,0)}\right)p\gamma\bigg{\\}}$
$\displaystyle=p(1-\gamma)\left(\sum\nolimits_{k=1}^{\infty}\pi_{(k,0,0)}\right)+p\gamma\left(\sum\nolimits_{k=2}^{\infty}\sum\nolimits_{j=k}^{\infty}\pi_{(j,k-1,0)}\right)$
$\displaystyle=p(1-\gamma)M_{1}+p\gamma M_{2}$ (173)

where $M_{1}$ is also defined and calculated in Lemma 2.

Combining (172) and (173), we have that

$\displaystyle\sum\nolimits_{k=n}^{\infty}\pi_{(k,n-1,0)}$
$\displaystyle=\left[p(1-\gamma)M_{1}+p\gamma
M_{2}\right]\left[(1-p)(1-\gamma)\right]^{n-2}+p\gamma
M_{2}\left[\sum\nolimits_{j=0}^{n-3}(1-p)^{j}(1-\gamma)^{n-2}\right]$
$\displaystyle=\left[p(1-\gamma)M_{1}+p\gamma
M_{2}\right]\left[(1-p)(1-\gamma)\right]^{n-2}+\gamma
M_{2}\Big{(}(1-\gamma)^{n-2}-[(1-p)(1-\gamma)]^{n-2}\Big{)}$
$\displaystyle=\left[p(1-\gamma)M_{1}-(1-p)\gamma
M_{2}\right]\left[(1-p)(1-\gamma)\right]^{n-2}+\gamma M_{2}(1-\gamma)^{n-2}$
$\displaystyle=\gamma M_{2}(1-\gamma)^{n-2}\qquad(n\geq 2)$ (174)

To obtain equation (174), notice that

$p(1-\gamma)M_{1}=(1-p)\gamma M_{2}$

which is proved in equation (165). Thus, we complete the proof of Lemma 3.

##  Appendix E The proof of Theorem 8

In this appendix, we determine the probability the AoI equals $n$ in steady
state and the AoI cumulative probabilities, assuming that the queue model used
in the status updating system is Ber/Geo/1/2.

First of all, the sums $\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}$, $n\geq 2$,
and $\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n,m,l)}$, $n\geq
3$ are calculated.

$\displaystyle\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}$ $\displaystyle={}$
$\displaystyle\sum\nolimits_{m=1}^{n-1}\bigg{\\{}\frac{p^{2}\gamma^{3}}{N(p,\gamma)(\gamma-p)}\bigg{[}(1-p)^{n}(1-\gamma)^{m}-(1-p)^{m}(1-\gamma)^{n}\bigg{]}+\frac{p\gamma^{3}}{N(p,\gamma)}\bigg{[}(1-\gamma)^{n-1}-(1-p)^{m}(1-\gamma)^{n-1}\bigg{]}\bigg{\\}}$
$\displaystyle={}$
$\displaystyle\frac{p^{2}\gamma^{3}}{N(p,\gamma)(\gamma-p)}\bigg{[}(1-p)^{n}\frac{(1-\gamma)-(1-\gamma)^{n}}{\gamma}-\frac{(1-p)-(1-p)^{n}}{p}(1-\gamma)^{n}\bigg{]}$
$\displaystyle\qquad\quad+\frac{p\gamma^{3}}{N(p,\gamma)}\bigg{[}(n-1)(1-\gamma)^{n-1}-\frac{(1-p)-(1-p)^{n}}{p}(1-\gamma)^{n-1}\bigg{]}$
$\displaystyle={}$
$\displaystyle\frac{p\gamma^{2}}{N(p,\gamma)(\gamma-p)}\bigg{\\{}p(1-\gamma)(1-p)^{n}-(1-p)\gamma(1-\gamma)^{n}+(\gamma-p)[(1-p)(1-\gamma)]^{n}\bigg{\\}}$
$\displaystyle\qquad\quad+\frac{\gamma^{3}}{N(p,\gamma)}\bigg{\\{}(np-1)(1-\gamma)^{n-1}+(1-p)[(1-p)(1-\gamma)]^{n-1}\bigg{\\}}$
(175)

The calculation of the latter sum is more complicated and tedious. We give the
details below.

$\displaystyle\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n,m,l)}$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n-l,m-l,0)}p(1-\gamma)^{l}$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\bigg{[}\frac{p^{2}\gamma^{3}}{N(p,\gamma)(\gamma-p)}\bigg{\\{}(1-p)^{n-l}(1-\gamma)^{m-l}-(1-p)^{m-l}(1-\gamma)^{n-l}\bigg{\\}}$
$\displaystyle\qquad\qquad\qquad\quad+\frac{p\gamma^{3}}{N(p,\gamma)}\bigg{\\{}(1-\gamma)^{n-l-1}-(1-p)^{m-l}(1-\gamma)^{n-l-1}\bigg{\\}}\bigg{]}p(1-\gamma)^{l}$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\bigg{[}\frac{(p\gamma)^{3}}{N(p,\gamma)(\gamma-p)}\bigg{\\{}(1-p)^{n-l}(1-\gamma)^{m}-(1-p)^{m-l}(1-\gamma)^{n}\bigg{\\}}$
$\displaystyle\qquad\qquad\qquad\quad+\frac{p^{2}\gamma^{3}}{N(p,\gamma)}\bigg{\\{}(1-\gamma)^{n-1}-(1-p)^{m-l}(1-\gamma)^{n-1}\bigg{\\}}\bigg{]}$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{m=2}^{n-1}\bigg{[}\frac{(p\gamma)^{3}}{N(p,\gamma)(\gamma-p)}\bigg{\\{}\frac{(1-p)^{n-m+1}-(1-p)^{n}}{p}(1-\gamma)^{m}-\frac{(1-p)-(1-p)^{m}}{p}(1-\gamma)^{n}\bigg{\\}}$
$\displaystyle\qquad\qquad\qquad\quad+\frac{p^{2}\gamma^{3}}{N(p,\gamma)}\bigg{\\{}(m-1)(1-\gamma)^{n-1}-\frac{(1-p)-(1-p)^{m}}{p}(1-\gamma)^{n-1}\bigg{\\}}\bigg{]}$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{m=2}^{n-1}\bigg{[}\frac{p^{2}\gamma^{3}}{N(p,\gamma)(\gamma-p)}\bigg{\\{}(1-p)^{n-m+1}(1-\gamma)^{m}-(1-p)^{n}(1-\gamma)^{m}-(1-p)(1-\gamma)^{n}+(1-p)^{m}(1-\gamma)^{n}\bigg{\\}}$
$\displaystyle\qquad\qquad\qquad\quad+\frac{p\gamma^{3}}{N(p,\gamma)}\bigg{\\{}(m-1)p(1-\gamma)^{n-1}-(1-p)(1-\gamma)^{n-1}+(1-p)^{m}(1-\gamma)^{n-1}\bigg{\\}}\bigg{]}$
$\displaystyle={}$
$\displaystyle\frac{p^{2}\gamma^{3}}{N(p,\gamma)(\gamma-p)}\bigg{\\{}(1-p)^{n+1}\sum\nolimits_{m=2}^{n-1}\left(\frac{1-\gamma}{1-p}\right)^{m}-(1-p)^{n}\frac{(1-\gamma)^{2}-(1-\gamma)^{n}}{\gamma}$
$\displaystyle\qquad\qquad\qquad\qquad\qquad-(n-2)(1-p)(1-\gamma)^{n}+\frac{(1-p)^{2}-(1-p)^{n}}{p}(1-\gamma)^{n}\bigg{\\}}$
$\displaystyle\qquad\qquad\qquad\quad+\frac{p\gamma^{3}}{N(p,\gamma)}\bigg{\\{}\left[\sum\nolimits_{m=2}^{n-1}(mp-1)\right](1-\gamma)^{n-1}+\frac{(1-p)^{2}-(1-p)^{n}}{p}(1-\gamma)^{n-1}\bigg{\\}}$
(176)

The two sums in (176) are computed as follows. Firstly, we have

$\displaystyle(1-p)^{n+1}\sum\nolimits_{m=2}^{n-1}\left(\frac{1-\gamma}{1-p}\right)^{m}=(1-p)^{n+1}\frac{\left(\frac{1-\gamma}{1-p}\right)^{2}-\left(\frac{1-\gamma}{1-p}\right)^{n}}{1-\frac{1-\gamma}{1-p}}=\frac{(1-\gamma)^{2}(1-p)^{n}-(1-p)^{2}(1-\gamma)^{n}}{\gamma-p}$

In addition,

$\displaystyle\left[\sum\nolimits_{m=2}^{n-1}(mp-1)\right](1-\gamma)^{n-1}=\left[\frac{(n+1)(n-2)}{2}p-(n-2)\right](1-\gamma)^{n-1}=\left[\frac{p}{2}n^{2}-\left(\frac{p}{2}+1\right)n+(2-p)\right](1-\gamma)^{n-1}$

Substituting above results into (176) gives that

$\displaystyle\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n,m,l)}$
$\displaystyle={}$
$\displaystyle\frac{p^{2}\gamma^{3}}{N(p,\gamma)(\gamma-p)}\bigg{\\{}\frac{p(1-\gamma)^{2}}{\gamma(\gamma-p)}(1-p)^{n}+\frac{(\gamma-2p)(1-p)^{2}}{p(\gamma-p)}(1-\gamma)^{n}-\frac{\gamma-p}{p\gamma}[(1-p)(1-\gamma)]^{n}-(n-2)(1-p)(1-\gamma)^{n}\bigg{\\}}$
$\displaystyle\qquad\quad+\frac{p\gamma^{3}}{N(p,\gamma)}\bigg{\\{}\left[\frac{p}{2}n^{2}-\left(\frac{p}{2}+1\right)n+(2-p)\right](1-\gamma)^{n-1}-\frac{1-p}{p}[(1-p)(1-\gamma)]^{n-1}\bigg{\\}}$
$\displaystyle={}$
$\displaystyle\frac{p^{3}\gamma^{2}(1-\gamma)^{2}}{N(p,\gamma)(\gamma-p)^{2}}(1-p)^{n}+\frac{p\gamma^{3}(\gamma-2p)(1-p)^{2}}{N(p,\gamma)(\gamma-p)^{2}}(1-\gamma)^{n}-\frac{\gamma^{2}(p+\gamma-p\gamma)}{N(p,\gamma)(1-\gamma)}[(1-p)(1-\gamma)]^{n}$
$\displaystyle\qquad\quad-\frac{p^{2}\gamma^{3}(1-p)}{N(p,\gamma)(\gamma-p)}(n-2)(1-\gamma)^{n}+\frac{\gamma^{3}[p^{2}n^{2}-(p^{2}+2p)n+2]}{2N(p,\gamma)}(1-\gamma)^{n-1}$
(177)

Now, the stationary AoI distribution can be determined. For $n\geq 3$, we only
need to add up all the stationary probabilities having identical first age-
component. That is

$\displaystyle\Pr\\{\Delta=n\\}$
$\displaystyle=\pi_{(n,0,0)}+\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}+\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n,m,l)}$
$\displaystyle=\frac{p(1-p)^{2}\gamma^{4}}{N(p,\gamma)(\gamma-p)^{2}}[(1-p)^{n}-(1-\gamma)^{n}]-\frac{p^{2}\gamma^{3}}{N(p,\gamma)}\left[\frac{1}{2(1-\gamma)}+\frac{1-p}{\gamma-p}\right]n(1-\gamma)^{n}+\frac{p^{2}\gamma^{3}}{2N(p,\gamma)}n^{2}(1-\gamma)^{n-1}$
(178)

The calculations to obtain expression (178) are direct, so that we omit the
details. Instead, we will check that (178) indeed forms a proper probability
distribution by proving that the sum of all the probabilities is equal to 1.

Notice that equation (175) is zero when $n=1$ and the second sum (177) equals
zero for both $n=1$ and $n=2$. Thus, actually the probability expression (178)
is valid for all $n\geq 1$.

Notice that

$\displaystyle\sum\nolimits_{n=1}^{\infty}\Pr\\{\Delta=n\\}$
$\displaystyle=\sum\nolimits_{n=1}^{\infty}\bigg{\\{}\frac{p(1-p)^{2}\gamma^{4}}{N(p,\gamma)(\gamma-p)^{2}}[(1-p)^{n}-(1-\gamma)^{n}]$
$\displaystyle\qquad\qquad-\frac{p^{2}\gamma^{3}}{N(p,\gamma)}\left[\frac{1}{2(1-\gamma)}+\frac{1-p}{\gamma-p}\right]n(1-\gamma)^{n}+\frac{p^{2}\gamma^{3}}{2N(p,\gamma)}n^{2}(1-\gamma)^{n-1}\bigg{\\}}$
$\displaystyle=\frac{p(1-p)^{2}\gamma^{4}}{N(p,\gamma)(\gamma-p)^{2}}\left[\frac{1-p}{p}-\frac{1-\gamma}{\gamma}\right]-\frac{p^{2}\gamma^{3}}{N(p,\gamma)}\left[\frac{1}{2(1-\gamma)}+\frac{1-p}{\gamma-p}\right]\frac{1-\gamma}{\gamma^{2}}+\frac{p^{2}\gamma^{3}}{2N(p,\gamma)}\frac{2-\gamma}{\gamma^{3}}$
(179)
$\displaystyle=\frac{(1-p)^{2}\gamma^{3}}{N(p,\gamma)(\gamma-p)}-\frac{p^{2}\gamma}{2N(p,\gamma)}-\frac{p^{2}\gamma(1-p)(1-\gamma)}{N(p,\gamma)(\gamma-p)}+\frac{p^{2}(2-\gamma)}{2N(p,\gamma)}$
$\displaystyle=\frac{(1-p)\gamma}{N(p,\gamma)(\gamma-p)}\Big{\\{}(1-p)\gamma^{2}-p^{2}(1-\gamma)\Big{\\}}+\frac{p^{2}(1-\gamma)}{N(p,\gamma)}$
$\displaystyle=\frac{(1-p)\gamma(p+\gamma-p\gamma)+p^{2}(1-\gamma)}{N(p,\gamma)}$
(180)

In equation (179), we compute the sum

$\displaystyle\sum\nolimits_{n=1}^{\infty}n^{2}(1-\gamma)^{n-1}=\sum\nolimits_{n=1}^{\infty}n(n-1)(1-\gamma)^{n-1}+\sum\nolimits_{n=1}^{\infty}n(1-\gamma)^{n-1}=(1-\gamma)\frac{2}{\gamma^{3}}+\frac{1}{\gamma^{2}}=\frac{2-\gamma}{\gamma^{3}}$

and (180) is obtained since
$(1-p)\gamma^{2}-p^{2}(1-\gamma)=(\gamma-p)(p+\gamma-p\gamma)$.

Further, the numerator of (180) can be rewritten as

$\displaystyle(1-p)\gamma(p+\gamma-p\gamma)+p^{2}(1-\gamma)$
$\displaystyle={}$
$\displaystyle\gamma(p+\gamma-p\gamma)-p\gamma(p+\gamma-p\gamma)+p^{2}(1-\gamma)$
$\displaystyle={}$
$\displaystyle\gamma(p+\gamma-2p\gamma)+p\gamma^{2}-p\gamma(p+\gamma-p\gamma)+p^{2}(1-\gamma)$
$\displaystyle={}$
$\displaystyle\gamma(p+\gamma-2p\gamma)-p^{2}\gamma(1-\gamma)+p^{2}(1-\gamma)$
$\displaystyle={}$
$\displaystyle\gamma(p+\gamma-2p\gamma)+p^{2}(1-\gamma)^{2}$
$\displaystyle={}$ $\displaystyle N(p,\gamma)$ (181)

which proves that the sum of all the probabilities is 1.

Now, we compute the AoI cumulative probabilities. Let $k\geq 1$, it shows that

$\displaystyle\Pr\\{\Delta\leq k\\}=\sum\nolimits_{n=1}^{k}\Pr\\{\Delta=n\\}$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{n=1}^{k}\bigg{\\{}\frac{p(1-p)^{2}\gamma^{4}}{N(p,\gamma)(\gamma-p)^{2}}[(1-p)^{n}-(1-\gamma)^{n}]-\frac{p^{2}\gamma^{3}}{N(p,\gamma)}\left[\frac{1}{2(1-\gamma)}+\frac{1-p}{\gamma-p}\right]n(1-\gamma)^{n}+\frac{p^{2}\gamma^{3}}{2N(p,\gamma)}n^{2}(1-\gamma)^{n-1}\bigg{\\}}$

Notice that we have following sums

$\sum\nolimits_{n=1}^{k}n(1-\gamma)^{n-1}=\frac{1-(1-\gamma)^{k}-k\gamma(1-\gamma)^{k}}{\gamma^{2}}$

and

$\sum\nolimits_{n=1}^{k}n^{2}(1-\gamma)^{n-1}=\sum\nolimits_{n=1}^{k}n(n-1)(1-\gamma)^{n-1}+\sum\nolimits_{n=1}^{k}n(1-\gamma)^{n-1}$

in which

$\displaystyle\sum\nolimits_{n=1}^{k}n(n-1)(1-\gamma)^{n-1}$
$\displaystyle=(1-\gamma)\left[\sum\nolimits_{n=0}^{k}x^{n}\right]^{\prime\prime}_{x=1-\gamma}$
$\displaystyle=(1-\gamma)\bigg{\\{}\frac{2}{\gamma^{3}}-\frac{\gamma^{2}k^{2}(1-\gamma)^{k-1}+(2\gamma-\gamma^{2})k(1-\gamma)^{k-1}+2(1-\gamma)^{k}}{\gamma^{3}}\bigg{\\}}$
$\displaystyle=\frac{2(1-\gamma)-\gamma^{2}k^{2}(1-\gamma)^{k}-(2\gamma-\gamma^{2})k(1-\gamma)^{k}-2(1-\gamma)^{k+1}}{\gamma^{3}}$

Therefore, the probability that AoI is greater than $k$ is equal to

$\displaystyle\Pr\\{\Delta\leq k\\}$
$\displaystyle=\frac{p(1-p)^{2}\gamma^{4}}{N(p,\gamma)(\gamma-p)^{2}}\bigg{\\{}\frac{(1-p)-(1-p)^{k+1}}{p}-\frac{(1-\gamma)-(1-\gamma)^{k+1}}{\gamma}\bigg{\\}}$
$\displaystyle\quad-\frac{p^{2}\gamma^{3}}{N(p,\gamma)}\left[\frac{1}{2(1-\gamma)}+\frac{1-p}{\gamma-p}\right](1-\gamma)\frac{1-(1-\gamma)^{k}-k\gamma(1-\gamma)^{k}}{\gamma^{2}}$
$\displaystyle\qquad+\frac{p^{2}\gamma^{3}}{2N(p,\gamma)}\bigg{\\{}\frac{2(1-\gamma)-\gamma^{2}k^{2}(1-\gamma)^{k}-(2\gamma-\gamma^{2})k(1-\gamma)^{k}-2(1-\gamma)^{k+1}}{\gamma^{3}}+\frac{1-(1-\gamma)^{k}-k\gamma(1-\gamma)^{k}}{\gamma^{2}}\bigg{\\}}$
$\displaystyle=1-\frac{(1-p)^{3}\gamma^{4}}{N(p,\gamma)(\gamma-p)^{2}}(1-p)^{k}+\frac{p(1-\gamma)}{N(p,\gamma)(\gamma-p)}\left[\frac{(1-p)^{2}\gamma^{3}}{\gamma-p}+p^{2}(1-\gamma)\right](1-\gamma)^{k}$
$\displaystyle\qquad+\frac{p^{2}\gamma}{N(p,\gamma)}\left[\frac{(1-p)(1-\gamma)\gamma}{\gamma-p}-\frac{2-\gamma}{2}\right]k(1-\gamma)^{k}-\frac{(p\gamma)^{2}}{2N(p,\gamma)}k^{2}(1-\gamma)^{k}\qquad(k\geq
1)$ (182)

We omit the calculations to obtain the final result (182), and finish the
proof of Theorem 8.

##  Appendix F Proof of Theorem 11

Here, we prove that the probabilities $\pi_{(n,0,0)}$ are equal to

$\pi_{(n,0,0)}=p(1-p)^{n-1}-p(1-\gamma)\left[1+\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}n\right][(1-p)(1-\gamma)]^{n-1}\quad(n\geq
1)$ (183)

Equations (76) shows that

$\pi_{(n,0,0)}=\pi_{(n-1,0,0)}(1-p)+\left(\sum\nolimits_{k=n}^{\infty}\pi_{(k,n-1,0)}\right)(1-p)\gamma\qquad(n\geq
2)$ (184)

Using formula (78), the sum in (184) is dealt with as

$\displaystyle\sum\nolimits_{k=n}^{\infty}\pi_{(k,n-1,0)}$ $\displaystyle={}$
$\displaystyle\sum\nolimits_{k=n}^{\infty}\left\\{\pi_{(k-n+1,0,0)}p(1-\gamma)[(1-p)(1-\gamma)]^{n-2}+p\gamma(n-1)t_{k-n+1}[(1-p)(1-\gamma)]^{n-2}\right\\}$
$\displaystyle={}$
$\displaystyle\left(\sum\nolimits_{k=1}^{\infty}\pi_{(k,0,0)}\right)p(1-\gamma)[(1-p)(1-\gamma)]^{n-2}+p\gamma(n-1)\left(\sum\nolimits_{k=1}^{\infty}t_{k}\right)[(1-p)(1-\gamma)]^{n-2}$
(185)

Define $t_{0}=\sum\nolimits_{n=1}^{\infty}\pi_{(n,0,0)}$, then we have the
equation

$t_{0}+\sum\nolimits_{k=1}^{\infty}t_{k}=1$ (186)

since $t_{0}$ and $t_{k}$, $k\geq 1$ constitutes the probability distribution
of the second age-component in state vector $(n,m,l)$. Applying also the last
line in (76), which says that
$\sum\nolimits_{k=1}^{\infty}\pi_{(k,0,0)}=\frac{\pi_{(1,0,0)}}{p\gamma}$, we
obtain that

$\displaystyle\sum\nolimits_{k=n}^{\infty}\pi_{(k,n-1,0)}$ $\displaystyle={}$
$\displaystyle\frac{\pi_{(1,0,0)}}{p\gamma}p(1-\gamma)[(1-p)(1-\gamma)]^{n-2}+p\gamma(n-1)\left(1-\frac{\pi_{(1,0,0)}}{p\gamma}\right)[(1-p)(1-\gamma)]^{n-2}$
$\displaystyle={}$
$\displaystyle\frac{\pi_{(1,0,0)(1-\gamma)}}{\gamma}[(1-p)(1-\gamma)]^{n-2}+\left(p\gamma-\pi_{(1,0,0)}\right)(n-1)[(1-p)(1-\gamma)]^{n-2}$
(187)

Therefore,

$\pi_{(n,0,0)}=\pi_{(n-1,0,0)}(1-p)+\pi_{(1,0,0)}[(1-p)(1-\gamma)]^{n-1}+(1-p)\gamma\left(p\gamma-\pi_{(1,0,0)}\right)(n-1)[(1-p)(1-\gamma)]^{n-2}$
(188)

Using equation (188) repeatedly yields that

$\displaystyle\pi_{(n,0,0)}$
$\displaystyle=\pi_{(1,0,0)}(1-p)^{n-1}+\pi_{(1,0,0)}\left(\sum\nolimits_{j=0}^{n-2}(1-p)^{j}[(1-p)(1-\gamma)]^{n-1-j}\right)$
$\displaystyle\qquad+(1-p)\gamma\left(p\gamma-\pi_{(1,0,0)}\right)\left(\sum\nolimits_{j=0}^{n-2}(1-p)^{j}(n-1-j)[(1-p)(1-\gamma)]^{n-2-j}\right)$
$\displaystyle=\pi_{(1,0,0)}(1-p)^{n-1}+\pi_{(1,0,0)}\frac{1-\gamma}{\gamma}\left\\{(1-p)^{n-1}-[(1-p)(1-\gamma)]^{n-1}\right\\}$
$\displaystyle\qquad+(1-p)\gamma\left(p\gamma-\pi_{(1,0,0)}\right)\frac{(1-p)^{n-1}-[1+(n-1)\gamma][(1-p)(1-\gamma)]^{n-1}}{(1-p)\gamma^{2}}$
$\displaystyle=p(1-p)^{n-1}-p(1-\gamma)[(1-p)(1-\gamma)]^{n-1}-\left(p\gamma-\pi_{(1,0,0)}\right)n[(1-p)(1-\gamma)]^{n-1}$
(189)

Let $n=1$ in (189), it gives the obvious equation
$\pi_{(1,0,0)}=\pi_{(1,0,0)}$. Thus, expression is appliable for all $n\geq
1$.

Using the last line of (76) again, it shows that

$\frac{\pi_{(1,0,0)}}{p\gamma}=\sum\nolimits_{n=1}^{\infty}\left\\{p(1-p)^{n-1}-p(1-\gamma)[(1-p)(1-\gamma)]^{n-1}-\left(p\gamma-\pi_{(1,0,0)}\right)n[(1-p)(1-\gamma)]^{n-1}\right\\}$

from which we derive that

$\pi_{(1,0,0)}=\frac{p(1-p)\gamma^{3}}{(p+\gamma-p\gamma)^{2}-p\gamma}$ (190)

Notice that the denominator of $\pi_{(1,0,0)}$ is always positive since we
have

$(p+\gamma-p\gamma)^{2}-p\gamma>(p+\gamma-p\gamma)^{2}-\gamma^{2}=\left[(p+\gamma-p\gamma)+\gamma\right]p(1-\gamma)>0$

Substitute (190) into (189), eventually we prove that the probability
$\pi_{(n,0,0)}$ is equal to

$\displaystyle\pi_{(n,0,0)}$
$\displaystyle=p(1-p)^{n-1}-p(1-\gamma)[(1-p)(1-\gamma)]^{n-1}-\frac{p^{2}\gamma(1-\gamma)(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}n[(1-p)(1-\gamma)]^{n-1}$
$\displaystyle=p(1-p)^{n-1}-p(1-\gamma)\left[1+\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}n\right][(1-p)(1-\gamma)]^{n-1}$
(191)

This completes the proof of Theorem 11.

##  Appendix G Proof of Lemma 5

In this appendix, we prove Lemma 5 which determines the following sums

$t_{n}=\sum\nolimits_{k=n+1}^{\infty}\sum\nolimits_{j=0}^{n-1}\pi_{(k,n,j)}\qquad(n\geq
1)$

and $t_{0}=\sum\nolimits_{k=1}^{\infty}\pi_{(k,0,0)}$.

We have that

$\displaystyle t_{n}$
$\displaystyle=\sum\nolimits_{k=n+1}^{\infty}\sum\nolimits_{j=0}^{n-1}\pi_{(k,n,j)}$
$\displaystyle=\sum\nolimits_{k=n+1}^{\infty}\pi_{(k,n,0)}+\sum\nolimits_{k=n+1}^{\infty}\sum\nolimits_{j=1}^{n-1}\pi_{(k,n,j)}$
$\displaystyle=\sum\nolimits_{k=n+1}^{\infty}\pi_{(k,n,0)}+\sum\nolimits_{k=n+1}^{\infty}\sum\nolimits_{j=1}^{n-1}\left(\sum\nolimits_{y=0}^{n-j-1}\pi_{(k-j,n-j,y)}\right)p(1-\gamma)[(1-p)(1-\gamma)]^{j-1}$
(192)
$\displaystyle=\sum\nolimits_{k=n+1}^{\infty}\pi_{(k,n,0)}+p(1-\gamma)\sum\nolimits_{j=1}^{n-1}\left(\sum\nolimits_{k=n-j+1}^{\infty}\sum\nolimits_{y=0}^{n-j-1}\pi_{(k,n-j,y)}\right)[(1-p)(1-\gamma)]^{j-1}$
$\displaystyle=\sum\nolimits_{k=n+1}^{\infty}\pi_{(k,n,0)}+p(1-\gamma)\sum\nolimits_{j=1}^{n-1}t_{n-j}[(1-p)(1-\gamma)]^{j-1}$
(193)

where in equation (192), the expression (79) is substituted. Applying (78),
the first sum of (193) is calculated as

$\displaystyle\sum\nolimits_{k=n+1}^{\infty}\pi_{(k,n,0)}$ $\displaystyle={}$
$\displaystyle\sum\nolimits_{k=n+1}^{\infty}\left\\{\pi_{(k-n,0,0)}p(1-\gamma)[(1-p)(1-\gamma)]^{n-1}+p\gamma
nt_{k-n}[(1-p)(1-\gamma)]^{n-1}\right\\}$ $\displaystyle={}$
$\displaystyle\left(\sum\nolimits_{k=1}^{\infty}\pi_{(k,0,0)}\right)p(1-\gamma)[(1-p)(1-\gamma)]^{n-1}+p\gamma\left(\sum\nolimits_{k=1}^{\infty}t_{k}\right)n[(1-p)(1-\gamma)]^{n-1}$
$\displaystyle={}$
$\displaystyle\frac{\pi_{(1,0,0)}(1-\gamma)}{\gamma}[(1-p)(1-\gamma)]^{n-1}+\left(p\gamma-\pi_{(1,0,0)}\right)n[(1-p)(1-\gamma)]^{n-1}$
(194)

in which we have used the relations

$\sum\nolimits_{k=1}^{\infty}\pi_{(k,0,0)}=\frac{\pi_{(1,0,0)}}{p\gamma}\text{
and
}\sum\nolimits_{k=1}^{\infty}t_{k}=1-t_{0}=1-\sum\nolimits_{k=1}^{\infty}\pi_{(k,0,0)}$

First of all, we can derive that

$t_{0}=\frac{\pi_{(1,0,0)}}{p\gamma}=\frac{(1-p)\gamma^{2}}{(p+\gamma-p\gamma)^{2}-p\gamma}$
(195)

since $\pi_{(1,0,0)}$ is known in (190).

For $n\geq 2$, we have that

$t_{n}=p(1-\gamma)\sum\nolimits_{j=1}^{n-1}t_{n-j}[(1-p)(1-\gamma)]^{j-1}+\frac{\pi_{(1,0,0)}(1-\gamma)}{\gamma}[(1-p)(1-\gamma)]^{n-1}+\left(p\gamma-\pi_{(1,0,0)}\right)n[(1-p)(1-\gamma)]^{n-1}$
(196)

According to (196), similarly we can write the expression of $t_{n-1}$.
Observing that the following difference

$t_{n}-(1-p)(1-\gamma)t_{n-1}=p(1-\gamma)t_{n-1}+\left(p\gamma-\pi_{(1,0,0)}\right)[(1-p)(1-\gamma)]^{n-1}$

gives the equation

$t_{n}=t_{n-1}(1-\gamma)+\left(p\gamma-\pi_{(1,0,0)}\right)[(1-p)(1-\gamma)]^{n-1}$
(197)

Using (197) repeatedly, we have

$\displaystyle t_{n}$
$\displaystyle=t_{1}(1-\gamma)^{n-1}+\left(p\gamma-\pi_{(1,0,0)}\right)\left(\sum\nolimits_{j=0}^{n-2}(1-\gamma)^{j}[(1-p)(1-\gamma)]^{n-1-j}\right)$
$\displaystyle=t_{1}(1-\gamma)^{n-1}+\left(p\gamma-\pi_{(1,0,0)}\right)\frac{1-p}{p}\left\\{(1-\gamma)^{n-1}-[(1-p)(1-\gamma)]^{n-1}\right\\}$
$\displaystyle=t_{1}(1-\gamma)^{n-1}+\frac{p\gamma(1-p)(1-\gamma)(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left\\{(1-\gamma)^{n-1}-[(1-p)(1-\gamma)]^{n-1}\right\\}$
(198)

Let $n=1$, equation (198) reduces to $t_{1}=t_{1}$, thus (198) is valid for
$n\geq 1$.

Since

$\displaystyle\sum\nolimits_{n=1}^{\infty}t_{n}$
$\displaystyle=\sum\nolimits_{n=1}^{\infty}\left[t_{1}(1-\gamma)^{n-1}+\frac{p\gamma(1-p)(1-\gamma)(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left\\{(1-\gamma)^{n-1}-[(1-p)(1-\gamma)]^{n-1}\right\\}\right]$
$\displaystyle=\frac{t_{1}}{\gamma}+\frac{p\gamma(1-p)(1-\gamma)(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left(\frac{1}{\gamma}-\frac{1}{p+\gamma-p\gamma}\right)$
$\displaystyle=\frac{t_{1}}{\gamma}+\frac{p^{2}(1-p)(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{2}-p\gamma}$
(199) $\displaystyle=1-t_{0}$
$\displaystyle=1-\sum\nolimits_{k=1}^{\infty}\pi_{(k,0,0)}$
$\displaystyle=1-\frac{\pi_{(1,0,0)}}{p\gamma}$
$\displaystyle=\frac{p(1-\gamma)(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}$
(200)

From equations (199) and (200), we can solve that

$t_{1}=\frac{p\gamma(1-\gamma)\left[(p+\gamma-p\gamma)-p(1-p)(1-\gamma)\right]}{(p+\gamma-p\gamma)^{2}-p\gamma}$
(201)

Substituting (201) into equaiton (198) obtains

$t_{n}=\frac{p\gamma(1-\gamma)(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}(1-\gamma)^{n-1}-\frac{p\gamma(1-p)(1-\gamma)(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}[(1-p)(1-\gamma)]^{n-1}\qquad(n\geq
1)$ (202)

Combining results (195) and (202), we finally determine the sums $t_{n}$,
$n\geq 0$. This completes the proof of Lemma 5.

##  Appendix H Proof of Theorem 14

In this appendix, we calculate the probability expression of the AoI and
determine the stationary AoI-distribution for the system with Ber/Geo/1/2*
queue.

First of all, when $n=1$ we have

$\Pr\\{\Delta=1\\}=\pi_{(1,0,0)}=\frac{p(1-p)\gamma^{3}}{(p+\gamma-p\gamma)^{2}-p\gamma}$
(203)

In general, the probability that the AoI takes value $n$ is obtained by
collecting all the stationary probabilities having $n$ as first age-component.
Therefore, it shows that

$\Pr\\{\Delta=n\\}=\pi_{(n,0,0)}+\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}+\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n,m,l)}$
(204)

Substituting equation (86), we compute the first sum in (204) as follows.

$\displaystyle\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}$
$\displaystyle=\sum\nolimits_{m=1}^{n-1}\bigg{\\{}p^{2}(1-p)^{n-2}[(1-\gamma)^{m}-(1-\gamma)^{n}]+\frac{(p\gamma)^{2}(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}m(1-p)^{m-1}(1-\gamma)^{n-1}$
$\displaystyle\qquad-\frac{p^{2}\gamma(1-\gamma)(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}[p(1-\gamma)n+(\gamma-p)m][(1-p)(1-\gamma)]^{n-2}\bigg{\\}}$
$\displaystyle=p^{2}(1-p)^{n-2}\left\\{\frac{(1-\gamma)-(1-\gamma)^{n}}{\gamma}-(n-1)(1-\gamma)^{n}\right\\}$
$\displaystyle\qquad+\frac{(p\gamma)^{2}(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left\\{\frac{1-(1-p)^{n-1}-p(n-1)(1-p)^{n-1}}{p^{2}}\right\\}(1-\gamma)^{n-1}$
$\displaystyle\qquad-\frac{p^{2}\gamma(1-\gamma)(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left[p(1-\gamma)n(n-1)+(\gamma-p)\frac{n(n-1)}{2}\right][(1-p)(1-\gamma)]^{n-2}$
$\displaystyle=\frac{p^{2}(1-\gamma)}{(1-p)\gamma}\left\\{(1-p)^{n-1}-[1+(n-1)\gamma][(1-p)(1-\gamma)]^{n-1}\right\\}$
$\displaystyle\qquad+\frac{(p+2\gamma-2p\gamma)\gamma^{2}}{(p+\gamma-p\gamma)^{2}-p\gamma}\left\\{(1-\gamma)^{n-1}-[1+(n-1)p][(1-p)(1-\gamma)]^{n-1}\right\\}$
$\displaystyle\qquad-\frac{p^{2}\gamma(1-\gamma)(p+\gamma-p\gamma)(p+\gamma-2p\gamma)}{2[(p+\gamma-p\gamma)^{2}-p\gamma]}n(n-1)[(1-p)(1-\gamma)]^{n-2}$
(205)

which gives equation (98).

In order to verify that the sum of all the stationary probabilities is 1, here
we continue to calculate the sum of (205) over the range $n\geq 2$. During the
following calculations, the formulas

$\sum\nolimits_{n=1}^{\infty}nx^{n-1}=\frac{1}{(1-x)^{2}},\qquad\sum\nolimits_{n=2}^{\infty}n(n-1)x^{n-2}=\frac{2}{(1-x)^{3}}\qquad(0<x<1)$
(206)

will be used multiple times. We show that

$\displaystyle\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}$
$\displaystyle={}$
$\displaystyle\frac{p^{2}(1-\gamma)}{(1-p)\gamma}\left(\frac{1-p}{p}-\frac{(1-p)(1-\gamma)}{p+\gamma-p\gamma}-\frac{\gamma(1-p)(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\right)$
$\displaystyle+\frac{(p+2\gamma-2p\gamma)\gamma^{2}}{(p+\gamma-p\gamma)^{2}-p\gamma}\left(\frac{1-\gamma}{\gamma}-\frac{(1-p)(1-\gamma)}{p+\gamma-p\gamma}-\frac{p(1-p)(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\right)$
$\displaystyle-\frac{p^{2}\gamma(1-\gamma)(p+\gamma-p\gamma)(p+\gamma-2p\gamma)}{2[(p+\gamma-p\gamma)^{2}-p\gamma]}\frac{2}{(p+\gamma-p\gamma)^{3}}$
$\displaystyle={}$
$\displaystyle\frac{p^{2}(1-\gamma)}{(1-p)\gamma}\frac{(1-p)\gamma^{2}}{p(p+\gamma-p\gamma)^{2}}+\frac{(p+2\gamma-2p\gamma)\gamma^{2}}{(p+\gamma-p\gamma)^{2}-p\gamma}\frac{p^{2}(1-\gamma)}{\gamma(p+\gamma-p\gamma)^{2}}-\frac{p^{2}\gamma(1-\gamma)(p+\gamma-2p\gamma)}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)^{2}}$
$\displaystyle={}$
$\displaystyle\frac{p\gamma(1-\gamma)}{(p+\gamma-p\gamma)^{2}}+\frac{p^{2}\gamma(1-\gamma)(p+2\gamma-2p\gamma)}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)^{2}}-\frac{p^{2}\gamma(1-\gamma)(p+\gamma-2p\gamma)}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)^{2}}$
$\displaystyle={}$
$\displaystyle\frac{p\gamma(1-\gamma)}{(p+\gamma-p\gamma)^{2}}+\frac{p^{2}\gamma^{2}(1-\gamma)}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)^{2}}$
$\displaystyle={}$
$\displaystyle\frac{p\gamma(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\left\\{1+\frac{p\gamma}{(p+\gamma-p\gamma)^{2}-p\gamma}\right\\}$
$\displaystyle={}$
$\displaystyle\frac{p\gamma(1-\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}$ (207)

Therefore, we obtain the partial sum of the stationary probabilities. Next,
the latter sum in (204) is computed. Using probability expression (96), we
have

$\displaystyle\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n,m,l)}$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\bigg{\\{}p^{3}(1-p)^{n-m+l-2}(1-\gamma)^{m}\left[1-(1-\gamma)^{n-m}\right]$
$\displaystyle\quad-\frac{p^{2}\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left[p^{2}(1-\gamma)(n-m)+(1-p)\gamma\right](1-p)^{n-m+l-2}(1-\gamma)^{n-1}$
$\displaystyle\quad+\frac{(p\gamma)^{2}(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}(1-p)^{l-1}(1-\gamma)^{n-1}\left[1-(1-p)^{m-l}\right]+\frac{(p\gamma)^{2}(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}[(1-p)(1-\gamma)]^{n-1}\bigg{\\}}$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{m=2}^{n-1}\bigg{(}p^{2}\left\\{(1-p)^{n-m-1}-(1-p)^{n-2}\right\\}\left\\{(1-\gamma)^{m}-(1-\gamma)^{n}\right\\}$
$\displaystyle\quad-\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left[p^{2}(1-\gamma)(n-m)+(1-p)\gamma\right]\left\\{(1-p)^{n-m-1}(1-\gamma)^{n-1}-(1-p)^{n-2}(1-\gamma)^{n-1}\right\\}$
$\displaystyle\quad+\frac{p\gamma^{2}(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left\\{(1-\gamma)^{n-1}-(1-p)^{m-1}(1-\gamma)^{n-1}-p(m-1)(1-p)^{m-1}(1-\gamma)^{n-1}\right\\}$
$\displaystyle\quad+\frac{(p\gamma)^{2}(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}(m-1)[(1-p)(1-\gamma)]^{n-1}\bigg{)}$
(208)

in which

$\displaystyle S_{1}$
$\displaystyle=\sum\nolimits_{m=2}^{n-1}p^{2}\left\\{(1-p)^{n-m-1}-(1-p)^{n-2}\right\\}\left\\{(1-\gamma)^{m}-(1-\gamma)^{n}\right\\}$
$\displaystyle=p^{2}\sum\nolimits_{m=2}^{n-1}\left\\{(1-p)^{n-m-1}(1-\gamma)^{m}-(1-p)^{n-m-1}(1-\gamma)^{n}-(1-p)^{n-2}(1-\gamma)^{m}+(1-p)^{n-2}(1-\gamma)^{n}\right\\}$
$\displaystyle=p^{2}\bigg{(}\frac{(1-\gamma)^{2}}{\gamma-p}\left[(1-p)^{n-2}-(1-\gamma)^{n-2}\right]-\frac{(1-\gamma)^{2}}{p}\left[(1-\gamma)^{n-2}-[(1-p)(1-\gamma)]^{n-2}\right]$
$\displaystyle\qquad-\frac{(1-\gamma)^{2}}{\gamma}\left[(1-p)^{n-2}-[(1-p)(1-\gamma)]^{n-2}\right]+(1-\gamma)^{2}(n-2)[(1-p)(1-\gamma)]^{2}\bigg{)}$
$\displaystyle=p^{2}(1-\gamma)^{2}\left\\{\frac{p}{\gamma(\gamma-p)}(1-p)^{n-2}-\frac{\gamma}{p(\gamma-p)}(1-\gamma)^{n-2}+\left[\frac{1}{p}+\frac{1}{\gamma}+(n-2)\right][(1-p)(1-\gamma)]^{n-2}\right\\}$
(209)

and

$\displaystyle S_{2}$
$\displaystyle=\sum\nolimits_{m=2}^{n-1}\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left[p^{2}(1-\gamma)(n-m)+(1-p)\gamma\right]\left\\{(1-p)^{n-m-1}(1-\gamma)^{n-1}-(1-p)^{n-2}(1-\gamma)^{n-1}\right\\}$
$\displaystyle=\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left(\sum\nolimits_{m=2}^{n-1}\left[p^{2}(1-\gamma)(n-m)+(1-p)\gamma\right](1-p)^{n-m-1}\right)(1-\gamma)^{n-1}$
$\displaystyle\qquad-\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left\\{p^{2}(1-\gamma)^{2}\frac{(n-1)(n-2)}{2}[(1-p)(1-\gamma)]^{n-2}+\gamma(n-2)[(1-p)(1-\gamma)]^{n-1}\right\\}$
(210)

Let $n-m=j$, it shows that

$\displaystyle\sum\nolimits_{m=2}^{n-1}\left[p^{2}(1-\gamma)(n-m)+(1-p)\gamma\right](1-p)^{n-m-1}$
$\displaystyle={}$
$\displaystyle\sum\nolimits_{j=1}^{n-2}\left[p^{2}(1-\gamma)j+(1-p)\gamma\right](1-p)^{j-1}$
$\displaystyle={}$ $\displaystyle
p^{2}(1-\gamma)\frac{1-(1-p)^{n-2}-p(n-2)(1-p)^{n-2}}{p^{2}}+\gamma\frac{(1-p)-(1-p)^{n-1}}{p}$
$\displaystyle={}$
$\displaystyle\frac{p(1-\gamma)-p(1-\gamma)(1-p)^{n-2}-p^{2}(1-\gamma)(n-2)(1-p)^{n-2}+(1-p)\gamma-(1-p)\gamma(1-p)^{n-2}}{p}$
$\displaystyle={}$
$\displaystyle\frac{p+\gamma-2p\gamma}{p}-\frac{p+\gamma-2p\gamma}{p}(1-p)^{n-2}-p(1-\gamma)(n-2)(1-p)^{n-2}$
(211)

Therefore, we obtain that

$\displaystyle S_{2}$
$\displaystyle=\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\bigg{\\{}\frac{(p+\gamma-2p\gamma)(1-\gamma)}{p}(1-\gamma)^{n-2}$
$\displaystyle\qquad-\frac{(p+\gamma-2p\gamma)(1-\gamma)}{p}[(1-p)(1-\gamma)]^{n-2}-p(1-\gamma)^{2}(n-2)[(1-p)(1-\gamma)]^{n-2}\bigg{\\}}$
$\displaystyle\qquad-\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left\\{p^{2}(1-\gamma)^{2}\frac{(n-1)(n-2)}{2}[(1-p)(1-\gamma)]^{n-2}+\gamma(n-2)[(1-p)(1-\gamma)]^{n-1}\right\\}$
$\displaystyle=\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\bigg{(}\frac{(p+\gamma-2p\gamma)(1-\gamma)}{p}\bigg{\\{}(1-\gamma)^{n-2}-[(1-p)(1-\gamma)]^{n-2}$
$\displaystyle\qquad-p(n-2)[(1-p)(1-p\gamma)]^{n-2}\bigg{\\}}-\frac{p^{2}(1-\gamma)^{2}}{2}(n-1)(n-2)[(1-p)(1-\gamma)]^{n-2}\bigg{)}$
(212)

At last, the rest part of equation (208) is dealt with together.

$\displaystyle S_{3}$
$\displaystyle=\frac{p\gamma^{2}(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\sum\nolimits_{m=2}^{n-1}\bigg{\\{}(1-\gamma)^{n-1}-(1-p)^{m-1}(1-\gamma)^{n-1}-p(m-1)(1-p)^{m-1}(1-\gamma)^{n-1}\bigg{\\}}$
$\displaystyle\qquad+\frac{(p\gamma)^{2}(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\frac{(n-1)(n-2)}{2}[(1-p)(1-\gamma)]^{n-1}$
$\displaystyle=\frac{p\gamma^{2}(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\bigg{\\{}(n-2)(1-\gamma)^{n-1}-\frac{(1-p)(1-\gamma)^{n-1}-[(1-p)(1-\gamma)]^{n-1}}{p}$
$\displaystyle\qquad-\frac{(1-p)(1-\gamma)^{n-1}-[(1-p)(1-\gamma)]^{n-1}}{p}+(n-2)[(1-p)(1-\gamma)]^{n-2}\bigg{\\}}$
$\displaystyle\qquad+\frac{(p\gamma)^{2}(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\frac{(n-1)(n-2)}{2}[(1-p)(1-\gamma)]^{n-1}$
$\displaystyle=\frac{\gamma^{2}(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\bigg{\\{}p(n-2)(1-\gamma)^{n-1}-2(1-p)(1-\gamma)^{n-1}+2[(1-p)(1-\gamma)]^{n-1}+p(n-2)[(1-p)(1-\gamma)]^{n-1}\bigg{\\}}$
$\displaystyle\qquad+\frac{(p\gamma)^{2}(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\frac{(n-1)(n-2)}{2}[(1-p)(1-\gamma)]^{n-1}$
(213)

Collecting results (209), (212), and (213), eventually we have that

$\displaystyle\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n,m,l)}=S_{1}-S_{2}+S_{3}$
$\displaystyle={}$ $\displaystyle
p^{2}(1-\gamma)^{2}\left\\{\frac{p}{\gamma(\gamma-p)}(1-p)^{n-2}-\frac{\gamma}{p(\gamma-p)}(1-\gamma)^{n-2}+\left[\frac{1}{p}+\frac{1}{\gamma}+(n-2)\right][(1-p)(1-\gamma)]^{n-2}\right\\}$
$\displaystyle-\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\bigg{(}\frac{(p+\gamma-2p\gamma)(1-\gamma)}{p}\bigg{\\{}(1-\gamma)^{n-2}-[(1-p)(1-\gamma)]^{n-2}$
$\displaystyle\qquad-p(n-2)[(1-p)(1-p\gamma)]^{n-2}\bigg{\\}}-\frac{p^{2}(1-\gamma)^{2}}{2}(n-1)(n-2)[(1-p)(1-\gamma)]^{n-2}\bigg{)}$
$\displaystyle+\frac{\gamma^{2}(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\bigg{\\{}p(n-2)(1-\gamma)^{n-1}-2(1-p)(1-\gamma)^{n-1}+2[(1-p)(1-\gamma)]^{n-1}+p(n-2)[(1-p)(1-\gamma)]^{n-1}\bigg{\\}}$
$\displaystyle\qquad+\frac{(p\gamma)^{2}(p+\gamma-p\gamma)}{2[(p+\gamma-p\gamma)^{2}-p\gamma]}(n-1)(n-2)[(1-p)(1-\gamma)]^{n-1}$
(214)

which is exactly the probability expression (99).

It can be directly verified that (214) is zero when we let $n=2$. Thus,
equation (204) is actually valid for $n\geq 2$. Finally, combining (203),
(84), (205) and (214), we obtain the distribution of AoI and complete the
proof of Theorem 14.

To ensure that equation (97) is indeed a proper probability distribution, we
have to check

$\sum\nolimits_{n=1}^{\infty}\Pr\\{\Delta=n\\}=\sum\nolimits_{n=1}^{\infty}\pi_{(n,0,0)}+\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}+\sum\nolimits_{n=3}^{\infty}\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n,m,l)}=1$

where

$\sum\nolimits_{n=1}^{\infty}\pi_{(n,0,0)}=\frac{\pi_{(1,0,0)}}{p\gamma}=\frac{(1-p)\gamma^{2}}{(p+\gamma-p\gamma)^{2}-p\gamma}$

and we have obtained in (207) that

$\sum\nolimits_{n=2}^{\infty}\sum\nolimits_{m=1}^{n-1}\pi_{(n,m,0)}=\frac{p\gamma(1-\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}$
(215)

Therefore, to prove the claim, it suffices to show that

$\displaystyle\sum\nolimits_{n=3}^{\infty}\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n,m,l)}$
$\displaystyle=\sum\nolimits_{n=3}^{\infty}\left(S_{1}-S_{2}+S_{3}\right)$
$\displaystyle=1-\frac{(1-p)\gamma^{2}}{(p+\gamma-p\gamma)^{2}-p\gamma}-\frac{p\gamma(1-\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}$
$\displaystyle=1-\frac{\gamma(p+\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}$
(216)

Since

$\displaystyle\gamma(p+\gamma-2p\gamma)$
$\displaystyle=[\gamma+(p-p\gamma)-(p-p\gamma)][(p+\gamma-p\gamma)-p\gamma]$
$\displaystyle=[(p+\gamma-p\gamma)-p(1-\gamma)][(p+\gamma-p\gamma)-p\gamma]$
$\displaystyle=(p+\gamma-p\gamma)^{2}-p(p+\gamma-p\gamma)+p^{2}\gamma(1-\gamma)$
$\displaystyle=(p+\gamma-p\gamma)^{2}-p\gamma-p^{2}(1-\gamma)+p^{2}\gamma(1-\gamma)$
$\displaystyle=(p+\gamma-p\gamma)^{2}-p\gamma-p^{2}(1-\gamma)^{2}$ (217)

such that we have to verify that

$\sum\nolimits_{n=3}^{\infty}\sum\nolimits_{m=2}^{n-1}\sum\nolimits_{l=1}^{m-1}\pi_{(n,m,l)}=\sum\nolimits_{n=3}^{\infty}\left(S_{1}-S_{2}+S_{3}\right)=\frac{p^{2}(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{2}-p\gamma}$
(218)

Firstly,

$\displaystyle\sum\nolimits_{n=3}^{\infty}S_{1}$
$\displaystyle=\sum\nolimits_{n=3}^{\infty}p^{2}(1-\gamma)^{2}\left\\{\frac{p}{\gamma(\gamma-p)}(1-p)^{n-2}-\frac{\gamma}{p(\gamma-p)}(1-\gamma)^{n-2}+\left[\frac{1}{p}+\frac{1}{\gamma}+(n-2)\right][(1-p)(1-\gamma)]^{n-2}\right\\}$
$\displaystyle=p^{2}(1-\gamma)^{2}\left\\{\frac{p}{\gamma(\gamma-p)}\frac{1-p}{p}-\frac{\gamma}{p(\gamma-p)}\frac{1-\gamma}{\gamma}+\left(\frac{1}{p}+\frac{1}{\gamma}\right)\frac{(1-p)(1-\gamma)}{p+\gamma-p\gamma}+\frac{(1-p)(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\right\\}$
$\displaystyle=p^{2}(1-\gamma)^{2}\left\\{\frac{1-p}{\gamma(\gamma-p)}-\frac{1-\gamma}{p(\gamma-p)}+\frac{(p+\gamma)(1-p)(1-\gamma)}{p\gamma(p+\gamma-p\gamma)}+\frac{(1-p)(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\right\\}$
$\displaystyle=p^{2}(1-\gamma)^{2}\left\\{\frac{p+\gamma-1}{p\gamma}+\frac{(p+\gamma)(1-p)(1-\gamma)}{p\gamma(p+\gamma-p\gamma)}+\frac{(1-p)(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\right\\}$
$\displaystyle=p^{2}(1-\gamma)^{2}\left\\{\frac{(p+\gamma-1)(p+\gamma-p\gamma)+(p+\gamma)(1-p)(1-\gamma)}{p\gamma(p+\gamma-p\gamma)}+\frac{(1-p)(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\right\\}$
(219)

where

$\displaystyle(p+\gamma-1)(p+\gamma-p\gamma)+(p+\gamma)(1-p)(1-\gamma)$
$\displaystyle={}$
$\displaystyle(p+\gamma)(p+\gamma-p\gamma)-(p+\gamma-p\gamma)+(p+\gamma)(1-p)(1-\gamma)$
$\displaystyle={}$ $\displaystyle(p+\gamma)-(p+\gamma-p\gamma)$ (220)
$\displaystyle={}$ $\displaystyle p\gamma$

To obtain equation (220), notice that $p+\gamma-p\gamma+(1-p)(1-\gamma)=1$.
Thus,

$\sum\nolimits_{n=3}^{\infty}S_{1}=p^{2}(1-\gamma)^{2}\left\\{\frac{1}{p+\gamma-p\gamma}+\frac{(1-p)(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\right\\}=\frac{p^{2}(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{2}}$
(221)

Next, the second part of the entire sum (218) is handled.

$\displaystyle\sum\nolimits_{n=3}^{\infty}S_{2}$ $\displaystyle={}$
$\displaystyle\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\sum\nolimits_{n=3}^{\infty}\bigg{(}\frac{(p+\gamma-2p\gamma)(1-\gamma)}{p}\bigg{\\{}(1-\gamma)^{n-2}-[(1-p)(1-\gamma)]^{n-2}-p(n-2)[(1-p)(1-p\gamma)]^{n-2}\bigg{\\}}$
$\displaystyle\qquad-\frac{p^{2}(1-\gamma)^{2}}{2}(n-1)(n-2)[(1-p)(1-\gamma)]^{n-2}\bigg{)}$
$\displaystyle={}$
$\displaystyle\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\bigg{(}\frac{(p+\gamma-2p\gamma)(1-\gamma)}{p}\bigg{\\{}\frac{1-\gamma}{\gamma}-\frac{(1-p)(1-\gamma)}{p+\gamma-p\gamma}-\frac{p(1-p)(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\bigg{\\}}$
$\displaystyle-\frac{p^{2}(1-p)(1-\gamma)^{3}}{2}\frac{2}{(p+\gamma-p\gamma)^{3}}\bigg{)}$
(222)

in which

$\displaystyle\frac{(p+\gamma-2p\gamma)(1-\gamma)}{p}\bigg{\\{}\frac{1-\gamma}{\gamma}-\frac{(1-p)(1-\gamma)}{p+\gamma-p\gamma}-\frac{p(1-p)(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\bigg{\\}}$
$\displaystyle={}$
$\displaystyle\frac{(p+\gamma-2p\gamma)(1-\gamma)}{p}\bigg{\\{}(1-\gamma)\left[\frac{1}{\gamma}-\frac{1-p}{p+\gamma-p\gamma}\right]-\frac{p(1-p)(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\bigg{\\}}$
$\displaystyle={}$
$\displaystyle\frac{(p+\gamma-2p\gamma)(1-\gamma)}{p}\bigg{\\{}\frac{p(1-\gamma)}{\gamma(p+\gamma-p\gamma)}-\frac{p(1-p)(1-\gamma)}{(p+\gamma-p\gamma)^{2}}\bigg{\\}}$
$\displaystyle={}$
$\displaystyle\frac{(p+\gamma-2p\gamma)(1-\gamma)}{p}\frac{p(1-\gamma)}{(p+\gamma-p\gamma)}\left[\frac{1}{\gamma}-\frac{1-p}{p+\gamma-p\gamma}\right]$
$\displaystyle={}$
$\displaystyle\frac{p(1-\gamma)^{2}(p+\gamma-2p\gamma)}{\gamma(p+\gamma-p\gamma)^{2}}$
(223)

Therefore, we can write

$\displaystyle\sum\nolimits_{n=3}^{\infty}S_{2}$
$\displaystyle=\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left(\frac{p(1-\gamma)^{2}(p+\gamma-2p\gamma)}{\gamma(p+\gamma-p\gamma)^{2}}-\frac{p^{2}(1-p)(1-\gamma)^{3}}{(p+\gamma-p\gamma)^{3}}\right)$
$\displaystyle=\frac{p\gamma(p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\frac{p(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{2}}\left[\frac{p+\gamma-2p\gamma}{\gamma}-\frac{p(1-p)(1-\gamma)}{p+\gamma-p\gamma}\right]$
$\displaystyle=\frac{p^{2}\gamma(1-\gamma)^{2}}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)}\frac{(p+\gamma-p\gamma)^{2}-p\gamma}{\gamma(p+\gamma-p\gamma)}$
$\displaystyle=\frac{p^{2}(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{2}}$ (224)

The last part is calculated as follows.

$\displaystyle\sum\nolimits_{n=3}^{\infty}S_{3}$
$\displaystyle=\frac{\gamma^{2}(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\sum\nolimits_{n=3}^{\infty}\bigg{\\{}p(n-2)(1-\gamma)^{n-1}-2(1-p)(1-\gamma)^{n-1}+2[(1-p)(1-\gamma)]^{n-1}$
$\displaystyle\qquad+p(n-2)[(1-p)(1-\gamma)]^{n-1}\bigg{\\}}+\frac{(p\gamma)^{2}(p+\gamma-p\gamma)}{2[(p+\gamma-p\gamma)^{2}-p\gamma]}\sum\nolimits_{n=3}^{\infty}(n-1)(n-2)[(1-p)(1-\gamma)]^{n-1}$
$\displaystyle=\frac{\gamma^{2}(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\left\\{\frac{p(1-\gamma)^{2}}{\gamma^{2}}-\frac{2(1-p)(1-\gamma)^{2}}{\gamma}+\frac{2(1-p)^{2}(1-\gamma)^{2}}{p+\gamma-p\gamma}+\frac{p(1-p)^{2}(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{2}}\right\\}$
$\displaystyle\qquad+\frac{(p\gamma)^{2}(p+\gamma-p\gamma)}{2[(p+\gamma-p\gamma)^{2}-p\gamma]}\frac{2(1-p)^{2}(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{3}}$
(225)

where

$\displaystyle\frac{p(1-\gamma)^{2}}{\gamma^{2}}-\frac{2(1-p)(1-\gamma)^{2}}{\gamma}+\frac{2(1-p)^{2}(1-\gamma)^{2}}{p+\gamma-p\gamma}+\frac{p(1-p)^{2}(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{2}}$
$\displaystyle={}$
$\displaystyle\frac{p(1-\gamma)^{2}}{\gamma^{2}}-2(1-p)(1-\gamma)^{2}\left[\frac{1}{\gamma}-\frac{1-p}{p+\gamma-p\gamma}\right]+\frac{p(1-p)^{2}(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{2}}$
$\displaystyle={}$
$\displaystyle\frac{p(1-\gamma)^{2}}{\gamma^{2}}-\frac{2p(1-p)(1-\gamma)^{2}}{\gamma(p+\gamma-p\gamma)}+\frac{p(1-p)^{2}(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{2}}$
$\displaystyle={}$
$\displaystyle\frac{p(1-\gamma)^{2}}{\gamma^{2}}-\frac{p(1-p)(1-\gamma)^{2}}{p+\gamma-p\gamma}\left[\frac{2}{\gamma}-\frac{1-p}{p+\gamma-p\gamma}\right]$
$\displaystyle={}$
$\displaystyle\frac{p(1-\gamma)^{2}}{\gamma^{2}}-\frac{p(1-p)(1-\gamma)^{2}}{p+\gamma-p\gamma}\frac{2p+\gamma-p\gamma}{\gamma(p+\gamma-p\gamma)}$
$\displaystyle={}$
$\displaystyle\frac{p(1-\gamma)^{2}}{\gamma}\left[\frac{1}{\gamma}-\frac{(1-p)(2p+\gamma-p\gamma)}{(p+\gamma-p\gamma)^{2}}\right]$
$\displaystyle={}$
$\displaystyle\frac{p(1-\gamma)^{2}}{\gamma}\frac{(p+\gamma-p\gamma)^{2}-(1-p)\gamma(2p+\gamma-p\gamma)}{\gamma(p+\gamma-p\gamma)^{2}}$
$\displaystyle={}$
$\displaystyle\frac{p(1-\gamma)^{2}}{\gamma}\frac{(p+\gamma-p\gamma)^{2}-[(p+\gamma-p\gamma)-p][(p+\gamma-p\gamma)+p]}{\gamma(p+\gamma-p\gamma)^{2}}$
$\displaystyle={}$
$\displaystyle\frac{p(1-\gamma)^{2}}{\gamma}\frac{(p+\gamma-p\gamma)^{2}-[(p+\gamma-p\gamma)^{2}-p^{2}]}{\gamma(p+\gamma-p\gamma)^{2}}$
$\displaystyle={}$
$\displaystyle\frac{p^{3}(1-\gamma)^{2}}{\gamma^{2}(p+\gamma-p\gamma)^{2}}$
(226)

such that we obtain that

$\displaystyle\sum\nolimits_{n=3}^{\infty}S_{3}$
$\displaystyle=\frac{\gamma^{2}(p+2\gamma-2p\gamma)}{(p+\gamma-p\gamma)^{2}-p\gamma}\frac{p^{3}(1-\gamma)^{2}}{\gamma^{2}(p+\gamma-p\gamma)^{2}}+\frac{(p\gamma)^{2}(1-p)^{2}(1-\gamma)^{2}}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)^{2}}$
$\displaystyle=\frac{p^{3}(1-\gamma)^{2}(p+2\gamma-2p\gamma)}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)^{2}}+\frac{(p\gamma)^{2}(1-p)^{2}(1-\gamma)^{2}}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)^{2}}$
$\displaystyle=\frac{p^{2}(1-\gamma)^{2}}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)^{2}}\bigg{\\{}p(p+2\gamma-2p\gamma)+(1-p)^{2}\gamma^{2}\bigg{\\}}$
$\displaystyle=\frac{p^{2}(1-\gamma)^{2}}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)^{2}}\bigg{\\{}[(p+\gamma-p\gamma)-(1-p)\gamma][(p+\gamma-p\gamma)+(1-p)\gamma]+(1-p)^{2}\gamma^{2}\bigg{\\}}$
$\displaystyle=\frac{p^{2}(1-\gamma)^{2}}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)^{2}}\bigg{\\{}(p+\gamma-p\gamma)^{2}-(1-p)^{2}\gamma^{2}+(1-p)^{2}\gamma^{2}\bigg{\\}}$
$\displaystyle=\frac{p^{2}(1-\gamma)^{2}}{[(p+\gamma-p\gamma)^{2}-p\gamma](p+\gamma-p\gamma)^{2}}(p+\gamma-p\gamma)^{2}$
$\displaystyle=\frac{p^{2}(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{2}-p\gamma}$
(227)

Combining equations (221), (224), and (227), it was observed that

$\sum\nolimits_{n=3}^{\infty}\left(S_{1}-S_{2}+S_{3}\right)=\frac{p^{2}(1-\gamma)^{2}}{(p+\gamma-p\gamma)^{2}-p\gamma}$
(228)

which satisfies (218), such that all the stationary probabilities indeed add
up to 1. This completes this appendix.

## References

  * [1] S. Kaul, M. Gruteser, V. Rai, and J. Kenney, “Minimizing age of information in vehicular networks,” _8th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks_ , pp. 350–358, 2011.
  * [2] R. D. Yates, Y. Sun, D. R. B. III, S. K. Kaul, E. Modiano, and S. Ulukus, “Age of information: An introduction and survey,” _IEEE Journal on Selected Areas in Communications_ , pp. 1–1, 2021.
  * [3] S. Kaul, R. Yates, and M. Gruteser, “Real-time status: How often should one update?” _2012 Proceedings IEEE INFOCOM_ , pp. 2731–2735, 2012.
  * [4] ——, “Status updates through queues,” _46th Annual Conference on Information Sciences and Systems (CISS)_ , 2012.
  * [5] J. P. Hespanha, “Modeling and analysis of stochastic hybrid systems,” _IEE Proceedings - Control Theory and Applications_ , vol. 153, no. 5, pp. 520–535, 2007.
  * [6] R. D. Yates and S. Kaul, “The age of information: Real-time status updating by multiple sources,” _IEEE Transactions on Information Theory_ , vol. 65, no. 3, pp. 1807–1827, 2019.
  * [7] M. Moltafet, M. Leinonen, and M. Codreanu, “Moment generating function of the aoi in a two-source system with packet management,” _IEEE Wireless Communications Letters_ , pp. 1–1, 2020.
  * [8] O. Dogan and N. Akar, “The multi-source preemptive m/ph/1/1 queue with packet errors: Exact distribution of the age of information and its peak,” _arXiv:2007.11656v1_ , 2020.
  * [9] S. K. Kaul and R. D. Yates, “Timely updates by multiple sources: The m/m/1 queue revisited,” _2020 54th Annual Conference on Information Sciences and Systems (CISS)_ , pp. 1–6, 2020.
  * [10] R. D. Yates and S. K. Kaul, “Age of information in uncoordinated unslotted updating,” _2020 IEEE International Symposium on Information Theory (ISIT)_ , pp. 1759–1764, 2020.
  * [11] M. Moltafet, M. Leinonen, and M. Codreanu, “Average aoi in multi-source systems with source-aware packet management,” _IEEE Transactions on Communications_ , vol. 69, no. 2, pp. 1121–1133, 2021.
  * [12] A. Maatouk, M. Assaad, and A. Ephremides, “Minimizing the age of information: Noma or oma?” _IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)_ , pp. 102–108, 2019.
  * [13] R. D. Yates, “The age of information in networks: Moments, distributions, and sampling,” _IEEE Transactions on Information Theory_ , vol. 66, no. 9, pp. 5712–5728, 2020.
  * [14] C. Kam, S. Kompella, G. D. Nguyen, and A. Ephremides, “Effect of message transmission path diversity on status age,” _IEEE Transactions on Information Theory_ , vol. 62, no. 3, pp. 1360–1374, 2016.
  * [15] M. Costa, M. Codreanu, and A. Ephremides, “Age of information with packet management,” _2014 IEEE International Symposium on Information Theory_ , pp. 1583–1587, 2014.
  * [16] M. A. Abd-Elmagid, H. S. Dhillon, and N. Pappas, “A reinforcement learning framework for optimizing age of information in rf-powered communication systems,” _IEEE Transactions on Communications_ , vol. 68, no. 8, pp. 4747–4760, 2020.
  * [17] S. Kriouile and M. Assaad, “Minimizing the age of incorrect information for real-time tracking of markov remote sources,” _arXiv:2102.03245_ , 2021.
  * [18] V. Tripathi and E. Modiano, “Age debt: A general framework for minimizing age of information,” _arXiv:2101.10225_ , 2021.
  * [19] H. Tang, P. Ciblat, J. Wang, M. Wigger, and R. D. Yates, “Cache updating strategy minimizing the age of information with time-varying files’ popularities,” _arXiv:2010.04787_ , 2021.
  * [20] M. Moltafet, M. Leinonen, M. Codreanu, and N. Pappas, “Power minimization for age of information constrained dynamic control in wireless sensor networks,” _arXiv:2007.05364_ , 2021.
  * [21] A. Maatouk, S. Kriouile, M. Assad, and A. Ephremides, “On the optimality of the whittle’s index policy for minimizing the age of information,” _IEEE Transactions on Wireless Communications_ , vol. 20, no. 2, pp. 1263–1277, 2021.
  * [22] H. Tang, J. Wang, L. Song, and J. Song, “Minimizing age of information with power constraints: Multi-user opportunistic scheduling in multi-state time-varying channels,” _IEEE Journal on Selected Areas in Communications_ , vol. 38, no. 5, pp. 854–868, 2020.
  * [23] H. Tang, J. Wang, Z. Tang, and J. Song, “Scheduling to minimize age of synchronization in wireless broadcast networks with random updates,” _IEEE Transactions on Wireless Communications_ , vol. 19, no. 6, pp. 4023–4037, 2020.
  * [24] M. K. Abdel-Aziz, S. Samarakoon, C.-F. Liu, M. Bennis, and W. Saad, “Optimized age of information tail for ultra-reliable low-latency communications in vehicular networks,” _IEEE Transactions on Communications_ , vol. 68, no. 3, pp. 1911–1924, 2020.
  * [25] J. P. Champati, H. Al-Zubaidy, and J. Gross, “Statistical guarantee optimization for age of information for the d/g/1 queue,” _IEEE INFOCOM 2018 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)_ , pp. 130–135, 2018.
  * [26] ——, “Statistical guarantee optimization for aoi in single-hop and two-hop fcfs systems with periodic arrivals,” _IEEE Transactions on Communications_ , vol. 69, no. 1, pp. 365–381, 2021.
  * [27] V. Tripathi, R. Talak, and E. Modiano, “Age of information for discrete time queues,” _arXiv:1901.10463v1_ , 2019.
  * [28] A. Kosta, N. Pappas, A. Ephremides, and V. Angelakis, “Non-linear age of information in a discrete time queue: Stationary distribution and average performance analysis,” _ICC 2020 - 2020 IEEE International Conference on Communications (ICC)_ , pp. 1–6, 2020.
  * [29] N. Akar, O. Doğan, and E. U. Atay, “Finding the exact distribution of (peak) age of information for queues of ph/ph/1/1 and m/ph/1/2 type,” _IEEE Transactions on Communications_ , vol. 68, no. 9, pp. 5661–5672, 2020.
  * [30] N. Akar, “Discrete-time queueing model of age of information with multiple information sources,” _IEEE Internet of Things Journal_ , pp. 1–1, 2021\.
  * [31] O. Dogan and N. Akar, “The multi-source preemptive m/ph/1/1 queue with packet errors: Exact distribution of the age of information and its peak,” _arXiv:2007.11656v1_ , 2020.
  * [32] O. Ayan, H. M. Gürsu, A. Papa, and W. Kellerer, “Probability analysis of age of information in multi-hop networks,” _IEEE Networking Letters_ , vol. 2, no. 2, pp. 76–80, 2020.
  * [33] J.-B. Seo and J. Choi, “On the outage probability of peak age-of-information for d/g/1 queuing systems,” _IEEE Communications Letters_ , vol. 23, no. 6, pp. 1021–1024, 2019.