the representation space $V$ of $U$ consists of all functions
$h:G\rightarrow\mathbb{C}$ for which $h(xn)=\chi(n)^{-1}h(x)\forall x\in
G,n\in N$ and $U$ acts on such a function to give $U(y)h(x)=h(y^{-1}x),x,y\in
G$. The condition

$\int_{G}f(x)U(x)dx=0$

gives

$\int_{G}f(x)h(x^{-1}y)dx=0\forall y\in G\forall h\in V$

Now for $h\in V$, we have

$h(xn)=\chi(n)^{-1}h(x),x\in G,n\in N$

and hence for any representation $\pi$ of $G$,

$\int h(xn)\pi(x^{-1})dx=\chi(n)^{-1}\hat{h}(\pi)$

or

$\int h(x)\pi(n.x^{-1})dx=\chi(n)^{-1}\hat{h}(\pi)$

or

$\pi(n)\hat{h}(\pi)=\chi(n^{-1})\hat{h}(\pi),n\in N$

Define

$P_{\pi,N}=\int_{N}\pi(n)dn$

Assume without loss of generality that $\pi$ is a unitary representation of
$G$. Then, $P_{\pi,N}$ is the orthogonal projection onto the space of all
those vectors $w$ in the representation space of $\pi$ for which
$\pi(n)w=w\forall n\in N$. Then the above equation gives on integrating w.r.t
$n\in N$,

$P_{\pi,N}\hat{h}(\pi)=0,h\in V,\chi\neq 1$

and

$P_{\pi,N}\hat{h}(\pi)=\hat{h}(\pi),h\in V,\chi=1$

on using the Schur orthogonality relations for the characters of $N$. We
therefore have The equation

$\int_{G}f(x)h(x^{-1}y)dx=0$

gives us

$\int_{G}f(x)h(x^{-1}y)\pi(y^{-1})dxdy=0$

or

$\int f(x)h(z)\pi(z^{-1}x^{-1})dxdz=0$

or

$\hat{h}(\pi)\hat{f}(\pi)=0,h\in V$

It also implies

$\int f(x)h(x^{-1}y)\pi(y)dxdy=0$

or

$\int f(x)h(z)\pi(xz)dxdz=0$

or

$\hat{f}(\pi)^{*}\hat{h}(\pi)^{*}=0$

which is the same as the above. Suppose on the other hand that $f$ satisfies

$\int_{N}f(xny)dn=0,x,y\in G---(1)$

Then, it follows that

$\int_{G\times N}f(xny)\pi(y^{-1})dydn=0$

which implies

$\int_{G\times N}f(z)\pi(z^{-1}xn)dzdn=0$

This implies

$\hat{f}(\pi)\pi(x)P_{\pi,N}=0,x\in G$

It also implies

$\int f(xny)\pi(x^{-1})dxdn=0$

which implies

$\int f(z)\pi(nyz^{-1})dzdn=0$

or equivalently,

$P_{\pi,N}\pi(y)\hat{f}(\pi)=0,y\in G$

In particular, (1) implies

$P_{\pi,N}\hat{f}(\pi)=\hat{f}(\pi)P_{\pi,N}=0$

for all representations $\pi$ of $G$. The condition (1) implies that
$P\pi(y)\hat{f}(\pi)=0$ where $P=P_{\pi,N}$ for all representations $\pi$ and
hence for all functions $h$ on $G$ it also implies that
$P\hat{h}(\pi)\hat{f}(\pi)=0$. More precisely, it implies that for all
representations $\pi$ and all functions $h$ on $G$,

$P_{\pi,N}\hat{h}(\pi)\hat{f}(\pi)=0---(2)$

Now suppose that $h_{1}$ belongs to the the representation space of
$U=Ind^{G}_{N}\chi_{0}$ where $\chi_{0}$ is the identity character on $N$, ie,
one dimensional unitary representation of $N$ equal to unity at all elements
of $G$. Then, it is clear that $h_{1}$ can be expressed as

$h_{1}(x)=\int_{N}h(xn)dn,x\in N$

for some function $h$ on $G$ and conversely if $h$ is an arbitrary function on
$G$ for which the above integral exists, then $h_{1}$ will belong to the
representation space of $U$. That is because $h_{1}$ as defined above
satisfies

$h_{1}(xn)=h_{1}(x),x\in G,n\in N$

The condition that $f$ be orthogonal to the matrix elements of $U$ is that

$\int f(x)U(x^{-1})dx=0$

and this condition is equivalent to the condition that for all $h$ on $G$,

$\int_{G}f(x)h_{1}(xy)dx=0,y\in G,h=\int_{N}h(xn)dn$

and this condition is in turn equivalent to

$\int f(x)h_{1}(xy)\pi(y^{-1})dxdy=0$

or

$\int f(x)h_{1}(z)\pi(z^{-1}x)dxdz=0$

or equivalently,

$\hat{h}_{1}(\pi)\hat{f}(\pi)^{*}=0$

for all unitary represenations $\pi$ of $G$ and for all functions $h$ on $G$.
Now,

$\hat{h}_{1}(\pi)=\int h(xn)\pi(x^{-1})dxdn=\int
h(z)\pi(nz^{-1})dzdn=P.\hat{h}(\pi)$

and hence the condition is equivalent to

$P_{\pi,N}\hat{h}(\pi)\hat{f}(\pi)^{*}=0---(3)$

for all unitary representations $\pi$ of $G$ and for all functions $h$ on $G$.
This condition is the same as (2) if we observe that $\hat{f}(\pi)$ can be
replaced by $\hat{f}(\pi)^{*}$ because $f(x)$ being orthogonal to the matrix
elements of $U(x)$ is equivalent to $f(x^{-1})$ being orthogonal to the same
because for compact groups, a finite dimensional representation is equivalent
to a unitary representation which means that $U$ is an appropriate basis is a
unitary representation. Thus, we have proved that a necessary and sufficient
condition that a function $f$ be orthogonal to all the matrix elements of the
representation of $G$ induced by the identity character $\chi_{0}$ on $N$ is
that $f$ be a cusp form, ie, it should satisfies

$\int_{N}f(xny)dn=0\forall x,y\in G$

Now suppose $\chi$ is an arbitary character of $N$ (ie a one dimensional
unitary representation of $N$). Then let $U=Ind^{G}_{N}\chi$. The If $h_{1}$
belongs to the representation space of $U$, then
$h_{1}(xn)=\chi(n)^{-1}h_{1}(x),x\in G,n\in N$. Now let $h(x)$ be any function
on $G$. Define

$h_{1}(x)=\int_{N}h(xn)\chi(n)dn$

Then,

$h_{1}(xn_{0})=\int_{N}h(xn_{0}n)\chi(n)dn=\int h(xn)\chi(n_{0}^{-1}n)dn$
$=\chi(n_{0})^{-1}h_{1}(x),n_{0}\in N,x\in G$

It follows that $h_{1}$ belongs to the representation space of $U$. conversely
if $h_{1}$ belongs to this representation space, then

$h_{1}(x)=\int_{N}h_{1}(xn)\chi(n)dn$

Thus we have shown that $h_{1}$ belongs to the representation space of
$U=Ind^{G}_{N}\chi$ iff there exists a function $h$ on $G$ such that

$h_{1}(x)=\int h(xn)\chi(n)dn$

Note that this condition fails for non-compact groups since the integral $dn$
on $N$ cannot be normalized in general. Then, the condition for $f$ to be
orthogonal to $U$ is that

$\int_{G}f(x)U(x^{-1})dx=0---(4)$

and this condition is equivalent to

$\int f(x)h_{1}(xy)dx=0$

for all $h_{1}$ in the representation space of $U$ or equivalently,

$\int f(x)h(xyn)\chi(n)dxdn=0$

for all functions $h$ on $G$. This condition is equivalent to

$\int f(x)h(xyn)\chi(n)\pi(y^{-1})dxdydn=0$

for all unitary representations $\pi$ of $G$ which is in turn equivalent to

$\int_{G\times G\times N}f(x)h(z)\chi(n)\pi(nz^{-1}x)dxdzdn=0$

or equivalently,

$P_{\chi,\pi,N}.\hat{h}(\pi).\hat{f}(\pi)^{*}=0---(5)$

for all functions $h$ on $G$ and for all unitary representations $\pi$ of $G$
where

$P_{\chi,\pi,N}=P=\int_{N}\chi(n)\pi(n)dn$

Now consider the following condition on a function $f$ defined on $G$:

$\int_{N}f(xny)\chi(n)dn=0,x,y\in G$

This condition is equivalent to

$\int_{G\times N}f(xny)\chi(n)\pi(y^{-1})dydn=0$

for all unitary representations $\pi$ of $G$ or equivalently,

$\int_{G\times N}f(z)\chi(n)\pi(z^{-1}xn)dzdn=0$

and this is equivalent to

$\hat{f}(\pi)\pi(x)P=0,x\in G---(6)$

which is in turn equivalent to

$\hat{f}(\pi)\hat{h}(\pi)P=0---(7)$

for all functions $h$ on $G$ as follows by multiplying $(5)$ with $h(x^{-1})$
and integrating over $G$. This condition is the same as

$P_{\chi,\pi,N}\hat{h}(\pi)^{*}\hat{f}(\pi)^{*}=0$

or since $h(x)$ can be replaced by $h(x^{-1})$, the condition is equivalent to

$P_{\chi,\pi,N}\hat{h}(\pi)\hat{f}(\pi)^{*}=0---(8)$

for all functions $h$ on $G$ and all unitary representations $\pi$ of $G$.
This condition is the same as (5). Thus, we have established that a necessary
and sufficient condition for $f$ to be orthogonal to all the matrix elements
of $U=Ind^{G}_{N}\chi$ is that $f$ should satisfy

$\int_{N}f(xny)\chi(n)dn=0,x,y\in G$

In other words, $f$ should be a generalized cusp form corresponding to the
character $\chi$ of $N$.

###  4.2 On the dynamics of breathing in the presence of a mask

Let $V$ denote the volume between the face and the mask. Oxygen passes from
outside the mask to within which the person breathes in. While breathing out,
the person ejects $CO_{2}$ along with of course some fraction of oxygen. Let
$n_{o}(t)$ denote the number of oxygen molecules per unit volume at time $t$
and $n_{c}(t)$ denote the number of carbon-dioxide molecules per unit volume
at time $t$ in the region between the face and the mask. Let $n_{r}(t)$ denote
the number of molecules of the other gases in the same region per unit volume.
Let $\lambda_{c}(t)dt$ denote the probability of a $CO_{2}$ molecule escaping
away the volume between the mask and the face in time $[t,t+dt]$. Then let
$g_{c}(t)dt$ denote the number of $CO_{2}$ molecules being ejected from the
nose and mouth of the person per unit time. Then, we have a simple
conservation equation

$dn_{c}(t)/dt=-n_{c}(t)\lambda_{c}(t)+g_{c}(t)/V$

To this differential equation, we can add stochastic terms and express it as

$dn_{c}(t)/dt=-n_{c}(t)\lambda_{c}(t)+g_{c}(t)/V+w_{c}(t)$

We can ask the question of determining the statistics of the first time
$\tau_{c}$ at which the $CO_{2}$ concentration $n_{c}(t)$ exceeds a certain
threshold $N_{c}$. Sometimes there can be a chemical reaction between the
$CO_{2}$ trapped in the volume with some other gases. Likewise other gases
within the volume can react and produce $CO_{2}$. If we take into account
these chemical reactions, then there will be additional terms in the above
equation describing the rate of change of concentration of $CO_{2}$. For
example, consider a reaction of the form

$a.CO_{2}+b.X\rightarrow c.Y+d.Z$

where $a,b,c,d$ are positive integers. $X$ denotes the concentration of the
other gases, ie, gases other than $CO_{2}$ within the volume. At time $t=0$,
there are $n_{c}(0)$ molecules of $CO_{2}$ and $n_{X}(0)$ molecules of $X$ and
zero molecules of $Y$ and $Z$. Let $K$ denote the rate constant at fixed
temperature $T$. Then, we have an equation of the form

$dn_{c}(t)/dt=-K.n_{c}(t)^{a}n_{X}(t)^{b}---(1)$

according to Arrhenius’ law of mass action. After time $t$, therefore,
$n_{c}(0)-n_{c}(t)$ molecules of $CO_{2}$ have reacted with
$(n_{c}(0)-n_{c}(t))b/a$ moles of $X$. Thus,

$n_{X}(t)=n_{X}(0)-(n_{c}(0)-n_{c}(t))b/a$

from which we deduce

$dn_{X}(t)/dt=(b/a)dn_{c}(t)/dt=(-Kb/a)n_{c}(t)^{a}.n_{X}(t)^{b}---(2)$

equations (1) and (2) have to be jointly solved for $n_{c}(t)$ and $n_{X}(t)$.
This can be reduced to a single differential equation for $n_{c}(t)$:

$dn_{c}(t)/dt=-K.n_{c}(t)^{a}(n_{X}(0)-(n_{c}(0)-n_{c}(t))b/a)^{b}---(3)$

Likewise, we could also have a chemical reaction in which $CO_{2}$ is produced
by a chemical reaction between two compounds $U,V$. We assume that such
reaction is of the form

$pU+qV\rightarrow r.CO_{2}+s.M$

Let $n_{U}(0),n_{V}(0)$ denote the number of molecules of $U$ and $V$ at time
$t=0$. Then, the rate of at which their concentrations change with time is
again given by Arrhenius’ law of mass action

$dn_{U}(t)/dt=-K_{1}n_{U}(t)^{p}n_{V}(t)^{q}$
$n_{V}(t)=n_{V}(0)-q(n_{U}(0)-n_{U}(t))/p$

and then the number of $CO_{2}$ molecules generated after time $t$ due to this
reaction will be given by

$n_{c}(t)=(r/p)(n_{U}(0)-n_{U}(t))$

and also

$n_{V}(t)=n_{V}(0)-(q/p)(n_{U}(0)-n_{U}(t))=n_{V}(0)-(q/r)n_{c}(t)$

and hence

$dn_{c}(t)/dt=(-r/p)dn_{U}(t)/dt=(r/p)K_{1}n_{U}(t)^{p}.n_{V}(t)^{q}$
$=(K_{1}r/p)(n_{U}(0)-pn_{c}(t)/r)^{p}(n_{V}(0)-(q/r)n_{c}(t))^{q}$

It follows that combining these two processes we can write Now consider the
combination of two such reactions. Writing down explicitly these reactions, we
have

$aCO_{2}+bX\rightarrow c.Y+d.Z,p.U+q.V\rightarrow r.CO_{2}+s.Z$

At time $t=0$, we have initial conditions on
$n_{c}(0),n_{X}(0),n_{U}(0),n_{V}(0)$. Then the rate at which $CO_{2}$ gets
generated by the combination of these two reactions and the other non-
chemical-reaction processes of diffusion exit etc. is given by

$dn_{c}(t)/dt=-Kn_{c}(t)^{a}n_{X}(t)^{b}+(K_{1}r/p)n_{U}(t)^{p}n_{V}(t)^{q}+f(n_{c}(t),t)$

where

$dn_{X}(t)/dt=(-Kb/a)n_{c}(t)^{a}n_{X}(t)^{b},dn_{U}(t)/dt=-K_{1}n_{U}(t)^{p}n_{V}(t)^{q},$
$dn_{V}(t)/dt=(-K_{1}q/p)n_{U}(t)^{p}n_{V}(t)^{q}$

Here, $f(n_{c}(t),t)$ denotes the rate at which $CO_{2}$ concentration changes
due to the other effects described above. Of course, we could have more
complex chemical reactions in of the sort

$a_{0}CO_{2}+a_{1}X_{1}+...+a_{n}X_{n}\rightarrow b_{1}Y_{1}+...+b_{m}Y_{m},$

for which the law of mass action would give

$dn_{c}(t)/dt=-Kn_{c}(t)^{a_{0}}n_{X_{1}}(t)^{a_{1}}...n_{X_{n}}(t)^{a_{n}},$
$n_{X_{j}}(t)=n_{X_{j}}(0)-(a_{j}/a_{0})(n_{c}(0)-n_{c}(t)),j=1,2,...,n,$
$n_{Y_{j}}(t)=n_{Y_{j}}(0)+(b_{j}/a_{0})(n_{c}(0)-n_{c}(t)),j=1,2,...,m$

###  4.3 Plancherel formula for $SL(2,\mathbb{R})$

Let $G=KAN$ be the usual Iwasawa decomposition and let $B$ denote the
compact/elliptic Cartan subgroup $B=\\{exp(\theta(X-Y)),\theta\in[0,2\pi)\\}$.
Let $L$ denote the non-compact/hyperbolic Cartan subgroup
$L=\\{exp(tH),t\in\mathbb{R}\\}$. Let $G_{B}$ denote the conjugacy class of
$B$, ie, $G_{B}$ is the elliptic subgroup $gBg{-1},g\in G$ and likewise
$G_{L}$ will denote the conjugacy class of $L$ ie, the hyperbolic subgroup
$gLg^{-1},g\in G$. These two subgroups have only the identity element in
common. Now let $\Theta_{m}$ denote the discrete series characters. We know
that

$\Theta_{m}(u(\theta))=-sgn(m)\frac{exp(im\theta)}{exp(i\theta)-exp(-i\theta)},m\in\mathbb{Z},u(\theta)=exp(\theta(X-Y))$
$\Theta_{m}(a(t))=\frac{exp(mt)}{exp(t)-exp(-t)},a(t)=exp(tH)$

Weyl’s integration formula gives

$\Theta_{m}(f)=\int_{G}\Theta_{m}(x)f(x)dx=$
$\int_{0}^{2\pi}\Theta_{m}(u(\theta))F_{f,B}(theta)\Delta_{B}(\theta)d\theta$
$+\int_{\mathbb{R}}\Theta_{m}(a(t))F_{f,L}(t)\Delta_{L}(t)dt$ $=\int
sgn(m)exp(im\theta)F_{f,B}(\theta)d\theta+\int_{\mathbb{R}}exp(-|mt|)F_{f,L}(t)dt$
$=sgn(m)\hat{F}_{f,B}(m)+\int_{\mathbb{R}}exp(-|mt|)F_{f,L}(t)dt$

where $F_{f,B}$ and $F_{f,L}$ are respectively the orbital integrals of $f$
corresponding to the elliptic and hyperbolic groups and if $a(\theta)$ is a
function of $\theta$, then $\hat{a}(m)$ denotes its Fourier transform, ie, its
Fourier series coefficients. Thus,

$F_{f,B}(\theta)=\Delta_{B}(\theta)\int_{G/B}f(xu(\theta)x^{-1})dx$
$F_{f,L}(a(t))=\Delta_{L}(t)\int_{G/L}f(xa(t)x^{-1})dx$

where

$\Delta(\theta)=(exp(i\theta)-exp(-i\theta)),\Delta(t)=exp(t)-exp(-t)$

Now

$\sum_{modd}|m|\Theta_{m}(f)=\int_{0}^{2\pi}\Delta_{L}(t)F_{f,L}(t)\sum_{modd}|m|\Theta_{m}(a(t))dt$
$+\int_{\mathbb{R}}\Delta_{B}(\theta)F_{f,B}(\theta).\sum_{modd}|m|\Theta_{m}(u(\theta))d\theta$

Now,

$\sum_{modd}|m|\Theta_{m}(a(t))=\sum_{modd}|m|exp(-|mt|)/(exp(t)-exp(-t))$
$\sum_{m\geq 1,modd}mexp(-mt)=\sum_{k\geq 0}(2k+1)exp(-(2k+1)t)$ $\sum_{m\geq
0}m.exp(-mt)=-d/dt(1-exp(-t))^{-1}=exp(-t)(1-exp(-t))^{-2},t\geq 0$

So

$\sum_{k\geq
0}(2k+1)exp(-(2k+1)t)=exp(-t)(2exp(-2t)(1-exp(-2t))^{-2}+(1-exp(-2t))^{-1})$
$=exp(-t)(1+exp(-2t))/(1-exp(-2t))^{2}=(exp(t)+exp(-t))/(exp(t)-exp(-t))^{2}$

Thus,

$\sum_{modd}|m|exp(-|mt|)=2\sum_{m\geq 0}m.exp(-m|t|)=tanh(t)=w_{1}(t)$

say.

$\sum_{modd}|m|\Theta_{m}(u(\theta))=\sum_{modd}m.exp(im\theta)/(exp(i\theta)-exp(-i\theta))=w_{2}(\theta)$

say. Then, the Fourier transform of $F_{f,B}^{\prime}(\theta)$ (ie Fourier
series coefficients)

$\hat{F}_{f,B}^{\prime}(m)=\int F_{f,B}^{\prime}(\theta)exp(im\theta)d\theta=$
$-\int F_{f,B}(\theta)(im)exp(im\theta)d\theta=-im\hat{F}_{f,B}(m)$

Now, we’ve already seen that

$\Theta_{m}(f)=sgn(m)\hat{F}_{f,B}(m)+\int_{\mathbb{R}}exp(-|mt|)F_{f,L}(t)dt$

So

$-im\Theta_{m}(f)=sgn(m)(-im\hat{F}_{f,B}(m))-im\int exp(-|mt|)F_{f,L}(t)dt$
$=sgn(m)\hat{F}_{f,B}^{\prime}(m)-im\int exp(-|mt|)F_{f,L}(t)dt$

Multiplying both sides by $sgn(m)$ gives us

$-i|m|\Theta_{m}(f)=\hat{F}_{f,B}^{\prime}(m)-i|m|\int exp(-|mt|)F_{f,L}(t)dt$

Summing over odd $m$ gives

$-i\sum_{modd}|m|\Theta_{m}(f)=\sum_{modd}\hat{F}_{f,B}^{\prime}(m)-i\int
w_{1}(t)F_{f,L}(t)dt$

Now,

$\sum_{m}\hat{F}_{f,B}^{\prime}(m)=F_{f,B}^{\prime}(0)-i\int
w_{1}(t)F_{f,L}(t)dt$

Now,

$F_{f,B}(\theta)=(exp(i\theta)-exp(-i\theta))\int_{G/B}f(xu(\theta)x^{-1})dx$

which gives on differentiating w.r.t $\theta$ at $\theta=0$

$F_{f,B}^{\prime}(0)=2if(1)$

Thus, we get the celebrated Plancherel formula for $SL(2,\mathbb{R})$,

$2f(1)=-\sum_{m}|m|\Theta_{m}(f)+\int w_{1}(t)F_{f,L}(t)dt$

Now we know that the Fourier transform

$\hat{F}_{f,L}(\mu)=\int_{\mathbb{R}}F_{f,L}(t)exp(it\mu)dt$

equals $T_{i\mu}(f)$, the character of the Principal series corresponding to
$i\mu,\mu\in\mathbb{R}$. Thus defining

$\hat{w}_{1}(\mu)=\int_{\mathbb{R}}w_{1}(t)exp(it\mu)dt$

the Plancherel formula can be expressed in explicit form:

$2f(1)=-\sum_{m}|m|\Theta_{m}(f)+\int_{\mathbb{R}}\hat{w}(\mu)^{*}.T_{i\mu}(f)d\mu$

This formula was generalized to all semisimple Lie groups and also to p-adic
groups by Harish-Chandra.

###  4.4 Some applications of the Plancherel formula for a Lie group to image
processing

Let $G$ be a group and let its irreducible characters be
$\chi,\chi\in\hat{G}$. let $d\mu(\chi)$ be its Plancherel measure:

$\delta_{e}(g)=\int\chi(g).d\mu(\chi)$

or equivalently,

$f(e)=\int_{\hat{G}}\chi(f)d\mu(\chi)$

for all $f$ in an appropriate Schwartz space. Consider an invariant for image
pairs of the form

$I_{\chi}(f_{1},f_{2})=\int_{G\times G}f_{1}(x)\chi(xy^{-1})f_{2}(y)dxdy$

Then if $\psi$ is any class function on $\hat{G}$, we have

$\int_{\hat{G}}\chi(\psi).I_{\chi}(f_{1},f_{2})d\mu(\chi)=$ $\int
f_{1}(x)\chi(\psi^{0})\chi(xy^{-1})f_{2}(y)dxdyd\mu(\chi)$ $=\int
f_{1}(x)\psi_{1}(xy^{-1})f_{2}(y)dxdy$

where

$\psi_{1}(x)=\int\chi(\psi^{0})\chi(x)d\mu(\chi),\psi^{0}(x)=\psi(x^{-1})$

so that

$\chi(\psi^{0})=\int\psi(x^{-1})\chi(x)dx$

Remark: let $\pi_{\chi}$ be an irreducible representation having character
$\chi$. Then, we can write

$f(x)=\int Tr(\hat{f}(\pi_{\chi})\pi_{\chi}(x))d\mu(\chi)$

where

$\hat{f}(\pi_{\chi})=\int f(x)\pi_{\chi}(x^{-1}))dx$

Thus, in particular, if $f$ is a class function,

$f(x)=\int f(y)Tr(\pi_{\chi}(y^{-1})\pi_{\chi}(x))d\mu(\chi)dy$ $=\int
f(y)\chi(y^{-1}x)d\mu(\chi)dy$

as follows from the Plancherel formula.

Problem: What is the relationship between $\psi_{1}(x)$ and $\psi(x)$ when
$\psi$ is a class function ?

Note that if the group is compact, then if $\psi$ is a class function and
$\pi$ is an irreducible unitary representation of dimension $d$, we have

$\hat{\psi}(\pi)=\int\psi(y)\pi(y)^{*})dy=\int\psi(xyx^{-1})\pi(y^{-1})dydx$
$=\int\psi(y)\pi(x^{-1}y^{-1}x)dydx$

Now,

$\int\pi_{ab}(x^{-1}yx)dx=\int\pi_{ac}(x^{-1})\pi_{cd}(y)\pi_{db}(x)dx$
$=\pi_{cd}(y)\int\bar{\pi}_{ca}(x).\pi_{db}(x)dx=d^{-1}\pi_{cd}(y)\delta(c,d)\delta(a,b)$
$=d^{-1}\pi_{cc}(y)\delta(a,b)=d^{-1}\chi(y)\delta(a,b)$

where $\chi=Tr(\pi)$ is the character of $\pi$. This means that

$\hat{\psi}(\pi)=d^{-1}\chi(\psi)I$

Thus, for a class function on a compact group, we have

$\psi(g)=\sum_{\chi\in\hat{G}}d(\chi)^{-1}Tr(\chi(\psi^{0})\pi(g))$
$=\sum_{\chi\in\hat{G}}d(\chi)^{-1}\chi(\psi^{0})\chi(g)$
$=\sum_{\chi\in\hat{G}}d(\chi)^{-1}\chi(g)\int\psi(x)\bar{\chi}(x)dx$

This is the same as saying that

$\delta_{g}(x)=\sum_{\chi\in\hat{G}}d(\chi)^{-1}\chi(g)\bar{\chi}(x)$

Is this result true for an arbitrary semisimple Lie group when we replace
$d(\chi)$ by the Plancherel measure/formal dimension on the space of
irreducible characters ? In other words, is the result

$\delta_{g}(x)=\int_{\hat{G}}\chi(g)\bar{\chi}(x)d\mu(\chi)$

true ?

##  Chapter 5 Some problems in quantum information theory

A problem in quantum information theory used in the proof of the coding
theorem for Cq channels : Let $S,T$ be two non-negative operators with $S\leq
1$, ie, $0\leq S\leq 1,T\geq 0$. Then

$1-(S+T)^{-1/2}S(S+T)^{-1/2}\leq 2-2S+4T$

or equivalently,

$(S+T)^{-1/2}S(S+T)^{-1/2}\geq 2S-1-4T$

To prove this, we define the operators

$A=\sqrt{T}((T+S)^{-1/2}-1),B=\sqrt{T}$

Then, the equation

$(A-B)^{*}(A-B)\geq 0$

or equivalently,

$A^{*}A+B^{*}B\geq A^{*}B+B^{*}A$

gives

$((T+S)^{-1/2}-1)T+T((T+S)^{-1/2}-1)\leq T+((T+S)^{-1/2}-1)T((T+S)^{-1/2}-1)$

This equation can be rearranged go give

$2(T+S)^{-1/2}T+2T(T+S)^{-1/2}-4T\leq(T+S)^{-1/2}T.(T+S)^{-1/2}$
$=(T+S)^{-1/2}(T+S-S)(T+S)^{-1/2}=1-(T+S)^{-1/2}S(T+S)^{-1/2}$

A rearrangement of this equation gives

$4T+1+(S+T)^{-1/2}S(S+T)^{-1/2}\geq
2(S+T)^{-1/2}S(S+T)^{-1/2}+2(S+T)^{-1/2}T+2T(S+T)^{-1/2}$

So to prove the stated result, it suffices to prove that

$(S+T)^{-1/2}S(S+T)^{-1/2}+(S+T)^{-1/2}T+T(S+T)^{-1/2}\geq S$

Let $A,B$ be two states. The hypothesis testing problem amounts to choosing an
operator $0\leq T\leq 1$ such that

$C(T)=Tr(A(1-T))+Tr(BT)$

is a minimum. Now,

$C(T)=1+Tr((B-A)T)$

It is clear that this is a minimum iff

$T=\\{B-A\leq 0\\}=\\{A-B\geq 0\\}$

ie $T$ is the projection onto the space spanned by those eigenvectors of $B-A$
that have positive eigenvalues or equivalently the space spanned by those
eigenvectors of $A-B$ that have non-negative eigenvalues. The minimum value of
the cost function is then

$C(min)=Tr(A\\{A<B\\})+Tr(B\\{A>B\\})$

Note that the minimum is attained when $T$ is a projection.

An inequality: Let $0\leq s\leq 1$. Then if $A,B$ are any two positive
operators, we have

$Tr(A\\{A<B\\})\leq Tr(A^{1-s}B^{s})$

To see this we note that

$Tr(A^{1-s}B^{s})\leq Tr((B+(A-B)_{+})^{1-s}B^{s})\leq
Tr(B+(A-B)_{+})^{1-s}(B+(A-B)_{+})^{s})$
$=Tr(B+(A-B)_{+})=Tr(B)+Tr((A-B)_{+})=Tr(B)+Tr((A-B)\\{A-B>0\\})$
$=Tr(B\\{A-B<0\\})+Tr(A.\\{A-B>0\\})$

Other related inequalities are as follows:

$Tr(A^{1-s}B^{s})\geq Tr((B-(B-A)_{+})^{1-s}B^{s})$ $\geq
Tr((B-(B-A)_{+})^{1-s}.(B-(B-A)_{+})^{s})=Tr((B-(B-A)_{+})$
$=Tr(B)-Tr((B-A)_{+})=Tr(B)-Tr((B-A)\\{B>A\\})$
$=Tr(B.\\{A>B\\})+Tr(A.\\{B>A\\})$

In particular, when $A,B$ are states, the minimum probability of
discrimination error has the following upper bound:

$P(error)=Tr(A.\\{B>A\\})+Tr(B.\\{A>B\\})\leq min_{0\leq s\leq
1}Tr(A^{1-s}B^{s})$ $=min_{0\leq s\leq 1}exp(\phi(s|A|B))$

where

$\phi(s|A|B)=log(Tr(A^{1-s}B^{s}))$

To obtain the minimum in the upper bound, we set the derivative w.r.t $s$ to
zero to get

$d/ds(Tr(A^{1-s}B^{s})=0$

or

$Tr(log(A)A^{1-s}B^{s})=Tr(A^{1-s}B^{s}log(B))$

It is not clear how to solve this equation for $s$. Now define

$\psi_{1}(s)=log(Tr(A^{1-s}B^{s}))/s,\psi_{2}(s)=log(Tr(A^{1-s}B^{s})/(1-s)$

Then an elementary application of Le’Hopital’s rule gives us

$\psi_{1}(0+)=lim_{s\downarrow 0}\psi_{1}(s)=-D(A|B)=-Tr(A.(log(A)-log(B)),$
$\psi_{2}(1-)=lim_{s\uparrow 1}\psi_{2}(s)=-D(B|A)=-Tr(B.(log(B)-log(A))$

Consider now the problem of distinguishing between tensor product states
$A^{\otimes n}$ and $B^{\otimes n}$. Let $P_{n}(e)$ denote the minimum error
probability. Then, by the above,

$P_{n}(e)\leq min_{0\leq s\leq 1}exp(logTr(A^{\otimes n(1-s)}B^{\otimes ns})$

Now,

$Tr(A^{\otimes n(1-s)}B^{\otimes ns})=(Tr(A^{1-s}B^{s}))^{n}$

Therefore, we can write

$limsup_{n\rightarrow\infty}n^{-1}log(P_{n}(e))\leq$ $min_{0\leq s\leq
1}logTr(A^{1-s}B^{s})$

Define

$F(s)=Tr(A^{1-s}B^{s})$

Then,

$F^{\prime}(s)=-Tr(A^{1-s}log(A)B^{s})+Tr(A^{1-s}B^{s}log(B))$
$F^{\prime\prime}(s)=Tr(A^{1-s}(logA)^{2}B^{s})+Tr(A^{1-s}B^{s}(log(B))^{2})$
$-Tr(A^{1-s}log(A)B^{s}log(B))+Tr(A^{1-s}.log(A).B^{s}.log(B))$
$=Tr(A^{1-s}(logA)^{2}B^{s})+Tr(A^{1-s}B^{s}(log(B))^{2})\geq 0$

Hence $F(s)$ is a convex function and hence $F^{\prime}(s)$ is increasing.
Further,

$F^{\prime}(0)=-Tr(A.log(A))+Tr(A.log(B))=-D(A|B)\leq
0,F^{\prime}(1)=-Tr(B.log(A))+Tr(B.log(B))=D(B|A)\geq 0$

Hence $F^{\prime}(s)$ changes sign in $[0,1]$ and hence it must be zero at
some $s\in[0,1]$, say at $s_{0}$ and since $F$ is convex, it attains its
minimum at $s_{0}$.

Let $\rho,\sigma$ be two states and let

$\rho=\sum_{i}p(i)|e_{i}><e_{i}|,\sigma=\sum_{i}q(i)|f_{i}><f_{i}|$

be their respective spectral decompositions. Define

$p(i)|<e_{i}|f_{j}>|^{2}=P(i,j),q(j)|<e_{i}|f_{j}>|^{2}=Q(i,j)$

Then $P,Q$ are both classical probability distributions. We have for $0\leq
T\leq 1$,

$Tr(\rho(1-T))=\sum_{i,j}<e_{i}|\rho|f_{j}><f_{j}|1-T|e_{i}>$
$=\sum_{i,j}p(i)<e_{i}|f_{j}>(<f_{j}|e_{i}>-<f_{j}|T|e_{i}>)$
$=1-\sum_{i,j}p(i)<e_{i}|f_{j}>t(j,i)$

and likewise,

$Tr(\sigma
T)=\sum_{i,j}<e_{i}|\sigma|f_{j}>t(j,i)=\sum_{i,j}q(j)<e_{i}|f_{j}>t(j,i)$

where

$t(j,i)=<f_{j}|T|e_{i}>$

So,

$Tr(\rho(1-T))+Tr(\sigma T)=1+\sum_{i,j}t(j,i)(q(j)-p(i))<e_{i}|f_{j}>t(j,i)$

Now,

$|\sum_{i,j}p(i)<e_{i}|f_{j}>t(j,i)|^{2}\leq\sum_{i,j}p(i)|e_{i}|f_{j}>|^{2}.\sum_{i,j}p(i)|t(j,i)|^{2}$
$=\sum_{i,j}p(i)|t(j,i)|^{2}$

Thus,

$Tr(\rho(1-T))\geq 1-\sum_{i,j}p(i)|t(j,i)|^{2}$ $Tr(\sigma
T)=\sum_{i,j}q(j)<e_{i}|f_{j}>t(j,i)$

So

$Tr(\sigma T)^{2}\leq\sum_{i,j}q(j)|t(j,i)|^{2}$

Then, if $c$ is any positive real number,

$Tr(\rho(1-T))-c.Tr(\sigma T)\geq
1-\sum_{i,j}p(i)|t(j,i)|^{2}-c.\sum_{i,j}q(j)|t(j,i)|^{2}$

Suppose now that $T$ is a projection. Then,

$Tr(\rho(1-T))=\sum_{i}p(i)<e_{i}|1-T|e_{i}>=\sum_{i}p(i)<e_{i}|(1-T)^{2}|e_{i}>$
$=\sum_{i,j}p(i)|<e_{i}|1-T|f_{j}>|^{2}\geq_{i,j}a(i,j)|<e_{i}|1-T|f_{j}>|^{2}$

and likewise,

$Tr(\sigma
T)=\sum_{i,j}q(j)|<f_{j}|T|e_{i}>|^{2}\geq\sum_{i,j}a(i,j)|<f_{j}|T|e_{i}>|^{2}$

where

$a(i,j)=min(p(i),q(j))$

Now,

$|<e_{i}|1-T|f_{j}>|^{2}+<e_{i}|T|f_{j}>|^{2}\geq(1/2)|<e_{i}|1-T|f_{j}>+<e_{i}|T|f_{j}>|^{2}$
$=(1/2)|<e_{i}|f_{j}>|^{2}$

Thus,

$Tr(\rho(1-T))+Tr(\sigma T)\geq(1/2)\sum_{i,j}a(i,j)|<e_{i}|f_{j}>|^{2}$
$=(1/2)\sum_{i,j}P(i,j)\chi_{p(i)\leq
q(j)}+(1/2)\sum_{i,j}Q(i,j)\chi_{p(i)>q(j)}$
$=(1/2)\sum_{i,j}P(i,j)\chi_{P(i,j)\leq
Q(i,j)}+(1/2)\sum_{i,j}Q(i,j)\chi_{P(i,j)>Q(i,j)}$

Now, for tensor product states, this result gives

$Tr(\rho^{\otimes n}(1-T_{n}))+Tr(\sigma^{\otimes n}T_{n})\geq$
$(1/2)P^{n}(P^{n}\leq Q^{n})+(1/2)Q^{n}(P^{n}>Q^{n})$

for any measurement $T_{n}$ in the tensor product space, ie, $0\leq T_{n}\leq
1$. Now by the large deviation principle as $n\rightarrow\infty$, we have

$n^{-1}.log(P^{n}(P^{n}\geq Q^{n}))\approx-inf_{x\leq 0}I_{1}(x)$

where $I_{1}$ is the rate function under $P^{n}$ of the random family
$n^{-1}log(P^{n}/Q^{n})$. Evaluating the moment generating function of this
r.v gives us

$M_{n}(ns)=\mathbb{E}_{P^{n}}exp(s.log(P^{n}/Q^{n}))=\mathbb{E}_{P^{n}}(P^{n}/Q^{n})^{s}$
$=(\mathbb{E}_{P}(P/Q)^{s})^{n}=(\sum P^{1-s}Q^{s})^{n}=M_{s}(P,Q)^{n}$

and hence the rate function is

$I_{1}(x)=sup_{s}(sx-logM_{s}(P,Q))=$ $sup_{s}(sx-
log(\sum_{\omega}P(\omega)^{1-s}Q(\omega)^{s}))$

Then,

$inf_{x\leq 0}I_{1}(x)=sup_{s\geq
0}-log(\sum_{\omega}P(\omega)^{1-s}Q(\omega)^{s}))$ $=-inf_{s\geq
0}log(\sum_{\omega}P(\omega)^{1-s}Q(\omega)^{s}))$

This asymptotic result can also be expressed as

$liminf_{n\rightarrow\infty}inf_{0\leq T_{n}\leq 1}[n^{-1}log(Tr(\rho^{\otimes
n}(1-T_{n}))+Tr(\sigma^{\otimes n}T_{n}))]$ $\geq-inf_{0\leq s\leq 1}[log(\sum
P^{1-s}Q^{s})]-inf_{0\leq s\leq 1}[log\sum Q^{1-s}P^{s}]$

Relationship between quantum relative entropy and classical relative entropy:

$Tr(\rho.log(\rho))=\sum_{i}p(i)log(p(i))$
$Tr(\rho.log(\sigma))=\sum_{i,j}p(i)log(q(j))<e_{i}|f_{j}>|^{2}$
$D(\rho|\sigma)=Tr(\rho.log(\rho)-\rho.log(\sigma))$

Therefore,

$D(\rho|\sigma)=\sum_{i,j}p(i)<e_{i}|f_{j}>|^{2}(log(p(i))+log(|<e_{i}|f_{j}>|^{2}))$
$-\sum_{i,j}p(i)<e_{i}|f_{j}>|^{2}(log(q(j))+log(|<e_{i}|f_{j}>|^{2}))$

(Note the cancellations)

$=\sum_{i,j}P(i,j)log(P(i,j))-\sum_{i,j}P(i,j)log(Q(i,j))$ $=D(P|Q)$

Let $X$ be a r.v. with moment generating function

$M(\lambda)=\mathbb{E}[exp(\lambda X)]$

Its logarithmic moment generating function is

$\Lambda(\lambda)=log(M(\lambda))$

Let $\mu=\mathbb{E}(X)$ be its mean. Then,

$\mu=\Lambda^{\prime}(0)$

Further,

$\Lambda^{\prime\prime}(\lambda)\geq 0\forall\lambda$

ie $\Lambda$ is a convex function. Hence $\Lambda^{\prime}(\lambda)$ is
increasing for all $\lambda$. Now suppose $x\geq\mu$. Consider

$\psi(x,\lambda)=\lambda.x-\Lambda(\lambda)$

Since

$\partial_{\lambda}^{2}\psi(x,\lambda)=-\Lambda^{\prime\prime}(\lambda)<0$

it follows that $\partial_{\lambda}\psi(x,\lambda)$ is decreasing or
equivalently that $\psi(x,\lambda)$ is concave in $\lambda$. Suppose
$\lambda_{0}$ is such that

$\partial_{\lambda}\psi(x,\lambda_{0})=0$

Then it follows that for a fixed value of $x$, $\psi(x,\lambda)$ attains its
maximum at $\lambda_{0}$. Now $\partial_{\lambda}\psi(x,\lambda_{0})=0$ and
$\partial_{\lambda}\psi(x,\lambda)$ is decreasing in $\lambda$. It follows
that $\partial_{\lambda}\psi(x,\lambda)\leq 0$ for $\lambda\leq\lambda_{0}$.
Hence, $\psi(x,\lambda)$ is decreasing for $\lambda\geq\lambda_{0}$. Also for
$\lambda<\lambda_{0}$, $\partial_{\lambda}\psi(x,\lambda)\geq 0$ (since
$\partial_{\lambda}\psi(x,\lambda_{0})=0$ and
$\partial_{\lambda}\psi(x,\lambda)$ is decreasing for all $\lambda$) which
means that $\psi(x,\lambda)$ is increasing for $\lambda<\lambda_{0}$. Now

$\partial_{\lambda}\psi(x,0)=x-\mu\geq 0$

by hypothesis. Since $\partial_{\lambda}\psi(x,\lambda)$ is decreasing in
$\lambda$, it follows that $\partial_{\lambda}\psi(x,\lambda)\geq 0$ for all
$\lambda\leq 0$. This means that $\psi(x,\lambda)$ is increasing for all
$\lambda\geq 0$. In particular,

$\psi(x,\lambda)\leq\psi(x,0)=x,\forall\lambda\leq 0$

Thus,

$I(x)=sup_{\lambda\in\mathbb{R}}\psi(x,\lambda)=sup_{\lambda\geq
0}\psi(x,\lambda)$

in the case when $x\geq\mu$. Consider now the case when $x\leq\mu$.

###  5.1 Quantum information theory and quantum blackhole physics

Let $\rho_{1}(0)$ be the state of the system of particles within the event
horizon of a Schwarzchild blackhole and $\rho_{2}(0)$ the state outside. At
time $t=0$, the state of the total system to the interior and the exterior of
the blackhole is

$\rho(0)=\rho_{1}(0)\otimes\rho_{2}(0)$

Here, as a prototype example, we are considering the Klein-Gordon field of
particles in the blackhole as well as outside it. Let $\chi(x)$ denote this KG
field. Its action functional is given by

$S[\chi]=(1/2)\int(g^{\mu\nu}(x)\sqrt{-g(x)}\chi_{,\mu}(x)\chi_{,\nu}(x)-\mu^{2}\sqrt{-g(x)}\chi(x)^{2})d^{4}x$
$=(1/2)\int[g^{\mu\nu}(x)\chi_{,\mu}(x)\chi_{,\nu}(x)-\mu^{2}\chi(x)^{2}]\sqrt{-g(x)}d^{4}x$

which in the case of the Schwarzchild blackhole reads

$S[\chi]=(1/2)\int[\alpha(r)^{-1}\chi_{,t}^{2}-\alpha(r)\chi_{,r}^{2}-r^{-2}\chi_{,\theta}^{2}-r^{-2}sin(\theta)^{-2}\chi_{,\phi}^{2}]r^{2}sin(\theta)dtdrd\theta
d\phi=\int\mathcal{L}d^{4}x$

The corresponding KG Hamiltonian can also be written down. The canonical
position field is $\chi$ and the canonical momentum field is

$\pi=\delta S/\delta\chi_{,t}=\alpha(r)^{-1}\chi_{,t}$

so that the Hamiltonian density is

$\mathcal{H}=\pi.\chi_{,t}-\mathcal{L}=$
$(1/2)[\alpha(r)^{-1}\chi_{,t}^{2}+\alpha(r)\chi_{,r}^{2}+r^{-2}\chi_{,\theta}^{2}+r^{-2}sin(\theta)^{-2}\chi_{,\phi}^{2}]$
$=(1/2)[\alpha(r)\pi^{2}+\alpha(r)\chi_{,r}^{2}+r^{-2}\chi_{,\theta}^{2}+r^{-2}sin(\theta)^{-2}\chi_{,\phi}^{2}+\mu^{2}\chi^{2}]$

The KG field Hamiltonian is then

$H=\int\mathcal{H}d^{3}x=\int\mathcal{H}r^{2}sin(\theta)drd\theta d\phi$

The radial Hamiltonian: Assume that the angular oscillations of the KG field
are as per the classical equations of motion of the field corresponding to an
angular momentum square of $l(l+1)$. Then, we can write

$\chi(t,r,\theta,\phi)=\psi(t,r)Y_{lm}(\theta,\phi)$

an substituting this into the action functional, performing integration by
parts using

$L^{2}Y_{lm}(\theta,\phi)=l(l+1)Y_{lm}(\theta,\phi),$
$L^{2}=-(sin(\theta)^{-1}\frac{\partial}{\partial\theta}sin(\theta)\frac{\partial}{\partial\theta}+\frac{1}{sin^{2}(\theta)}\frac{\partial^{2}}{\partial\phi^{2}}$

gives us the ”radial action functional” as

$S_{l}[\phi]=(1/2)\int(\alpha(r)^{-1}\psi_{,t}^{2}-\alpha(r)\psi_{,r}^{2}-l(l+1)\psi^{2}/r^{2}-\mu^{2}\psi^{2})r^{2}dtdr$

and the corresponding radial Hamiltonian as

$H_{l}(\psi,\pi)=(1/2)\int_{0}^{\infty}[\alpha(r)(\pi^{2}+\psi_{,r}^{2})+(\mu^{2}+l(l+1)/r^{2})\psi^{2}]r^{2}dr$

where $\psi=\psi(t,r),\pi=\pi(t,r)$, the canonical position and momentum
fields are functions of only time and the radial coordinate.

###  5.2 Harmonic analysis on the Schwarz space of $G=SL(2,\mathbb{R})$

Let $f_{mn}(x,\mu)$ denote the matrix elements of the unitary principal series
appearing in the Plancherel formula and let $\phi_{kmn}(x)$ the matrix
elements of the discrete series appearing in the Plancherel formula. Then we
have a splitting of the Schwarz space into the direct sum of the discrete
series Schwarz space and the principal series Schwarz space. Thus, for any
$f\in L^{2}(G)$, we have the Plancherel formula

$\int_{G}|f(x)|^{2}dx=\sum_{kmn}d(k)|<f|\phi_{kmn}>|^{2}+\int_{\mathbb{R}}|<f|f_{mn}(.,\mu)>|^{2}c(mn,\mu)d\mu$

where $d(k)^{\prime}s$ are the formal dimensions of the discrete series while
$c(mn,\mu)$ are determined from the wave packet theory for principal series,
ie, from the asymptotic behaviour of the principal series matrix elements. The
asymptotic behaviour of the principal and discrete matrix elements is obtained
from the differential equations

$\Omega.f_{mn}(x,\mu)=-\mu^{2}f_{mn}(x,\mu)$

where $\Omega$ is the Casimir element for $SL(2,\mathbb{R})$. We can express
$x\in G$ as $x=u(\theta_{1})a(t)u(\theta_{2})$ and $\Omega$ as a second order
linear partial differential operator in $\partial/\partial
t,\partial/\partial\theta_{k},k=1,2$. Then, observing that

$f_{mn}(u(\theta_{1})a(t)u(\theta_{2}),\mu)=exp(i(m\theta_{1}+n\theta_{2}))f_{mn}(a(t),\mu)$

we get on noting that the restriction of the Casimir operator to
$\mathbb{R}.H$ is given by

$(\Omega f)(a(t))=\Delta_{L}(t)^{-1}\frac{d^{2}}{dt^{2}}\Delta_{L}(t)f(a(t))$

where

$\Delta_{L}(t)=e^{t}-e^{-t}$

as follows by applying both sides to the finite dimensional irreducible
characters of $SL(2,\mathbb{R})$ restricted to $\mathbb{R}.H$ on noting that
the restriction of such a character is given by
$(exp(mt)-exp(-mt))/(exp(t)-exp(-t))$, we get a differential equation of the
form

$\frac{d^{2}}{dt^{2}}f_{mn}(a(t),\mu)-q_{mn}(t)f_{mn}(a(t),\mu)=-\mu^{2}f_{mn}(a(t),\mu)$

where $q_{mn}(t)$ is expressed in terms of the hyperbolic Sinh functions. From
this equation, the asymptotic behaviour of the matrix elements
$f_{mn}(a(t),\mu)$ as $|t|\rightarrow\infty$ may be determined. It should be
noted that the discrete series modules can be realized within the principal
series modules and hence the asymptotic behaviour of the matrix elements
$f_{mn}(a(t),\mu)$ will determine that of the principal series when $\mu$ is
non-integral and that of the discrete series when $\mu$ is integral.
Specifically, consider a principal series representation $\pi(x)$. Let $e_{m}$
denote its highest weight vector and define

$f_{m}(x)=<\pi(x)e_{m}|e_{m}>$

where now $x$ is regarded as an element of the Lie algebra $\mathfrak{g}$ of
$G$. We have for any $a\in U(\mathfrak{g})$, the universal enveloping algebra
of $\mathfrak{g}$, the fact that

$\pi(a).e_{m}=0\implies af_{m}(x)=f(x;a)=0$

This means that if we consider the $U(\mathfrak{g})$-module
$V=U(\mathfrak{g})f_{m}$ and the map $T:\pi(U(\mathfrak{g}))e_{m}\rightarrow
V$ defined by $T(\pi(a)e_{m})=a.f_{m}$, then it is clear that $T$ is a well
defined linear map with kernel

$K_{T}=\\{\pi(a)e_{m}:a\in U(\mathfrak{g}),a.f_{m}=0\\}$

Note that $\pi(a)e_{m}\in K_{T}$ implies $a.f_{m}=0$ implies $b.a.f_{m}=0$
implies $\pi(b)\pi(a)e_{m}=\pi(b.a)e_{m}\in K_{T}$ for any $a,b\in
U(\mathfrak{g})$. Thus, $K_{T}$ is $U(\mathfrak{g})$ invariant. Then, the
$U(\mathfrak{g})$-module $V$ is isomorphic to the quotient module
$\pi(U(\mathfrak{g}))e_{m}/K_{T}$. However, $\pi(U(\mathfrak{g}))e_{m}$ is an
irreducible (discrete series) module. Hence, the submodule $K_{T}$ of it must
be zero and hence, we obtain the fundamental conclusion that
$V=U(\mathfrak{g})f_{m}$ is an irreducible discrete series module. Note that
$f_{m}$ is a principal series (reducible since $m$ is an integer) matrix
element and this means that we have realized the discrete series
representations within the principal series. This also means that in order to
study the asymptotic properties of the matrix elements of the discrete series,
it suffices to study asymptotic properties of differential operators in
$U(\mathfrak{g})$ acting on the principal series matrix elements $f_{m}$.

###  5.3 Monotonicity of quantum entropy under pinching

Let $|\psi_{1}>,...,|\psi_{n}>$ be normalized pure states, not necessarily
orthogonal. Let $p(i),i=1,2,...,n$ be a probability distribution and let
$|i>,i=1,2,...,n$ be an orthonormal set. Consider the pure state

$|\psi>=\sum_{i=1}^{n}\sqrt{p(i)}|\psi_{i}>\otimes|i>$

Note that this is a state because

$<\psi|\psi>=\sum_{i,j}\sqrt{p(i)p(j)}<\psi_{j}|\psi_{i}><j|i>=\sum_{i,j}\sqrt{p(i)p(j)}<\psi_{j}|\psi_{i}>\delta(j,i)=\sum_{i}p(i)=1$

We have

$\rho_{1}=Tr_{2}(|\psi><\psi|)=\sum_{i}p(i)|\psi_{i}><\psi_{i}|$
$\rho_{2}=Tr_{1}(|\psi><\psi|=\sum_{i}\sqrt{p(i)p(j)}<\psi_{j}|\psi_{i}>|i><j|$

Then by Schmidt’s theorem,

$H(\rho_{1})=H(\rho_{2})\leq H(p)$

Now let $\sigma_{1},\sigma_{2}$ be two mixed states and let $p>0,p+q=1$.
Consider the spectral decompositions

$\sigma_{1}=\sum_{i}p(i)|e_{i}><e_{i}|,\sigma_{2}=\sum_{i}q(i)|f_{i}>f_{i}|$

Then,

$p\sigma_{1}+q\sigma_{2}=\sum_{i}(pp(i)|e_{i}><e_{i}|+qq(i)|f_{i}><f_{i}|)$

So by the above result,

$H(p\sigma_{1}+q\sigma_{2})\leq H(\\{pp(i),qq(i)\\})$
$=-\sum_{i}(pp(i)log(pp(i))+qq(i)log(qq(i)))$
$=-p.log(p)-q.log(q)+pH(\\{p(i)\\})+qH(\\{q_{i}\\})=H(p,q)+p.H(\sigma_{1})+q.H(\sigma_{2})$

where

$H(p,q)=-p.log(p)-q.log(q)+pH(\\{p(i)\\})$

Now consider several mixed states $\sigma_{1},...,\sigma_{n}$ and a
probability distribution $p(i),i=1,2,...,n$. Then

$\sum_{i=1}^{n}p(i)\sigma_{i}=p(1)\sigma_{1}+(1-p(1))\sum_{i=2}^{n}p(i)/(1-p(1))\sigma_{i}$

So the same argument and induction gives

$H(\sum_{i=1}^{n}p(i)\sigma_{i})\leq
H(p(1),1-p(1))+p(1)H(\sigma_{1})+(1-p(1))\sum_{i=2}^{n}(p(i)/(1-p(1)))H(\sigma_{i})$
$+H(p(i)/(1-p(1)),i=2,3,...,n)$
$=H(p(i),i=1,2,...,n)+\sum_{i=1}^{n}p(i)H(\sigma_{i})$

Remark: Let $\rho$ be a state and $\\{P_{i}\\}$ a PVM. Define the state

$\sigma=\sum_{i}P_{i}\rho.P_{i}=1-\sum_{i}P_{i}(1-\rho)P_{i}$

Then

$0\leq D(\rho|\sigma)=Tr(\rho.log(\rho))-Tr(\rho.log(\sigma))$
$=-H(\rho)-\sum_{i}Tr(\rho.P_{i}.log(P_{i}\rho.P_{i})P_{i})=-H(\rho)-\sum_{i}Tr(P_{i}\rho.P_{i}.log(P_{i}\rho.P_{i})$
$=-H(\rho)-\sum_{i}Tr(P_{i}\rho.P_{i}.log(\sum_{j}P_{j}\rho.P_{j}))=-H(\rho)+H(\sigma)$

Thus,

$H(\sigma)\geq H(\rho)$

which when stated in words, reads, pinching increases the entropy of a state.

Remark: We’ve made use of the following facts from matrix theory:

$log(\sum_{i}P_{i}.\rho.P_{i})=\sum_{i}P_{i}.log(\rho)P_{i}$

because the $P_{i}^{\prime}s$ satisfy $P_{j}P_{i}=0,j\neq i$. In fact, we
write

$\sum_{i}P_{i}\rho.P_{i}=1-\sum_{i}P_{i}(1-\rho)P_{i}=1-A$

and use

$log(1-z)=-z-z^{2}/2-z^{3}/3-...$

combined with $P_{j}P_{j}=P_{i}\delta(i,j)$ to get

$log(\sum_{i}P_{i}\rho.P_{i})=-A-A^{2}/2-A^{3}/3-...$

where

$A=\sum_{i}P_{i}(1-\rho)P_{i},A^{2}=\sum_{i}P_{i}(1-\rho)^{2}P_{i},...,A^{n}=-\sum_{i}P_{i}(1-\rho)^{n}P_{i}$

so that

$log(1-A)=\sum_{i}P_{i}.log(\rho)P_{i}$

A particular application of this result has been used above in the form

$H(\sum_{i}\sqrt{p(i)p(j)}<\psi_{j}|\psi_{i}>|i><j|)\leq
H(\sum_{i,k}\sqrt{p(i)p(j)}<\psi_{j}|\psi_{i}><k|i><j|k>|k><k|)$
$=H(\sum_{k}p(k)|k><k|)=H(\\{p(k)\\})=-\sum_{k}p(k).log(p(k))$

##  Chapter 6 Lectures on Statistical Signal Processing

[1] Large deviation principle applied to the least squares problem of
estimating vector parameters from sequential iid vector measurements in a
linear model.

$X(n)=H(n)\theta+V(n),n=1,2,...$

$V(n)^{\prime}s$ are iid and have the pdf $p_{V}$. Then $\theta$ is estimated
based on the measurements $X(n),n=1,2,...,N$ using the maximum likelihood
method as

$\hat{\theta}(N)=argmax_{\theta}N^{-1}\sum_{n=1}^{N}log(p_{V}(X(n)-H(n)\theta))$

Consider now the case when $H(n)$ is a matrix valued stationary process
independent of $V(.)$. Then, the above maximum likelihood estimate is replaced
by

$\hat{\theta}(N)=argmax_{\theta}\int[\Pi_{n=1}^{N}p_{V}(X(n)-H(n)\theta)]p_{H}(H(1),...,H(N))\Pi_{j=1}^{N}dH(j)$

In the special case when the $H(n)^{\prime}s$ are iid, the above reduces to

$\hat{\theta}(N)=argmax_{\theta}N^{-1}\sum_{n=1}^{N}log(\int
p_{V}(X(n)-H\theta)p_{H}(H)dH)$

The problem is to calculate the large deviation properties of
$\hat{\theta}(N)$, namely the rate at which
$P(|\hat{\theta}(N)-\theta|>\delta)$ converges to zero as
$N\rightarrow\infty$.

###  6.1 Large deviation analysis of the RLS algorithm

Consider the problem of estimating $\theta$ from the sequential linear model

$X(n)=H(n)\theta+V(n),n\geq 1$

sequentiall. The estimate based on data collected upto time $N$ is

$\theta(N)=[\sum_{n=1}^{N}H(n)^{T}H(n)]^{-1}[\sum_{n=1}^{N}H(n)^{T}X(n)]$

To cast this algorithm in recursive form, we define

$R(N)=\sum_{n=1}^{N}H(n)^{T}H(n),r(N)=\sum_{n=1}^{N}H(n)^{T}X(n)$

Then,

$R(N+1)=R(N)+H(N+1)^{T}H(N+1),r(N+1)=r(N)+H(N+1)^{T}X(N+1)$

Then,

$R(N+1)^{-1}=R(N)^{-1}-R(N)^{-1}H(N+1)^{T}(I+H(N+1)H(N+1)^{T})^{-1}H(N+1)R(N)^{-1}$

so that

$\theta(N+1)=\theta(N)+K(N+1)(X(N+1)-H(N+1)\theta(N))$

where $K(N+1)$ is a Kalman gain matrix expressible as a function of
$R(N),H(N+1)$. Our aim is to do a large deviation analysis of this recursive
algorithm. Define

$\delta\theta(N)=\theta(N)-\theta$

Then we have

$\delta\theta(N+1)=\delta\theta(N)+K(N+1)(X(N+1)-H(N+1)\theta-H(N+1)\delta\theta(N))$
$=\delta\theta(N)+K(N+1)(V(N+1)-H(N+1)\delta\theta(N))$

We assume that the noise process $V(.)$ is small and perhaps even non-
Gaussian. This fact is emphasized by replacing $V(n)$ with $\epsilon.V(n)$.
Then, the problem is to calculate the relationship between the LDP rate
function of $\delta\theta(N)$ and $\delta\theta(N+1)$.

###  6.2 Problem suggested by my colleague Prof. Dhananjay

Large deviation analysis of magnet with time varying magnetic moment falling
thorugh a coil. Assume that the coil is wound around a tube with $n_{0}$ turns
per unit length extending from a height of $h$ to $h+b$. The tube is
cylindrical and extends from $z=0$ to $z=d$. Taking relativity into account
via the retarded potentials, if $m(t)$ denotes the magnetic moment of the
magnet, then the vector potential generated by it is approximately obtained
from the retarded potential formula as

$A(t,r)=\mu m(t-r/c)\times r/4\pi r^{3}$

where we assume that the magnet is located near the origin. This approximates
to

$A(t,r)=\mu m(t)\times r/4\pi r^{3}-\mu m^{\prime}(t)\times r/4\pi cr^{2}$

Now Assume that the magnet falls through the central axis with its axis always
being oriented along the $-\hat{z}$ direction. Then, after time $t$, let its
$z$ coordinate be $\xi(t)$. We have the equation of motion

$M\xi^{\prime\prime}(t)=F(t)-Mg$

where $F(t)$ is the $z$ component of the force exerted by the magnetic field
generated by the current induced in the coil upon the magnet. Now, the flux of
the magnetic field generated by the magnet through the coil is given by

$\Phi(t)=n_{0}\int_{X_{2}+Y^{2}\leq R_{T}^{2},h\leq z\leq
h+b}B_{z}(t,X,Y,Z)dXdYdZ$

where

$B_{z}(t,X,Y,Z)=\hat{z}.B(t,X,Y,Z),$ $B(t,X,Y,Z)=curlA(t,{\bf
R}-\xi(t)\hat{z})$

where

${\bf R}=(X,Y,Z)$

and where $R_{T}$ is the radius of the tube. The current through the coil is
then

$I(t)=\Phi^{\prime}(t)/R_{0}$

where $R_{0}$ is the coil resistance. The force of the magnetic field
generated by this current upon the falling magnet can now be computed easily.
To do this, we first compute the magnetic field $B_{c}(t,X,Y,Z)$ generated by
the coil using the usual retarded potential formula

$B_{c}(t,X,Y,Z)=curlA_{c}(t,X,Y,Z)$
$A_{c}(t,X,Y,Z)=(\mu/4\pi)\int_{0}^{2n_{0}\pi}(I(t-|{\bf
R}-R_{T}(\hat{x}.cos(\phi)+\hat{y}.sin(\phi))-\hat{z}(b/2n_{0}\pi)\phi|/c).$
$.(-\hat{x}.sin(\phi)+\hat{y}.cos(\phi))/|{\bf
R}-R_{T}(\hat{x}.cos(\phi)+\hat{y}.sin(\phi))-\hat{z}(b/2n_{0}\pi)\phi|)R_{T}d\phi$

Note that $A_{c}(t,X,Y,Z)$ is a function of $\xi(t)$ and also of $m(t)$ and
$m^{\prime}(t)$. It follows that $I(t)$ is a function of
$\xi(t),\xi^{\prime}(t),m(t),m^{\prime}(t),m^{\prime\prime}(t)$. Then we
calculate the interaction energy between the magnetic fields generated by the
magnet and by the coil as

$U_{int}(\xi(t),\xi^{\prime}(t),t)=(2\mu)^{-1}\int(B_{c}(t,X,Y,Z),B(t,X,Y,Z))dXdYdZ$

and then we get for the force exerted by the coil’s magnetic field upon the
falling magnet as

$F(t)=-\partial U_{int}(\xi(t),\xi^{\prime}(t),t)/\partial\xi(t)$

Remark: Ideally since the interaction energy between the two magnetic fields
appears in the Lagrangian $\int(E^{2}/c^{2}-B^{2})d^{3}R/2\mu$, we should
write down the Euler-Lagrange equations using the Lagrangian

$L(t,\xi^{\prime}(t),\xi^{\prime}(t))=M\xi^{\prime}(t)^{2}/2-U_{int}(\xi(t),\xi^{\prime}(t),t)-Mg\xi(t)$

in the form

$\frac{d}{dt}\partial L/\partial\xi^{\prime}-\partial L/\partial\xi=0$

which yields

$M\xi^{\prime\prime}(t)-\frac{d}{dt}\partial U/\partial\xi^{\prime}+\partial
U_{int}/\partial\xi+Mg=0$

###  6.3 Fidelity between two quantum states

Let $\rho,\sigma$ be two quantum states in the same Hilbert space
$\mathcal{H}_{1}$. Let $P(\rho),P(\sigma)$ denote respectively the set of all
purifications of $\rho$ and $\sigma$ w.r.t the same reference Hilbert space
$\mathcal{H}_{2}$. Thus, if $|u>\in\mathcal{H}_{1}\otimes\mathcal{H}_{2}$ is a
purification of $\rho$, ie $|u>\in\mathcal{P}(\rho)$, then

$\rho=Tr_{2}|u><u|$

and likewise for $\sigma$. Then, the fidelity between $\rho$ and $\sigma$ is
defined as

$F(\rho,\sigma)=sup_{|u>\in P(\rho),|v>\in P(\sigma)}|<u|v>|$

Theorem:

$F(\rho,\sigma)=Tr(|\sqrt{\rho}.\sqrt{\sigma}|)$

Proof: Let $|u>\in P(\rho)$. Then, we can write

$|u>=\sum_{i}|u_{i}\otimes w_{i}>$

where $\\{|w_{i}>\\}$ is an onb for $\mathcal{H}_{2}$ and

$\rho=\sum_{i}|u_{i}><u_{i}|$

Likewise if $|v>$ is a purification of $\sigma$, we can write

$|v>=\sum_{i}|v_{i}\otimes w_{i}>$

where

$\sigma=\sum_{i}|v_{i}><v_{i}|$

But then

$<u|v>=\sum_{i}<u_{i}|v_{i}>$

Define the matrices

$X=\sum_{i}|u_{i}><w_{i}|,Y=\sum_{i}|v_{i}><w_{i}|$

Then we have

$\rho=XX^{*},\sigma=YY^{*}$

Hence, by the polar decomposition theorem, we can write

$X=\sqrt{\rho}U,Y=\sqrt{\sigma}V$

where $U,V$ are unitary matrices. Then,

$<u|v>=\sum_{i}<u_{i}|v_{i}>=Tr\sum_{i}|v_{i}><u_{i}|=Tr(YX^{*})=Tr(\sqrt{\sigma}VU^{*}\sqrt{\rho})$
$=Tr(\sqrt{\rho}\sqrt{\sigma}VU^{*})=Tr(\sqrt{\rho}.\sqrt{\sigma}W)$

where $W=VU^{*}$ is a unitary matrix. It is then clear from the singular value
decomposition of $\sqrt{\rho}.\sqrt{\sigma}$ that the maximum of $|<u|v>|$
which equals the maximum of $|Tr(\sqrt{\rho}.\sqrt{\sigma}W)|$ as $W$ varies
over all unitary matrices is simply the sum of the singular values of
$\sqrt{\rho}.\sqrt{\sigma}$, ie,

$F(\rho,\sigma)=Tr|\sqrt{\rho}\sqrt{\sigma}|$
$=Tr[(\sqrt{\rho}\sigma.\sqrt{\rho})^{1/2}]=Tr[(\sqrt{\sigma}.\rho.\sqrt{\sigma})^{1/2}]$

###  6.4 Stinespring’s representation of a TPCP map, ie, of a quantum
operation

Let $K$ be a quantum operation. Thus, for any state $\rho$,

$K(\rho)=\sum_{a=1}^{N}E_{a}\rho.E_{a}^{*},\sum_{a=1}^{N}E_{a}^{*}E_{a}=1$

We claim that $K$ can be represented as

$K(\rho)=Tr_{2}(U(\rho\otimes|u><u|)U^{*})$

where $\rho$ acts in $\mathcal{H}_{1}$, $|u>\in\mathcal{H}_{2}$ and $U$ is a
unitary operator in $\mathcal{H}_{1}\otimes\mathcal{H}_{2}$. To see this, we
define

$V^{*}=[E_{1}^{*},E_{2}^{*},...,E_{N}^{*}]$

Then,

$V^{*}V=I_{p}$

In other words, the columns of $V$ are orthonormal. We assume that

$\rho\in\mathbb{C}^{p\times p}$

so

$V\in\mathbb{C}^{Np\times p}$

Note that

$V\rho V^{*}=((E_{a}\rho E_{b}^{*}))_{1\leq a,b\leq N}$

as an $Np\times Np$ block structured matrix, each block being of size $p\times
p$. Now define

$|u>=[1,0,...,0]^{T}\in\mathbb{C}^{N}$

Then,

$|u><u|\otimes\rho=\left(\begin{array}[]{cc}\rho&0\\\
0&0\end{array}\right)\in\mathbb{C}^{Np\times Np}$

Since the columns of $V$ are orthonormal, we can add columns to it to make it
a square $Np\times Np$ unitary matrix $U^{*}$:

$U=[V,W]\in\mathbb{C}^{Np\times Np},U^{*}U=I_{Np}$

Then,

$U(|u><u|\otimes\rho)U^{*}=[V,W]\left(\begin{array}[]{cc}\rho&0\\\
0&0\end{array}\right)\left(\begin{array}[]{cc}V^{*}\\\
W^{*}\end{array}\right)$ $=V\rho V^{*}$

from which it follows that

$Tr_{1}(U(|u><u|\otimes\rho)U^{*})=Tr_{1}(V\rho V^{*})=\sum_{a}E_{a}\rho
E_{a}^{*}$

Remark: Consider the block structured matrix

${\bf X}=(({\bf B}_{ij}))_{1\leq i,j\leq N}$

where each ${\bf B}_{ij}$ is a $p\times p$ matrix. Then

$Tr_{1}({\bf X})=\sum_{i}{\bf B}_{ii}\in\mathbb{C}^{p\times p}$

and

$Tr_{2}({\bf X})=((Tr({\bf B}_{ij})))_{1\leq i,j\leq N}\in\mathbb{C}^{N\times
N}$

###  6.5 RLS lattice algorithm, statistical performance analysis

The RLS lattice algorithm involves updating the forward and backward
prediction parameters simultaneously in time and order. In order to carry out
a statistical performance analysis, we first simply look at the block
processing problem involving estimating the forward prediction filter
parameter vector ${\bf a}_{N,p}$ of order $p$ based on data collected upto
time $N$. Let

${\bf x}(n)=[x(n),x(n-1),...,x(0)]^{T},z^{-r}{\bf
x}(n)=[x(n-r),x(n-r-1),...,x(0),0,...,0]^{T},r=1,2,...$

where these vectors are all of size $N+1$. Form the data matrix of order $p$
at time $N$:

${\bf X}_{N,p}=[z^{-1}{\bf x}(n),...,z^{-p}{\bf x}(n)]\in\mathbb{R}^{N+1\times
p}$

Then the forward prediction filter coefficient estimate is

${\bf a}_{N,p}=({\bf X}_{N,p}^{T}{\bf X}_{N,p})^{-1}{\bf X}_{N,p}^{T}{\bf
x}(N)$

If $x(n),n\in\mathbb{Z}$ is a zero mean stationary Gaussian process with
autocorrelation $R(\tau)=\mathbb{E}(x(n+\tau)x(n))$, then the problem is to
calculate the statistics of ${\bf a}_{N,p}$. To this end, we observe that

$N^{-1}(z^{-i}{\bf x}(N))^{T}(z^{-j}{\bf x}(N))=$
$N^{-1}\sum_{n=max(i,j)}^{N}x(n-i)x(n-j)\approx R(j-i)$

So we write

$N^{-1}(z^{-i}{\bf x}(N))^{T}(z^{-j}{\bf x}(N))=R(j-i)+\delta R_{N}(j,i)$

where now $\delta R_{N}(j,i)$ is a random variable that is a quadratic
function of the Gaussian process $\\{x(n)\\}$ plus a constant scalar.
Equivalently,

$N^{-1}{\bf X}_{N,p}^{T}{\bf X}_{N,p}={\bf R}_{p}+\delta{\bf R}_{N,p}$

where

${\bf R}_{p}=((R(j-i)))_{1\leq i,j\leq p}$

is the statistical correlation matrix of size $p$ and $\delta{\bf R}_{N,p}$ is
a random matrix. Likewise,

$N^{-1}(z^{-i}{\bf x}(N))^{T}{\bf
x}(N)=N^{-1}\sum_{n=i}^{N}x(n-i)x(n)=R(i)+\delta R_{N}(0,i)$

Thus,

$N^{-1}{\bf X}_{N,p}^{T}{\bf x}(n)={\bf r}(1:p)+\delta{\bf r}_{N}(0,1:p)$

where

${\bf r}(1:p)=[R(1),...,R(p)]^{T},\delta{\bf r}_{N}(1:p)=[\delta
R_{N}(0,1),...,\delta R_{N}(0,p)]^{T}$

###  6.6 Approximate computation of the matrix elements of the principal and
discrete series for $G=SL(2,\mathbb{R})$

These matrix elements $f_{mn}(x,\mu)$ for $\mu$ non-integral are the
irreducible principal series matrix elements and $f_{mm}(x,m)=f_{m}(x)$ for
$k$ integral generate the Discrete series $D_{m}$-modules when acted upon by
$U(\mathfrak{g})$, the universal enveloping algebra of $G$. These matrix
elements satisfy the equations

$f_{mn}(u(\theta_{1})xu(\theta_{2}),\mu)=exp(i(m\theta_{1}+n\theta_{2}))f_{mn}(x,\mu),$
$\Omega f_{mn}(x,\mu)=-\mu^{2}f_{mn}(x,\mu)$

the second of which, in view of the first and the expression of the Casimir
operator $\Omega$ in terms of $t,\theta_{1},\theta_{2}$ in the singular value
decomposition $x=u(\theta_{1})a(t)u(\theta_{2})$ simplifies to an equation of
the form

$d^{2}f_{mn}(t,\mu)/dt^{2}-q_{mn}(t)f_{mn}(t,\mu)+\mu^{2}f_{mn}(t,\mu)=0$

where $f_{mn}(t,\mu)=f_{mn}(a(t),\mu)$. This determines the matrix elements of
the principal series. For the discrete series matrix elements, we observe that
these are

$\phi_{kmn}(x)=<\pi_{k}(x)e_{m}|e_{n}>$

where $\pi_{k}$ is the reducible principal series for integral $k$. Replacing
$k$ by $-k$ where $k=1,2,...$, the discrete series matrix elements are

$\phi_{-kmn}(x)=<\pi_{-k}(x)e_{m}|e_{n}>,m,n=-k-1,-k-2,...$

with

$\pi(X)e_{-k-1}=0$

Note that we can using the method of lowering operators, write

$\pi_{-k}(Y)^{r}e_{-k-1}=c(k,r)e_{-k-1-r},r=0,1,2,...$

where $c(k,r)$ are some real constants. Thus, writing

$f_{k+1}(x)=<\pi_{-k}(x)e_{-k-1}|e_{-k-1}>,x\in G,$

we have for the discrete series matrix elements,

$\phi_{-k,-k-1-r,-k-1-s}(x)=<\pi_{-k}(xY^{r})e_{-k-1}|\pi_{-k}(Y^{s})e_{-k-1}>$
$=<\pi_{-k}(Y^{*s}xY^{r})e_{-k-1}|e_{-k-1}>=f_{k+1}(Y^{*s};x;Y^{r})$

In this way, all the discrete series matrix elements can be determined by
applying differential operators on $f_{k+1}(x)$.

Now given an image field $h(x)$ on $SL(2,\mathbb{R})$, in order to extract
invariant features, or to estimate the group transformation element parameter
from a noisy version of it, we must first evaluate its matrix elements w.r.t
the discrete and principal series, ie,

$\int_{G}h(x)\phi_{-k,-k-1-r,-k-1-s}(x)dx,\int_{G}h(x)f_{mn}(x,\mu)dx$

Now,

$\int_{G}h(x)\phi_{-k,-k-1-r,-k-1-s}(x)dx=\int_{G}h(x)f_{k+1}(Y^{*s};x;Y^{r})dx$
$=\int_{G}h(Y^{s};x;Y^{*r})f_{k+1}(x)dx$

###  6.7 Proof of the converse part of the Cq coding theorem

Let $\phi(i),i=1,2,...,N$ be a code. Note that the size of the code is $N$.
Let $s\leq 0$ and define

$p_{s}=argmax_{p}I_{s}(p,W)$

where

$I_{s}(p,W)=min_{\sigma}D_{s}(p\otimes W|p\otimes\sigma)$

where

$p\otimes W=diag[p(x)W(x),x\in A],p\otimes\sigma=diag[p(x)sigma,x\in A]$

Note that

$D_{s}(p\otimes W|p\otimes\sigma]=log(Tr((p\otimes
W)^{1-s}(p\otimes\sigma)^{s})/(-s)$ $=(1/-s)\log
sum_{x}Tr(p(x)W(x))^{1-s}(p(x)\sigma)^{s})$
$=(1/-s)log\sum_{x}p(x)Tr(W(x)^{1-s}\sigma^{s})$

so that

$lim_{s\rightarrow 0}D_{s}(p\otimes W|p\otimes\sigma)=$
$\sum_{x}p(x)Tr(W(x)log(W(x))-Tr(W_{p}.log(\sigma))$

where

$W_{p}=\sum_{x}p(x)W_{x}$

Note that this result can also be expressed as

$lim_{s\rightarrow 0}D_{s}(p\otimes
W|p\otimes\sigma)=\sum_{x}p(x)D(W(x)|\sigma)$

We define

$f(t)=Tr(tW_{x}^{1-s}+(1-t)\sum_{y}p_{s}(y)W_{y}^{1-s})^{1/(1-s)},t\geq 0$

Note that

$I_{s}(p,W)=(Tr(\sum_{y}p(y)W_{y}^{1-s})^{1/(1-s)})^{1-s}$

and since by definition of $p_{s}$, we have

$I_{s}(p_{s},W)\geq I_{s}(p,W)\forall p,$

it follows that

$f(t)\leq f(0)\forall t\geq 0$

Thus

$f^{\prime}(0)\leq 0$

This gives

$Tr[(W_{x}^{1-s}-\sum_{y}p_{s}(y)W_{y}^{1-s}).(\sum_{y}p_{s}(y)W_{y}^{1-s})^{s/(1-s)}]\leq
0$

or equivalently,

$Tr(W_{x}^{1-s}.(\sum_{y}p_{s}(y)W_{y}^{1-s})^{s/(1-s)})\leq
Tr(\sum_{y}p_{s}(y)W_{y}^{1-s})^{1/(1-s)}$

Now define the state

$\sigma_{s}=(\sum_{y}p_{s}(y)W_{y}^{1-s})^{1/(1-s)}/Tr(\sum_{y}p_{s}(y)W_{y}^{1-s})^{1/(1-s)}$

Then we can express the above inequality as

$Tr(W_{x}^{1-s}\sigma_{s}^{s})\leq[Tr(\sum_{y}p_{s}(y)W_{y}^{1-s})^{1/(1-s)}]^{1-s}$

Remark: Recall that

$D_{s}(p\otimes
W|p\otimes\sigma]=(1/-s)log\sum_{x}p(x)Tr(W_{x}^{1-s}\sigma^{s})$
$=(1/-s)log(Tr(A\sigma^{s})$

where

$A=\sum_{x}p(x)W_{x}^{1-s}$

Note that we are all throughout assuming $s\leq 0$. Application of the reverse
Holder inequality gives for all states $\sigma$

$Tr(A\sigma^{s})\geq(Tr(A^{1/(1-s)}))^{1-s}.(Tr(\sigma))^{s}=(Tr(A^{1/(1-s)}))^{1-s}$

with equality iff $\sigma$ is proportional to $A^{1/(1-s)}$, ie, iff

$\sigma=A^{1/(1-s)}/Tr(A^{1/(1-s)})$

Thus,

$min_{\sigma}D_{s}(p\otimes W|p\otimes\sigma)=$
$(-1/s)[Tr(\sum_{x}p(x)W_{x}^{1-s})^{1/(1-s)}]^{1-s}$

The maximum of this over all $p$ is attained when $p=p_{s}$ and we denote the
corresponding value of $\sigma$ by $\sigma_{s}$. Thus,

$\sigma_{s}=\frac{(\sum_{x}p_{s}(x)W_{x}^{1-s})^{1/(1-s)}}{Tr(\sum_{x}p_{s}(x)W_{x}^{1-s})^{1/(1-s)}}$

Now let $\phi(i),i=1,2,...,N$ be a code with $\phi(i)\in A^{n}$. Let
$Y_{i},i=1,2,...,N$ be detection operators, ie, $Y_{i}\geq 0,\sum_{i}Y_{i}=1$.
Let $\epsilon(\phi)$ be the error probability corresponding to this code and
detection operators. Then

$1-\epsilon(\phi)=N^{-1}\sum_{i=1}^{N}Tr(W_{\phi(i)}Y_{i})$
$=N^{-1}Tr(S(\phi)T)$

where

$S(\phi)=N^{-1}diag[W_{\phi(i)},i=1,2,...,N],T=diag[Y_{i},i=1,2,...,N]$

Also define for any state $\sigma$, the state

$S(\sigma)=N^{-1}diag[\sigma_{n},...,\sigma_{n}]=N^{-1}.I_{N}\otimes\sigma_{n}$

We observe that

$Tr(S(\sigma_{n})T)==N^{-1}\sum_{i}Tr(\sigma_{n}Y_{i})=Tr(\sigma.N^{-1}.\sum_{i=1}^{N}Y_{i})$
$=N^{-1}Tr(\sigma_{n})=1/N$

where $\sigma_{n}=\sigma^{\otimes n}.$ Then for $s\leq 0$ we have by
monotonicity of quantum Renyi entropy, (The relative entropy between the
pinched states cannot exceed the relative entropy between the original states)

$N^{-s}(1-\epsilon(\phi))^{1-s}=(Tr(S(\phi)T))^{1-s}(Tr(S(\sigma_{ns})T))^{s}$
$\leq(Tr(S(\phi)T))^{1-s}(Tr(S(\sigma_{ns})T))^{s}+(Tr(S(\phi)(1-T)))^{1-s}(Tr(S(\sigma_{ns})(1-T)))^{s}$
$\leq Tr(S(\phi)^{1-s}S(\sigma_{ns})^{s})$

Note that the notation $\sigma_{ns}=\sigma_{s}^{\otimes n}$ is being used.
Now, we have

$(Tr(S(\phi)^{1-s}S(\sigma_{ns})^{s}))=(N^{-1}\sum_{i}Tr(W_{\phi(i)}^{1-s}\sigma_{ns}^{s}))$
$log(Tr(W_{\phi(i)}^{1-s}\sigma_{ns}^{s})$
$=\sum_{l=1}^{n}log(Tr(W_{\phi_{l}(i)}^{1-s}\sigma_{s}^{s}))$
$=n.n^{-1}\sum_{l=1}^{n}log(Tr(W_{\phi_{l}(i)}^{1-s}\sigma_{s}^{s}))$ $\leq
n.log(n^{-1}.\sum_{l=1}^{n}Tr(W_{\phi_{l}(i)}^{1-s}\sigma_{s}^{s}))$ $\leq
n.log(Tr(\sum_{y}p_{s}(y)W_{y}^{1-s})^{1/(1-s)}]^{1-s})$
$=n(1-s).log(Tr(\sum_{x}p_{s}(x)W_{x}^{1-s})^{1/(1-s)})$

Thus,

$Tr(S(\phi)^{1-s}S(\sigma_{ns})^{s})\leq(Tr(\sum_{x}p_{s}(x)W_{x}^{1-s})^{1/(1-s)})^{n(1-s)}$

and hence

$N^{-s}(1-\epsilon(\phi))^{1-s}\leq(Tr(\sum_{x}p_{s}(x)W_{x}^{1-s})^{1/(1-s)})^{n(1-s)}$

or equivalently,

$1-\epsilon(\phi)\leq
exp((s/(1-s))log(N)+n.log(Tr(\sum_{x}p_{s}(x)W_{x}^{1-s})^{1/(1-s)}))$
$=exp((s/(1-s))log(N)+n.log(Tr(\sum_{x}p_{s}(x)W_{x}^{1-s})^{1/(1-s)}))$
$=exp(ns((1-s)log(N)/n+s^{-1}log(Tr(\sum_{x}p_{s}(x)W_{x}^{1-s})^{1/(1-s)}))---(a)$

where $s\leq 0$. Now, for any probability distribution $p(x)$ on $A$, we have

$d/dsTr[(\sum_{x}p(x)W_{x}^{1-s})^{1/(1-s)}]=$
$(1/(1-s)^{2})Tr[(\sum_{x}p(x)W_{x}^{1-s})^{1/(1-s)}.log(\sum_{x}p(x)W_{x}^{1-s})]$
$-(1/(1-s))[Tr(\sum_{x}p(x)W_{x}^{1-s}log(W_{x}))].[Tr(\sum_{x}p(x)W_{x}^{1-s})^{s/(1-s)}]$

and as $s\rightarrow 0$, this converges to

$d/dsTr[(\sum_{x}p(x)W_{x}^{1-s})^{1/(1-s)}]|_{s=0}=$
$\sum_{x}p(x)Tr(W_{x}.(log(W_{p})-log(W_{x})))=-\sum_{x}p(x)D(W_{x}|W_{p})$
$=-(H(W_{p})-\sum_{p}(x)H(W_{x}))=-(H(Y)-H(Y|X))=-I_{p}(X,Y)$

where

$W_{p}=\sum_{x}p(x)W_{x}$

Write $N=N(n)$ and define

$R=limsup_{n}log(N(n))/n,\phi=\phi_{n}$

Also define

$I_{s}(X,Y)=-(1/s(1-s))log(Tr(\sum_{x}p_{s}(x)W_{x}^{1-s})^{1/(1-s)}))$

Then, we’ve just seen that

$lim_{s\rightarrow 0}I_{s}(X,Y)=I_{0}(X,Y)\leq I(X,Y)$

where $I_{0}(X,Y)=I_{p_{0}}(X,Y)$ and $I(X,Y)=sup_{p}I_{p}(X,Y)$. and

$1-\epsilon(\phi_{n})\leq exp((ns/(1-s))(log(N(n))/n-I_{s}(X,Y)))$

It thus follows that if

$R>I(X,Y)$

then by taking $s<0$ and $s$ sufficiently close to zero we would have

$R>I_{s}(X,Y)$

and then we would have

$limsup_{n}(1-\epsilon(\phi_{n}))\leq
limsup_{n}exp((ns/(1-s))(R-I_{s}(X,Y))=0$

or equivalently,

$liminf_{n}\epsilon(\phi_{n})=1$

This proves the converse of the Cq coding theorem, namely, that if the rate of
information transmission via any coding scheme is more that the Cq capacity
$I(X,Y)$, then the asymptotic probability of error in decoding using any
sequence of decoders will always converge to unity.

###  6.8 Proof of the direct part of the Cq coding theorem

Our proof is based on Shannon’s random coding method. Consider the same
scenario as above. Define

$\pi_{i}=\\{W_{\phi(i)}\geq 2NW_{p_{n}}\\},i=1,2,...,N$

where $N=N(n)$, $\phi(i)\in A^{n}$ with $\phi(i),i=1,2,...,N$ being iid with
probability distribution $p_{n}=p^{\otimes n}$ on $A^{n}$. Define the
detection operators

$Y(\phi(i))=(\sum_{j=1}^{N}\pi)^{-1/2}.\pi_{i}.(\sum_{j=1}^{N}\pi_{j})^{-1/2},i=1,2,...,N$

Then $0\leq Y_{i}\leq 1$ and $\sum_{i=1}^{N}Y_{i}=1$. Write

$S=\pi_{i},T=\sum_{j\neq i}\pi_{j}$

Then,

$Y(\phi(i))=(S+T)^{-1/2}S(S+T)^{-1/2},0\leq S\leq 1,T\geq 0$

and hence we can use the inequality

$1-(S+T)^{-1/2}S(S+T)^{-1/2}\leq 2-2S+4T$

Then,

$1-Y(\phi(i))=\leq 2-2\pi_{i}+4\sum_{j:j\neq i}\pi_{j}$

and hence

$\epsilon(\phi)=N^{-1}\sum_{i=1}^{N}Tr(W_{\phi(i)}(1-Y(\phi(i))))$ $\leq
N^{-1}\sum_{i=1}^{N}Tr(W(\phi(i))(2-2\pi_{i})+4N^{-1}\sum_{i\neq
j}Tr(W(\phi(i))\pi_{j})$

Note that since for $i\neq j$, $\phi(i)$ and $\phi(j)$ are independent random
variables in $A^{n}$, it follows that $W(\phi(i))$ and $\pi_{j}$ are also
independent operator valued random variables. Thus,

$\mathbb{E}(\epsilon(\phi))\leq(2/N)\sum_{i=1}^{N}\mathbb{E}(Tr(W(\phi(i))\\{W(\phi(i))\leq
2NW_{p_{n}}\\}))$ $+(4/N)\sum_{i\neq
j,i,j=1,2,...,N}Tr(\mathbb{E}(W(\phi(i))).\mathbb{E}\\{W(\phi(j))>2NW_{p_{n}}\\})$
$=2\sum_{x_{n}\in A^{n}}p_{n}(x_{n})Tr(W(x_{n})\\{W(x_{n})\leq
2N.W(p_{n})\\})$ $+4(N-1)\sum_{x_{n},y_{n}\in
A^{n}}p_{n}(x_{n})p_{n}(y_{n})Tr(W(x_{n})\\{W(y_{n})>2NW(p_{n})\\})$
$=(2/N)\sum_{x_{n}}p_{n}(x_{n})Tr(W(x_{n})\\{W(x_{n})\leq 2N.W(p_{n})\\})$
$+4(N-1)\sum_{y_{n}\in
A^{n}}p_{n}(y_{n})Tr(W(p_{n})\\{W(y_{n})>2NW(p_{n})\\})$ $\leq
2\sum_{x_{n}}p_{n}(x_{n})Tr(W(x_{n})^{1-s}(2NW(p_{n}))^{s})$
$+4N\sum_{y_{n}}p_{n}(y_{n})Tr(W(y_{n})^{1-s}W(p_{n})^{s}(2N)^{s-1})$

for $0\leq s\leq 1$. This gives

$\mathbb{E}(\epsilon(\phi_{n}))\leq$ $2^{s+2}N^{s}\sum_{x_{n}\in
A^{n}}p_{n}(x_{n})Tr(W(x_{n})^{1-s}W(p_{n})^{s})$ $=2^{s+2}N^{s}[\sum_{x\in
A}p(x)Tr(W(x)^{1-s}W(p)^{s})]^{n}$
$=2^{s+2}.exp(sn(log(N(n)/n+s^{-1}.log(\sum_{x}p(x)Tr(W(x)^{1-s}W(p)^{s}))))$

###  6.9 Induced representations

Let $G$ be a group and $H$ a subgroup. Let $\sigma$ be a unitary
representation of $H$ in a Hilbert space $V$. Let $\mathcal{F}$ be the space
of all $f:G\rightarrow V$ for which

$f(gh)=\sigma(h)^{-1}f)(g),h\in H,g\in G$

Then define

$U(g)f(x)=\rho(g,g^{-1}xH)f(g^{-1}x),g,x\in G,f\in\mathcal{F}$

where

$\rho(g,x)=(d\mu(xH)/d\mu(gxH))^{1/2},g,x\in G$

with $\mu$ a measure on $G/H$. $U$ is called the representation of $G$ induced
by the representation $\sigma$ of $H$. We define

$\parallel f\parallel=\int_{G}|f(g)|^{2}d\mu(g)$

Then, unitarity of $U(g)$ is easy to see:

$\int_{G/H}\parallel U(g)f(x)\parallel^{2}d\mu(x)=$
$\int_{G/H}(\rho(g,g^{-1}xH))^{2}|f(g^{-1}x)|^{2}d\mu(xH)=$
$\int(d\mu(g^{-1}xH)/d\mu(xH))|f(g^{-1}x)|^{2}d\mu(xH)=\int_{G/H}|f(g^{-1}x)|^{2}d\mu(g^{-1}xH)$
$=\int_{G/H}|f(x)|^{2}d\mu(xH)$

proving the claim. Note that

$(\rho(g_{1},g_{2}xH)\rho(g_{2},xH))^{2}=(d\mu(g_{2}xH)/d\mu(g_{1}g_{2}xH))(d\mu(xH)/d\mu(g_{2}xH))=$
$d\mu(xH)/d\mu(g_{1}g_{2}xH)=\rho(g_{1}g_{2},xH)^{2}$

ie

$\rho(g_{1}g_{2},xH)=\rho(g_{1},g_{2}xH)\rho(g_{2},xH),g_{1},g_{2},x\in G$

ie, $\rho$ is a scalar cocycle.

Remark: $|f(x)|^{2}$ can be regarded a function on $G/H$ since it assumes the
same value on $xH$ for any $x\in G$. Indeed for $x\in G,h\in H$,
$|f(xh)|^{2}=|\sigma(h)^{-1}f(x)|^{2}=|f(x|^{2}$ since $\sigma(h)$ is a
unitary operator.

There is yet another way to describe the induced representation, ie, in a way
that yields a unitary representation of $G$ that is equivalent to $U$. The
method is as follows: Let $X=G/H$ and let $\mathcal{F}_{1}$ be the space of
all square integrable functions $f:X\rightarrow V$ w.r.t the measure $\mu_{X}$
induced by $\mu$ on $X$. Let $\gamma$ be any cocycle of $(G,X)$ with values in
the space of unitary operators on $V$, ie,

$\gamma:G\times X\rightarrow\mathcal{U}(V)$

satisfies

$\gamma(g_{1}g_{2},x)=\gamma(g_{1},g_{2}x)\gamma(g_{2},x),g_{1},g_{2}\in
G,x\in X$

Then define a representation $U_{1}$ of $G$ in $\mathcal{F}_{1}$ by

$U_{1}(g)f(x)=\rho(g,g^{-1}x)\gamma(g,g^{-1}x)f(g^{-1}x),g\in G,x\in
X,f\in\mathcal{F}_{1}$

Let $s:X\rightarrow G$ be a section of $X=G/H$ such that $s(H)=e$. This means
that for any $x\in X,s(x)H=x$, ie, $s(x)$ belong to the coset $x$. Define

$b(g,x)=s(gx)^{-1}gs(x)\in G,g\in G,x\in X$

Then,

$b(g,x)H=s(gx)^{-1}gx=H$

because

$gs(x)H=gx,s(gx)H=gx$

Now observe that

$b(g_{1},g_{2}x)b(g_{2},x)=s(g_{1}g_{2}x)^{-1}g_{1}s(g_{2}x)s(g_{2}x)^{-1}g_{2}s(x)=s(g_{1}g_{2}x)^{-1}g_{1}g_{2}s(x)$
$=b(g_{1}g_{2},x),g_{1},g_{2}\in G,x\in X$

implying that $\gamma=\sigma(b)$ is a $(G,X,V)$ cocycle, ie,

$\gamma(g_{1}g_{2},x)=\sigma(b(g_{1}g_{2},x))=\sigma(b(g_{1},g_{2}x)b(g_{2},x))$
$=\sigma(b(g_{1},g_{2}x))\sigma(b(g_{2},x))=\gamma(g_{1},g_{2}x)\gamma(g_{2},x)$

with $\gamma$ assuming values in the set of unitary operators in $V$. Now let

$f(g)=\gamma(g,H)^{-1}f_{1}(gH)=T(f_{1})(g)\in V,f_{1}\in\mathcal{F}_{1},g\in
G$

Then

$U(g)T(f_{1})(g_{1})=U(g)f(g_{1})=\rho(g,g^{-1}g_{1}H)f(g^{-1}g_{1})$
$=\rho(g,g^{-1}g_{1}H)\gamma(g^{-1}g_{1},H)^{-1}f_{1}(g^{-1}g_{1}H)$
$=\rho(g,g^{-1}g_{1}H)(\gamma(g^{-1},g_{1}H)\gamma(g_{1},H))^{-1}f_{1}(g^{-1}g_{1}H)$

On the other hand,

$T(U_{1}(g)f_{1})(g_{1})=\gamma(g_{1},H)^{-1}U_{1}(g)f_{1}(g_{1}H)$
$=\gamma(g_{1},H)^{-1}\rho(g,g^{-1}g_{1}H)\gamma(g,g^{-1}g_{1}H)f_{1}(g^{-1}g_{1}H)$
$=\rho(g,g^{-1}g_{1}H)\gamma(g_{1},H)^{-1}\gamma(g,g^{-1}g_{1}H)f_{1}(g^{-1}g_{1}H)$

Now,

$\gamma(g_{1},H)=\gamma(gg^{-1}g_{1},H)=\gamma(g,g^{-1}g_{1}H)\gamma(g^{-1}g_{1},H)$

or equivalently,

$\gamma(g_{1},H)^{-1}\gamma(g,g^{-1}g_{1}H)=\gamma(g^{-1}g_{1},H)^{-1}$

Thus,

$T(U_{1}(g)f_{1})(g_{1})=\rho(g,g^{-1}g_{1}H)\gamma(g^{-1}g_{1},H)^{-1}f_{1}(g^{-1}g_{1}H)$
$=\rho(g,g^{-1}g_{1}H)T(f_{1})(g^{-1}g_{1})=U(g)T(f_{1})(g_{1})$

In other words,

$TU_{1}(g)=U(g)T$

Thus, $U_{1}$ and $U$ are equivalent unitary representations.

###  6.10 Estimating non-uniform transmission line parameters and line voltage
and current using the extended Kalman filter

Abstract: The distributed parameters of a non-uniform transmission line are
assumed to depend on a finite set of unknown parameters, for example their
spatial Fourier series coefficients. The line equations are setup taking
random voltage and current line loading into account with the random loading
being assumed to be white Gaussian noise in the time and spatial variables. By
passing over to the spatial Fourier series domain, these line stochastic
differential equations are expressed as a truncated sequence of coupled
stochastic differential equations for the spatial Fourier series components of
the line voltage and current. The coefficients that appear in these stochastic
differential equations are the spatial Fourier series coefficients of the
distributed line parameters and these coefficients in turn are linear
functions of the unknown parameters. By separating out these complex
stochastic differential equations into their real and imaginary part, the line
equations become a finite set of sde’s for the extended state defined to be
the real and imaginary components of the line voltage and current as well as
the parameters on which the distributed parameters depend. The measurement
model is assumed to be white Gaussian noise corrupted versions of the line
voltage and current sampled at a discrete set of spatial points on the line
and this measurement model can be expressed as a linear transformation of the
state variables, ie, of the spatial Fourier series components of the line
voltage and current plus a white measurement noise vector. The extended Kalman
filter equations are derived for this state and measurement model and our real
time line voltage and current and distributed parameter estimates show
excellent agreement with the true values based on MATLAB simulations.

###  6.11 Multipole expansion of the magnetic field produced by a time varying
current distribution

Let $J(t,r)$ be the current density. The magnetic vector potential generated
by it is given by

$A(t,r)=(\mu/4\pi)\int
J(t-|r-r^{\prime}|/c,r^{\prime})d^{3}r^{\prime}/|r-r^{\prime}|$
$=(\mu/4\pi)\sum_{n\geq
0}(1/c^{n}n!)\int(\partial_{t}^{n}J(t,r^{\prime}))|r-r^{\prime}|^{n-1}d^{3}r^{\prime}$

For $|r|$ greater than the source dimension and with $\theta$ denoting the
angle between the vectors $r$ and $r^{\prime}$, we have the expansion

$|r-r^{\prime}|^{\alpha}=(r^{2}+r^{{}^{\prime}2}-2r.r^{\prime})^{\alpha/2}$
$=r^{\alpha}(1+(r^{\prime}/r)^{2}-2r.r^{\prime}/r^{2})^{\alpha/2}=r^{\alpha}(1+(r^{\prime}/r)^{2}-2(r^{\prime}/r)cos(\theta))^{\alpha/2}$
$=r^{\alpha}\sum_{m\geq 0}P_{m,\alpha}(cos(\theta))(r^{\prime}/r)^{m}$

where the generating function of $P_{m,\alpha}(x),m=0,1,2,...$ is given by
$(1+x^{2}-2x.cos(\theta))^{\alpha/2}$. Then,

$A(t,r)=(\mu/4\pi)\sum_{n,m\geq
0}(1/c^{n}n!)r^{n-1-m}\int\partial_{t}^{n}J(t,r^{\prime})r{{}^{\prime}m}P_{n-1}(r.r^{\prime}/rr^{\prime})d^{3}r^{\prime}$

A useful approximation to this exact expansion is obtained by taking

$|r-r^{\prime}|^{\alpha}\approx(r^{2}-2r.r^{\prime})^{\alpha/2}\approx
r^{\alpha}(1-\alpha r.r^{\prime}/r^{2})$
$=r^{\alpha}-\alpha.r^{\alpha-2}r.r^{\prime}$

Further,

$\partial_{t}^{n}J(t,r^{\prime})(r.r^{\prime})=(r^{\prime}\times\partial_{t}^{n}J(t,r^{\prime}))\times
r+(\partial_{t}^{n}J(t,r^{\prime}).r)r^{\prime}$

In the special case when the source is a wire carrying a time varying current
$I(t)$, we can write

$J(t,r^{\prime})d^{3}r^{\prime}=I(t)dr^{\prime}$

and then the above equation gets replaced by

##  Chapter 7 Some aspects of stochastic processes

End semester exam, M.Tech Ist semester, Stochastic processes and queueing
theory
Attempt any five questions, each question carries ten marks

[1] In a single server queue, let the arrivals take place at the times
$S_{n}=\sum_{k=1}^{n}X_{k},n\geq 1$, let the service time of the $n^{th}$
customer be $Y_{n}$ and let his waiting time in the queue be $W_{n}$. Derive
the recursive relation

$W_{n+1}=max(0,W_{n}+Y_{n}-X_{n+1}),n\geq 1$

where $W_{1}=0$. If the $X_{n}^{\prime}s$ are iid and the $Y_{n}^{\prime}s$
are also iid and independent of $\\{X_{n}\\}$, then define
$T_{n}=Y_{n}-X_{n+1}$ and prove that $W_{n+1}$ has the same distribution as
that of $max_{0\leq k\leq n}F_{k}$ where $F_{n}=\sum_{k=1}^{n}T_{k},n\geq 1$
and $F_{0}=0$. When $X_{n},Y_{n}$ are respectively exponentially distributed
with parameters $\lambda,\mu$ respectively explain the sequence of steps based
on ladder epochs and ladder heights by which you will calculate the
distribution of the equilibrium waiting time
$W_{\infty}=lim_{n\rightarrow\infty}W_{n}$ in the case when $\mu>\lambda$.
What happens when $\mu\leq\lambda$ ?

Solution: The $(n+1)^{th}$ customer arrives at the time $S_{n+1}$. The
previous, ie, the $n^{th}$ customer departs at time $S_{n}+W_{n}+Y_{n}$. If
this time of his departure is smaller than $S_{n+1}$, (ie,
$S_{n}+W_{n}+Y_{n}-S_{n+1}=W_{n}+Y_{n}-X_{n+1}<0$), then the $(n+1)^{th}$
customer finds the queue empty on arrival and hence his waiting time
$W_{n+1}=0$. If on the other hand, the time of departure $S_{n}+W_{n}+Y_{n}$
of the $n^{th}$ customer is greater than $S_{n+1}$, the arrival time of the
$(n+1)^{th}$ customer, then the $n+1^{th}$ customer has to wait for a time

$W_{n+1}=(S_{n}+W_{n}+Y_{n})-S_{n+1}=W_{n}+Y_{n}-X_{n+1}$

before his service commences. Therefore, we have the recursion relation

$W_{n+1}=max(0,W_{n}+Y_{n}-X_{n+1}),n\geq 0$

Note that $W_{1}=0$. This can be expressed as

$W_{n+1}=max(0,W_{n}+T_{n}),T_{n}-Y_{n}-X_{n+1}$

By iteration, we find

$W_{2}=max(0,T_{1}),W_{3}=max(0,T_{2}+max(0,T_{1}))=max(0,max(T_{2},T_{1}+T_{2}))=max(0,T_{2},T_{1}+T_{2})$
$W_{4}=max(0,W_{3}+T_{3})=max(0,T_{3}+max(0,max(T_{2},T_{1}+T_{2}))=max(0,max(T_{3},max(T_{2}+T_{3},T_{1}+T_{2}+T_{3}))))$
$=max(0,T_{3},T_{2}+T_{3},T_{1}+T_{2}+T_{3})$

Continuing, we find by induction that

$W_{n+1}=max(0,\sum_{k=r}^{n}T_{k},r=1,2,...,n)$

Now the $T_{k}^{\prime}s$ are iid. Therefore, $W_{n+1}$ has the same
distribution as

$max(0,\sum_{k=r}^{n}T_{n+1-k},r=1,2,...,n)$
$=max(0,\sum_{k=1}^{r}T_{k},r=1,2,...,n)$

In particular, the equilibrium distribution of the waiting time, ie of
$W_{\infty}=lim_{n\rightarrow\infty}W_{n}$ is the same as that of

$max(0,\sum_{k=1}^{\infty}T_{k})$

provided that this limiting r.v. exists, ie, the provided that the infinite
series converges w.p.1. Let $F_{n}=\sum_{k=1}^{n}T_{k},n\geq 1,F_{0}=0$

Define $\tau_{1}$ denote the first time $k\geq 1$ at which $F_{k}>F_{0}=0$.
For $n\geq 1$, let $\tau_{n+1}$ denote the first time $k\geq 1$ at which
$F_{k}>F_{\tau_{n}}$, ie, the $n+1^{th}$ time $k$ at which $F_{k}$ exceeds all
of $F_{j},j=0,1,...,k-1$. Then, it is clear that $W_{\infty}$ has the same
distribution as that of $limF_{\tau_{n}}$, provided that $\tau_{n}$ exists for
all $n$. In the general case, we have

$P(W_{\infty}\leq x)=lim_{n\rightarrow\infty}P(F_{\tau_{n}}\leq x)$

where if for a given $\omega$, $\tau_{N}(\omega)$ exists for some positive
integer $N=N(\omega)$ but $\tau_{N+1}(\omega)$ does not exist, then we set
$\tau_{k}(\omega)=\tau_{N}(\omega)$ for all $k>N$. It is also clear using
stopping time theory, that the r.v’s
$U_{n+1}=F_{\tau_{n+1}}-F_{\tau_{n}},n=1,2,...$ are iid and non-negative and
we can write

$W_{\infty}=\sum_{n=0}^{\infty}U_{n},U_{0}=0$

Thus, the distribution of $W_{\infty}$ is formally the limit of $F_{U}^{*n}$
where $F_{U}$ is the distribution of $U_{1}=F_{\tau_{1}}$ with
$\tau_{1}=min(k\geq 1,F_{k}>0)$. Note also that for $x>0$,

$1-F_{U}(x)=P(U>x)=\sum_{n=0}^{\infty}P(max_{k\leq n}F_{k}\leq x<F_{n+1})$

[2] Consider a priority queue with two kinds of customers labeled
$C_{1},C_{2}$ and a single server. Customers of type $C_{1}$ arrive at the
rate $\lambda_{1}$ and customers of type $C_{2}$ arrive at the rate
$\lambda_{2}$. A customer of type $C_{1}$ is served at the rate $\mu_{1}$
while a customer of type $C_{2}$ is served at the rate $\mu_{2}$. If a
customer of type $C_{1}$ is present in the queue, then his service takes
immediate priority over a customer of type $C_{2}$, otherwise the queue runs
on a first come first served basis. Let $X_{1}(t)$ denote the number of type
$C_{1}$ customers and $X_{2}(t)$ the number of type $C_{2}$ customers in the
queue at time $t$. Set up the Chapman-Kolmogorov differential equations for

$P(t,n_{1},n_{2})=Pr(X_{1}(t)=n_{1},X_{2}(t)=n_{2}),n_{1},n_{2}=0,1,...$

and also set up the equilibrium equations for

$P(n_{1},n_{2})=lim_{t\rightarrow\infty}P(t,n_{1},n_{2})$

Solution: Let $\theta(n)=1ifn\geq 0$ and $\theta(n)=0$ if $n<0$ and let
$\delta(n,m)=1$ if $n=m$ else $\delta(n,m)=0$.

$P(t+dt,n_{1},n_{2})=P(t,n_{1}-1,n_{2})\lambda_{1}dt.\theta(n_{1}-1)+P(t,n_{1},n_{2}-1)\lambda_{2}dt.\theta(n_{2}-1)+$
$P(t,n_{1}+1,n_{2})\mu_{1}dt+P(t,n_{1},n_{2}+1)\mu_{2}.dt.\delta(n_{1},0)$
$+P(t,n_{1},n_{2})(1-(\lambda_{1}+\lambda_{2})dt-\mu_{1}.dt.\theta(n_{1}-1)-\mu_{2}dt.\theta(n_{2}-1).\delta(n_{1},0))$

Essentially, the term $\delta(n_{1},0)$ has been inserted to guarantee that
the $C_{2}$ type of customer is currently being served only when there is no
$C_{1}$ type of customer in the queue and the term $\theta(n_{2}-1)$ term has
been inserted to guarantee that the $C_{2}$ type of customer can depart from
the queue only if there is at least one such in the queue. The product
$\delta(n_{1},0)\theta(n_{2}-1)$ term corresponds to the situation in which
there is no $C_{1}$ type of customer and at least one $C_{2}$ type of customer
which means that a $C_{2}$ type of customer is currently being served and
hence will depart immediately after his service is completed. From these
equations follow the Chapman-Kolmogorov equations:

$\partial_{t}P(t,n_{1},n_{2})=$
$P(t,n_{1}-1,n_{2})\lambda_{1}\theta(n_{1}-1)+P(t,n_{1},n_{2}-1)\lambda_{2}.\theta(n_{2}-1)+$
$P(t,n_{1}+1,n_{2})\mu_{1}+P(t,n_{1},n_{2}+1)\mu_{2}.\delta(n_{1},0)$
$-P(t,n_{1},n_{2})(\lambda_{1}+\lambda_{2})-\mu_{1}.\theta(n_{1}-1)-\mu_{2}.\theta(n_{2}-1).\delta(n_{1},0))$

The equilibrium equations are obtained by setting the rhs to zero, ie
$\partial_{t}P(n_{1},n_{2})=0$.

[3] Write short notes on any three of the following:

[a] Construction of Brownian motion on $[0,1]$ from the Haar wavelet functions
including the proof of almost sure continuity of the sample paths.

Solution: Define the Haar wavelet functions as

$H_{n,k}(t)=2^{(n-1)/2}$ for $(k-1)/2^{n}\leq t\leq k/2^{n}$,
$H_{n,k}(t)=-2^{(n-1)/2}$ for $k/2^{n}<t\leq(k+1)/2^{n}$ whenever $k$ is in
the set $I(n)=\\{k:kodd,k=0,1,...,2^{n-1}\\}$ for n=1,2,… and put
$H_{0}(t)=1$. Here $t\in[0,1]$. It is then easily verified that the functions
$H_{nk},k\in I(n),n\geq 1,H_{00}$ form an orthonormal basis for $L^{2}[0,1]$.
We set $I(0)=\\{0\\}$.

Define the Schauder functions

$S_{nk}(t)=\int_{0}^{t}H_{nk}(s)ds$

Then, from basic properties of orthonormal bases, we have

$\sum_{nk}H_{nk}(t)H_{nk}(s)=\delta(t-s)$

and hence

$\sum_{nk}S_{nk}(t)S_{nk}(s)=min(t,s)$

Let $\xi(n,k),k\in I(n),n\geq 1,\xi(0,0)$ be iid $N(0,1)$ random variables and
define the random functions

$B_{N}(t)=\sum_{n=0}^{N}\sum_{k\in I(n)}\xi(n,k)S_{nk}(t),N\geq 1$

Then since $S_{nk}(t)$ are continuous functions, $B_{N}(t)$ is also continuous
for all $N$. Further, the $S_{nk}^{\prime}s$ for a fixed $n$ are non-
overlapping functions and

$max_{t}|S_{nk}(t)|=2^{-(n+1)/2}$

Thus,

$max_{t}\sum_{k\in I(n)}|S_{nk}(t)|=2^{-(n+1)/2}$

Let

$b(n)=max(|\xi(nk)|:k\in I(n)\\}$

Then, by the union bound

$P(b(n)>n)\leq
2^{n}.P(|\xi(nk)|>n)=2^{n+1}\int_{n}^{\infty}exp(-x^{2}/2)dx/\sqrt{2\pi}$
$\leq K.2^{n+1}n.exp(-n^{/}2)$

Thus,

$\sum_{n\geq 1}P(b(n)>n)<\infty$

which means that by the Borel-Cantelli lemma, for a.e.$\omega$, there is a
finite $N(\omega)$ such that for all $n>N(\omega)$, $b(n,\omega)<n$. This
means that for a.e.$\omega$,

$\sum_{n=1}^{N}\sum_{k\in I(n)}|\xi(n,k,\omega)||S_{nk}(t)|$
$\leq\sum_{n=1}^{N}n.2^{-(n+1)/2}$

which converges as $N\rightarrow\infty$. This result implies that the sequence
of continuous processes $B_{N}(t),t\in[0,1]$ converges uniformly to a
continuous process $B(t),t\in[0,1]$ a.e. Note that continuity of each term and
uniform convergence imply continuity of the limit. That $B(t),t\in[0,1]$ is a
Gaussian process with zero mean and covariance $min(t,s)$ as can be seen from
the joint characteristic function of the time samples of $B_{N}(.)$.
Alternately, $B(.)$ being an infinite linear superposition of Gaussian r.v’s
must be a Gaussian process, its mean zero is obvious since $\xi(n,k)$ have
mean zero and the covariance follows from

$\mathbb{E}(B_{N}(t)B_{N}(s))=\sum_{n,m=0}^{N}\sum_{k,r\in
I(n)}\mathbb{E}(\xi(nk)\xi(mr))S_{nk}(t)S_{mr}(s)$
$=\sum_{n,m=0}^{N}\sum_{k,r\in I(n)}\delta(n,m)\delta(k,r)S_{nk}(t)S_{mr}(s)$
$=\sum_{n=0}^{N}\sum_{k\in
I(n)}S_{nk}(t)S_{nk}(s)\rightarrow\sum_{n=0}^{\infty}\sum_{k\in
I(n)}S_{nk}(t)S_{nk}(s)=min(t,s)$

[b] $L^{2}$-construction of Brownian motion from the Karhunen-Loeve expansion.

The autocorrelation of $B(t)$ is

$R(t,s)=\mathbb{E}(B(t)B(s))=min(t,s)$

Its eigenfunctions $\phi(t)$ satisfy

$\int_{0}^{1}R(t,s)\phi(s)ds=\lambda\phi(t),t\in[0,1]$

Thus,

$\int_{0}^{t}sphi(s)ds+t\int_{t}^{1}\phi(s)ds=\lambda\phi(t)$

Differentiating,

$t\phi(t)+\int_{t}^{1}\phi(s)-t\phi(t)=\lambda\phi^{\prime}(t)$

Again differentiating

$-\phi(t)=\lambda\phi^{\prime\prime}(t)$

Let $\lambda=1/\omega^{2}$. Then, the solution is

$\phi(t)=A.cos(\omega t)+B.sin(\omega t)$
$\int_{0}^{t}s\phi(s)ds=A((t/\omega)sin(\omega t)+(cos(\omega
t)-1)/\omega^{2})+$ $B(-t.cos(\omega t)/\omega+sin(\omega t)/\omega^{2})$
$\int_{t}^{1}\phi(s)ds=A(sin(\omega)-sin(\omega
t))/\omega+(B/\omega)(cos(\omega t)-\cos(\omega))$

Substituting all these expressions into the original integral equation gives

$A((t/\omega)sin(\omega t)+(cos(\omega t)-1)/\omega^{2})+$ $B(-t.cos(\omega
t)/\omega+sin(\omega t)/\omega^{2})$ $+tA(sin(\omega)-sin(\omega
t))/\omega+(Bt/\omega)(cos(\omega t)-\cos(\omega))$ $=(A.cos(\omega
t)+B.sin(\omega t))/\omega^{2}$

On making cancellations and comparing coefficients,

$A.sin(\omega)=0,B.cos(\omega)=0,A/\omega^{2}=0,$

Therefore,

$A=0,\omega=(n+1/2)\pi,n\in\mathbb{Z}$

and for normalization

$\int_{0}^{1}\phi^{2}(t)dt=1$

we get

$B^{2}\int_{0}^{1}sin^{2}(\omega t)dt=1$

or

$(B^{2}/2)(1-sin(2\omega)/2\omega)=1$

or

$B=\sqrt{2}$

Thus, the KL expansion for Brownian motion is given by

$B(t)=\sum_{n\in\mathbb{Z}}\sqrt{\lambda_{n}}\xi(n)\phi_{n}(t)$
$=\sum_{n\in\mathbb{Z}}((n+1/2)\pi)^{-2}\xi(n).\sqrt{2}sin((n+1/2)\pi t)$

where $\xi(n),n=1,2,...$ are iid $N(0,1)$ r.v’s.

[c] Reflection principle for Brownian motion with application to the
evaluation of the distribution of the first hitting time of the process at a
given level.

[d] The Borel-Cantelli lemmas with application to proving almost sure
convergence of a sequence of random variables.

[e] The Feynman-Kac formula for solving Schrodinger’s equation using Brownian
motion with complex time.

[f] Solving Poisson’s equation $\nabla^{2}u(x)=s(x)$ in an open connected
subset of $\mathbb{R}^{n}$ with the Dirichlet boundary conditions using
Brownian motion stop times.

[4] A bulb in a bulb holder is changed when it gets fused. Let $X_{n}$ denote
the time for which the $n^{th}$ bulb lasts. If $n$ is even, then $X_{n}$ has
the exponential density $\lambda.exp(-\lambda.x)u(x)$ while if $n$ is odd then
$X_{n}$ has the exponential density $\mu.exp(-\mu.x)u(x)$. Calculate (a) the
probability distribution of the number of bulbs $N(t)=max(n:S_{n}\leq t)$
changed in $[0,t]$ where $S_{n}=X_{1}+...+X_{n},n\geq 1$ and $S_{0}=0$. Define

$\delta(t)=t-S_{N(t)},\gamma(t)=S_{N(t)+1}-t$

Pictorially illustrate and explain the meaning of the r.v’s
$\delta(t),\gamma(t)$. What happens to the distributions of these r.v’s as
$t\rightarrow\infty$ ?

Solution: Let $n\geq 1$.

$P(N(t)\geq n)=P(S_{n}\leq t)=P(\sum_{1\leq k\leq n,keven}X_{k}+\sum_{1\leq
k\leq n,nodd}X_{k}\leq t)$

Specifically,

$P(N(t)\geq 2n)=P(\sum_{k=1}^{n}X_{2k}+\sum_{k=1}^{n}X_{2k-1}\leq t)$
$=F^{*n}*G^{*n}(t)$

where

$F(t)=1-exp(-\lambda t),G(t)=1-exp(-\mu t)$

Note that with $f(t)=F^{\prime}(t),g(t)=G^{\prime}(t)$, we have

$f^{n*}(t)=\lambda^{n}t^{n-1}exp(-\lambda t)/(n-1)!,$
$g^{n*}(t)=\mu^{n}t^{n-1}exp(-\mu t)/(n-1)!$

Equivalently, in terms of Laplace transforms,

$L(f(t))=\lambda/(\lambda+s),L(g(t))=\mu/(\mu+s)$

so

$L(f^{*n}(t))=\lambda^{n}/(\lambda+s)^{n},L(g^{*n}(t))=\mu^{n}/(\mu+s)^{n}$

and hence

$f^{*n}(t)*g^{*n}(t)=L^{-1}((\lambda\mu)^{2}/(\lambda+s)^{n}(\mu+s)^{n})=\psi_{n}(t)$

say. Then,

$P(N(t)\geq 2n)=\int_{0}^{t}\psi_{n}(x)dx$

Likewise,

$P(N(t)\geq 2n-1)=P(S_{2n-1}\leq t)=\int_{0}^{t}\phi_{n}(x)dx$

where

$\phi_{n}(t)=g^{n*}(t)*f^{*(n-1)}(t)=L^{-1}(\lambda^{n-1}\mu^{n}/(\lambda+s)^{n-1}(\mu+s)^{n})$

$\delta(t)=t-S_{N(t)}$ is the time taken from the last change of the bulb to
the current time. $\gamma(t)=S_{N(t)+1}-t$ is the time taken from the current
time upto the next change of the bulb. Thus, for $0\leq x\leq t$,

$P(\delta(t)\leq x)=P(S_{N(t)}>t-x)=\sum_{n\geq 1}P(S_{n}>t-x,N(t)=n)$
$=\sum_{n\geq 1}P(t-x<S_{n}\leq t<S_{n+1})=\sum_{n\geq
1}P(t-x<S_{n}<t,X_{n+1}>t-S_{n})$ $=\sum_{n\geq
1}\int_{t-x}^{t}P(X_{n+1}>t-s)P(S_{n}\in ds)$

Likewise,

$P(\gamma(t)\leq x)=P(S_{N(t)+1}\leq t+x)=\sum_{n\geq 0}P(S_{n+1}\leq
t+x,S_{n}\leq t<S_{n+1})$ $=\sum_{n\geq 0}P(S_{n}\leq t<S_{n+1}\leq
t+x)=\sum_{n\geq 0}\int_{0}^{t}P(S_{n}\in ds)P(t-s<X_{n+1}<t+x)$

[5] [a] Let $\\{X(t):t\in\mathbb{R}\\}$ be a real stochastic process with
autocorrelation $\mathbb{E}(X(t_{1}).X(t_{2}))=R(t_{1},t_{2})$. Assume that
there exists a finite positive real number $T$ such that
$R(t_{1}+T,t_{2}+T)=R(t_{1},t_{2})\forall t_{1},t_{2}\in\mathbb{R}$. Prove
that $X(t)$ is a mean square periodic process, ie,

$\mathbb{E}[(X(t+T)-X(t))^{2}]=0\forall t\in\mathbb{R}$

Hence show that $X(t)$ has a mean square Fourier series:

$lim_{N\rightarrow\infty}\mathbb{E}[|X(t)-\sum_{n=-N}^{N}c(n)exp(jn\omega_{0}t)|^{2}]=0$

where $\omega_{0}=2\pi/T$ and

$c(n)=T^{-1}\int_{0}^{T}X(t)exp(-jn\omega_{0}t)dt$

[b] Define a cyclostationary and wide sense cyclostationary process with an
example. Give an example of a wide sense cyclostationary process that is not
cyclostationary.

Solution: A process $X(t),t\in\mathbb{R}$ is said to be cyclostationary if its
finite dimensional probability distribution functions are invariant under
shifts by integer multiples of some fixed positive $T$, ie, if

$P(X(t_{1})\leq x_{1},...,X(t_{N})\leq x_{N})=P(X(t_{1}+T)\leq
x_{1},...,X(t_{N}+T)\leq x_{N})$

for all $x_{1},...,x_{N},t_{1},...,t_{N},N\geq 1$. For example, the process

$X(t)=\sum_{n}c(n)exp(jn\omega t)$

where $c(n)$ are any r.v’s is cyclostationary with period $2\pi/\omega$. In
fact, this process is a periodic process. A wide sense cyclostationary process
is a process whose mean and autocorrelation function are periodic with period
$T$ for some fixed $T>0$, ie, if

$\mu(t)=\mathbb{E}(X(t)),R(t,s)=\mathbb{E}(X(t)X(s))$

then

$\mu(t+T)=\mu(t),R(t+T,s+T)=R(t,s)\forall t,s\in\mathbb{R}$

An example of a WSS process that is not stationary is

$X(t)=cos(\omega_{1}t+\phi_{1})+cos(\omega_{2}t+\phi_{2})+cos(\omega_{3}t+\phi_{1}+\phi_{2})$

where $\omega_{1}+\omega_{2}\neq\omega_{3}$ and $\phi_{1},\phi_{2}$ are iid
r.v’s uniformly distributed over $[0,2\pi)$. This process has second moments
that are invariant under time shifts but its third moments are not invariant
under time shifts. Likewise, an example of a WS-cyclostationary process that
is not cyclostationary is the process $X(t)+Y(t)$ with $Y(t)$ having the same
structure as $X(t)$ but independent of it with frequencies
$\omega_{4},\omega_{5},\omega_{6}$ (and phases
$\phi_{4},\phi_{5},\phi_{4}+\phi_{5}$ with
$\phi_{1},\phi_{2},\phi_{4},\phi_{5}$ being independent and uniform over
$[0,2\pi)$) such that
$(\omega_{1}+\omega_{2}-\omega_{3})/(\omega_{4}+\omega_{5}-\omega_{6})$ is
irrational. This process is also in fact WSS. To see why it is not
cyclostationary, we compute its third moments:

$\mathbb{E}(X(t)X(t+t_{1})X(t+t_{2}))=\mathbb{E}(cos(\omega_{3}t+\phi_{1}+\phi_{2}).cos(\omega_{1}(t+t_{1})+\phi_{1}).cos(\omega_{2}(t+t_{2})+\phi_{2}))$
$+\mathbb{E}(cos(\omega_{3}t+\phi_{1}+\phi_{2}).cos(\omega_{1}(t+t_{2})+\phi_{1}).cos(\omega_{2}(t+t_{1})+\phi_{2}))$
$=(1/4)(cos((\omega_{1}+\omega_{2}-\omega_{3})t+\omega_{1}t_{1}+\omega_{2}t_{2})$
$+cos((\omega_{1}+\omega_{2}-\omega_{3})t+\omega_{2}t_{1}+\omega_{1}t_{2}))$

and likewise

$\mathbb{E}(Y(t)Y(t+t_{1})Y(t+t_{2}))=$
$=(1/4)(cos((\omega_{4}+\omega_{5}-\omega_{6})t+\omega_{4}t_{1}+\omega_{5}t_{2})$
$+cos((\omega_{4}+\omega_{5}-\omega_{6})t+\omega_{5}t_{1}+\omega_{4}t_{2}))$

Thus, if

$Z(t)=X(t)+Y(t)$

then

$\mathbb{E}(Z(t)Z(t+t_{1})Z(t+t_{2}))=\mathbb{E}(X(t)X(t+t_{1})X(t+t_{2}))+\mathbb{E}(Y(t)Y(t+t_{1})Y(t+t_{2}))$
$=(1/4)(cos((\omega_{1}+\omega_{2}-\omega_{3})t+\omega_{1}t_{1}+\omega_{2}t_{2})$
$+cos((\omega_{1}+\omega_{2}-\omega_{3})t+\omega_{2}t_{1}+\omega_{1}t_{2}))$
$+(1/4)(cos((\omega_{4}+\omega_{5}-\omega_{6})t+\omega_{4}t_{1}+\omega_{5}t_{2})$
$+cos((\omega_{4}+\omega_{5}-\omega_{6})t+\omega_{5}t_{1}+\omega_{4}t_{2}))$

which is clearly not periodic in $t$ since
$(\omega_{1}+\omega_{2}-\omega_{3})/(\omega_{4}+\omega_{5}-\omega_{6})$ is
irrational.

[6] Explain the construction of the Lebesgue measure as a countably additive
non-negative set function on the Borel subsets of $\mathbb{R}$ whose value on
an interval coincides with the interval’s length.

Solution: We first define the Lebesgue measure $\mu$ on the field
$\mathcal{F}$ of finite disjoint unions of intervals of the form $(a,b]$ in
$(0,1]$ as the sum of lengths. Then, we define the outer measure $\mu^{*}$ on
the class of all subsets of $(0,1]$ as

$\mu^{*}(E)=inf\\{\sum_{n}\mu(E_{n}):E_{n}\in\mathcal{F},E\subset\bigcup_{n}E_{n}\\}$

We prove countable subadditivity of $\mu^{*}$. Then, we say that $E$ is a
measurable subset of $(0,1]$ if

$\mu^{*}(EA)+\mu^{*}(E^{c}A)=\mu^{*}(A)\forall A\subset(0,1]$

Then, if $\mathcal{M}$ denotes the class of all measurable subsets, we prove
that $\mathcal{M}$ is a field and that $\mu^{*}$ is finitely additive on
$\mathcal{M}$. We then prove that $\mathcal{M}$ is a $\sigma$ field and that
$\mu^{*}$ is countably additive on $\mathcal{M}$. Finally, we prove that
$\mathcal{M}$ contains the field $\mathcal{F}$ and hence the $\sigma$-field
generated by $\mathcal{F}$, namely, the Borel $\sigma$-field on $(0,1]$.

[7] [a] What is a compact subset of $\mathbb{R}^{n}$ ? Give an example and
characterize it means of (a) the Heine-Borel property and (b) the Bolzano-
Weierstrass property. Prove that the countable intersection of a sequence of
decreasing non-empty compact subsets of $\mathbb{R}^{n}$ is non-empty.

A compact subset $E$ of $\mathbb{R}^{n}$ is a closed and bounded subset. By
bounded, we mean that

$sup\\{|x|:x\in E\\}<\infty$

By closed, we mean that if $x_{n}\in E$ is a sequence in $E$ such that
$limx_{n}=x\in\mathbb{R}^{n}$ exists, then $x\in E$. For example,

$E=\\{x\in\mathbb{R}^{n}:|x|\leq 1\\}$

It can be shown that $E$ is compact iff every open cover of it has finite
subcover, ie, if $O_{\alpha},\alpha\in I$ are open sets in $\mathbb{R}^{n}$
such that

$E\subset\bigcup_{\alpha\in I}O_{\alpha}$

then there is a finite subset

$F=\\{\alpha_{1},...,\alpha_{N}\\}\subset I$

such that

$E\subset\bigcup_{k=1}^{N}O_{\alpha_{k}}$

(b) What is a regular measure on
$(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))?$. Show that every probability
measure on $(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))$ is regular.

Solution: A measure $\mu$ on $(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))$ is
said to be regular if given any $B\in\mathcal{B}(\mathbb{R}^{n})$ and
$\epsilon>0$, there exists an open set $F$ and a closed set $C$ in
$\mathbb{R}^{n}$ such that $C\subset B\subset F$ and $\mu(F-C)<\epsilon$. Let
us now show that any probability measure (and hence any finite measure) on
$(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))$ is regular. Let $P$ be a
probability measure on the above space. Define $\mathcal{C}$ to be the family
of all sets $B\in\mathcal{B}(\mathbb{R}^{n})$ such that $\forall\epsilon>0$,
there exists a closed set $C$ and an open set $F$ such that $C\subset B\subset
F$ and $P(F-C)<\epsilon$. We prove that $\mathcal{C}$ is a $\sigma$-field and
then since it contains all closed sets (If $C$ is a closed set and
$F_{n}=\\{x\in\mathbb{R}^{n}:d(x,C)<1/n\\}$, then $F_{n}$ is open, $C\subset
C\subset F_{n}$ and $P(F_{n}-C)\rightarrow 0$ because $F_{n}\downarrow C$), it
will follow that $\mathcal{C}=\mathcal{B}(\mathbb{R}^{n})$ which will complete
the proof of the result. Let $B\in\mathcal{C}$ and let $\epsilon>0$. Then,
there exists a closed $C$ and an open $F$ such that $C\subset B\subset F$ and
$P(F-C)<\epsilon$. Then $F^{c}$ is closed, $C^{c}$ is open, $F^{c}\subset
B^{c}\subset C^{c}$ and $P(C^{c}-F^{c})=P(F-C)<\epsilon$, thereby proving that
$B^{c}\in\mathcal{C}$, ie, $\mathcal{C}$ is closed under complementation. Now,
let $B_{n}\in\mathcal{C},n=1,2,...$ and let $\epsilon>0$. Then, for each $n$
we can choose a closed $C_{n}$ and an open $F_{n}$ such that $C_{n}\subset
B_{n}\subset F_{n}$ with $P(F_{n}-C_{n})<\epsilon/2^{n}$. Then,
$\bigcup_{n=1}^{N}C_{n}$ is closed for any finite integer $N$,
$\bigcup_{n=1}^{\infty}F_{n}$ is open and for $B=\bigcup_{n}B_{n}$, we have
that

$\bigcup_{n=1}^{N}C_{n}\subset B\subset\bigcup_{n}F_{n},N\geq 1$

and further

$P(\bigcup_{n}F_{n}-\bigcup_{n=1}^{N}C_{n})\leq
P((\bigcup_{n=1}^{N}(F_{n}-C_{n}))\bigcup\bigcup_{n>N}F_{n})$
$\leq\sum_{n=1}^{N}P(F_{n}-C_{n})+\sum_{n>N}P(F_{n})\leq\sum_{n=1}^{N}\epsilon/2^{n}+\sum_{n>N}P(F_{n})$

which converges as $N\rightarrow\infty$ to $\epsilon$ and hence for
sufficiently large $N$, it must be smaller than $2\epsilon$ proving thereby
that $B\in\mathcal{C}$, ie, $\mathcal{C}$ is closed under countable unions.
This completes the proof that $\mathcal{C}$ is a $\sigma$-field and hence the
proof of the result.

(c) What is a consistent family of finite dimensional probability
distributions on
$(\mathbb{R}^{\mathbb{Z}_{+}},\mathcal{B}(\mathbb{R}^{\mathbb{Z}_{+}}))?$

Solution: Such a family is a family of probability distributions
$F(x_{1},...,x_{N},n_{1},...,n_{N})$ on $\mathbb{R}^{N}$ for each set of
positive integers $n_{1},...,n_{N}$and $N=1,2,...$ satisfying

(a)

$F(x_{\sigma 1},...,x_{\sigma N},n_{\sigma 1},...,n_{\sigma
N})=F(x_{1},...,x_{N},n_{1},...,n_{N})$

for all permutations $\sigma$ of $(1,2,...,N)$ and

(b)

$lim_{x_{N}\rightarrow\infty}F(x_{1},...,x_{N},n_{1},...,n_{N})=F(x_{1},...,x_{N-1},n_{1},...,n_{N-1})$

(d) Prove Kolmogorov’s consistency theorem, that given any consistent family
of finite dimensional probability distributions on
$(\mathbb{R}^{\mathbb{Z}_{+}},\mathcal{B}(\mathbb{R}^{\mathbb{Z}_{+}}))$,
there exists a probability space and a real valued stochastic process defined
on this space whose finite dimensional distributions coincide with the given
ones.

Let $F(x_{1},...,x_{N};n_{1},...,n_{N}),n_{1},...,n_{N}\geq
0,x_{1},...,x_{N}\in\mathbb{R},N\geq 1$ be a consistent family of probability
distributions. Thus $F(x_{1},...,x_{N};1,2,...,N)=F_{N}(x_{1},...,x_{N})$ is a
probability distribution on $\mathbb{R}^{N}$ with the property that

$lim_{x_{N}\rightarrow\infty}F_{N}(x_{1},...,x_{N})=F_{N-1}(x_{1},...,x_{N-1})$

The aim is to construct a (countably additive) probability measure $P$ on
$(\mathbb{R}^{\mathbb{Z}_{+}},\mathcal{B}(\mathbb{R}^{\mathbb{Z}_{+}})$ such
that

$P(B\times\mathbb{R}^{\mathbb{Z}_{+}})=P_{N}(B),B\in\mathcal{B}(\mathbb{R}^{N})$

where

$P_{N}(B)=\int_{B}F_{N}(x_{1},...,x_{N})dx_{1}...dx_{N}$

Define

$Q(B\times\mathbb{R}^{Z_{+}})=P_{N}(B),B\in\mathcal{B}(\mathbb{R}^{N})$

Then by the consistency of $F_{N},N\geq 1$, $Q$ is a well defined finitely
additive probability measure on the field

$\mathcal{F}=\bigcup_{N\geq
1}(\mathcal{B}(\mathbb{R}^{N})\times\mathbb{R}^{\mathbb{Z}_{+}})$

Note that

$\mathcal{B}_{N}=\mathcal{B}(\mathbb{R}^{N})\times\mathbb{R}^{\mathbb{Z}_{+}}$

is a field on $\mathbb{R}^{\mathbb{Z}_{+}}$ for every $N\geq 1$ and that

$\mathcal{B}_{N}\subset\mathcal{B}_{N+1},N\geq 1$

It suffices to prove that $Q$ is countably additive on $\mathcal{F}$, or
equivalently, if $\tilde{B}_{N}\in\mathcal{\mathcal{missing}}B_{N}$ is a
sequence such that $\tilde{B}_{N}\downarrow\phi$, then
$Q(\tilde{B}_{N})\downarrow 0$. Equivalently, it suffices to show that if
$B_{N}\in\mathcal{B}(\mathbb{R}^{N})$ is a sequence such that

$\tilde{B}_{N}=B_{N}\times\mathbb{R}^{\mathbb{Z}_{+}}\downarrow\phi$

then

$P_{N}(B_{N})\downarrow 0$

Suppose that this is false. Then, without loss of generality, we may assume
that there is a $\delta>0$ such that $P_{N}(B_{N})\geq\delta\forall N\geq 1$.
By regularity of probability measures on $\mathbb{R}^{N}$ for $N$ finite, we
can find for each $N\geq 1$, a compact set
$K_{N}\in\mathcal{B}(\mathbb{R}^{N})$ such that $K_{N}\subset B_{N}$ and
$P_{N}(B_{N}-K_{N})<\delta/2^{N}$. Now consider

$\tilde{K}_{N}=(K_{1}\times\mathbb{R}^{N-1})\cap(K_{2}\cap\mathbb{R}^{N-2})\cap...\cap(K_{N-1}\times\mathbb{R})\times
K_{N}$

Then

$\tilde{K}_{N}\subset\mathbb{K}_{N}\subset\mathbb{R}^{N}$

and $\tilde{K}_{N}$ being a closed subset of the compact set $K_{N}$ is also
compact. $\tilde{K}_{N}$ is closed because each $K_{j},j=1,2,...,N$ is compact
and hence closed. Define

$\hat{K}_{N}=\tilde{K}_{N}\times\mathbb{R}^{\mathbb{Z}_{+}},N\geq 1$

Then, it is clear that $\hat{K}_{N}$ is decreasing and that

$\hat{K}_{N}\subset\tilde{B}_{N}=B_{N}\times\mathbb{R}^{\mathbb{Z}_{+}}$

Further,

$Q(\hat{K}_{N})=Q(\tilde{B}_{N})-Q(\tilde{B}_{N}-\hat{K}_{N})$

Now,

$Q(\tilde{B}_{N})=P_{N}(B_{N})\geq\delta$

and

$Q(\tilde{B}_{N}-\hat{K}_{N})=P_{N}(B_{N}-\bigcap_{k=1}^{N}(K_{k}\times\mathbb{R}^{N-k}))$
$=P_{N}(\bigcup_{k=1}^{N}(B_{N}-(K_{k}\times\mathbb{R}^{N-k}))$ $\leq
P_{N}(\bigcup_{k=1}^{N}(B_{k}-K_{k})\times\mathbb{R}^{N-k})$
$\leq\sum_{k=1}^{N}P_{k}(B_{k}-K_{k})=\sum_{k=1}^{N}\delta/2^{k}\leq\delta/2$

Thus,

$Q(\hat{K}_{N})\geq\delta/2$

and hence $\hat{K}_{N}$ is non-empty for each $N$. Hence, so is
$\tilde{K}_{N}$ for each $N$. Now
$\hat{K}_{N}\subset\tilde{B}_{N}\downarrow\phi$. Hence
$\hat{K}_{N}\downarrow\phi$ and we shall obtain a contradiction to this as
follows: Since $\tilde{K}_{N}$ is non-empty, we may choose

$(x_{N1},...,x_{NN})\in\tilde{K}_{N},N=1,2,...$

Then,

$x_{N1}\in K_{1},(x_{N1},x_{N2})\in K_{2},...,(x_{N1},...,x_{NN})\in K_{N}$

Since $K_{1}$ is compact, $x_{N1},N\geq 1$ has a convergent subsequence say
$\\{x_{N_{1},1}\\}$. Now, $(x_{N_{1},1},x_{N_{1},2})\in K_{2}$ and since
$K_{2}$ is compact, this has a convergent subsequence say
$(x_{N_{2},1},x_{N_{2},2})$. Then, $\\{x_{N_{2},1}\\}$ is a subsequence of
$\\{x_{N_{1},1}\\}$ and hence also converges. Continuing this way, we finally
obtain a convergent sequence $(y_{n,1},...,y_{n,N}),n=1,2,...$ in $K_{N}$ such
that $(y_{n,1},...y_{n,j})\in K_{j},j=1,2,...,n$ and of course
$(y_{n,1},...,y_{n,j}),n\geq 1$ converges. Let

$y_{j}=lim_{n}y_{n,j},j=1,2,...$

Then, it is clear that since $K_{j}$ is closed,

$(y_{1},...,y_{j})\in K_{j},j=1,2,...$

Hence

$(y_{1},y_{2},...y_{N})\in\tilde{K}_{N},N\geq 1$

and hence,

$(y_{1},y_{2},...)\in\hat{K}_{N},N\geq 1$

which means that

$(y_{1},y_{2},...)\in\bigcap_{N\geq 1}\hat{K}_{N}$

which contradicts the fact that the rhs is an empty set. This proves the
Kolmogorov consistency theorem.

###  7.1 Problem suggested by Prof.Dhananjay

When the magnet falls within the tube of radius $R_{0}$ with the coil having
$n_{0}$ turns per unit length and extending from $z=h$ to $z=h+a$ and with the
tube extending from $z=0$ to $z=d$, we write down the equations of motion of
the magnet. We usually may assume that at time $t$, the magnetic moment vector
of the magnet is ${\bf m}$ and that its centre does not deviate from the
central axis of the tube. This may be safely assumed provided that the tube’s
radius is small. However suppose we do not make this assumption so that we aim
a the most general possible non-relativistic situation in which the magnetic
fields are strong which may cause the position of the CM of the magnet to
deviate from the central axis of the cylinder. At time $t$, assume that the CM
of the magnet is located at ${\bf
r}_{0}(t)=(\xi_{1}(t),\xi_{2}(t),\xi_{3}(t))$. We require a differential
equation for $\xi(t)$. The magnet at time $t$ is assumed to have experienced a
rotation ${\bf R}(t)$ (a $3\times 3$ rotation matrix). Thus, it moment at time
$t$ is given by

${\bf m}(t)=m_{0}{\bf R}(t)\hat{z}$

assuming that at time $t=0$, the moment pointed along the positive $z$
direction. The magnetic field produced by the magnet at time $t$ is given by

${\bf B}_{m}(t,{\bf r})=curl(\mu/4\pi)({\bf m}(t)\times({\bf r}-{\bf
r}_{0}(t))/|{\bf r}-{\bf r}_{0}(t)|^{3})$

and the flux of this magnetic field through the coil is

$\Phi(t)=n_{0}\int_{h}^{h+a}dz\int_{x^{2}+y^{2}\leq
R_{0}^{2}}B_{mz}(t,x,y,z)dxdy$

The angular momentum vector of the magnet is proportional to its magnetic
moment. So by the torque equation, we have

$d{\bf m}(t)/dt=\gamma.{\bf m}(t)\times{\bf B}_{c}(t,{\bf r}_{0}(t))---(1)$

where ${\bf B}_{c}(t,{\bf r})$ is the magnetic field produced by the coil. The
coil current is

$I(t)=\Phi^{\prime}(t)/R_{L}$

where $R_{L}$ is the coil resistance. Then, the magnetic field produced by the
coil is

${\bf B}_{c}(t,{\bf
r})=(\mu/4\pi)I(t)\int_{0}^{2n_{0}\pi}[(-\hat{x}.sin(\phi)+\hat{y}.cos(\phi))\times({\bf
r}-R_{0}(cos(\phi)\hat{x}+sin(\phi)\hat{y})-(a/2\pi n_{0})\phi\hat{z}]$
$/|{\bf r}-R_{0}(cos(\phi)\hat{x}+sin(\phi)\hat{y})-(a/2\pi
n_{0})\phi\hat{z}|^{3}R_{0}d\phi$

This expression can be used in (1). Note that $I(t)$ depends on $\Phi(t)$
which in turn is a function of ${\bf m}(t)$. Since in fact $I(t)$ is
proportional to $\Phi^{\prime}(t)$, it follows that $I(t)$ is function of
${\bf m}(t)$ and ${\bf m}^{\prime}(t)$. Note that ${\bf B}_{c}(t,{\bf r})$ is
proportional to $I(t)$ and is therefore also a function of ${\bf m}(t)$ and
${\bf m}^{\prime}(t)$. However, we also note that ${\bf B}_{m}(t,{\bf r})$ is
also a function of ${\bf r}_{0}(t)$. Hence, $\Phi(t)$ is also a function of
${\bf r}_{0}(t)$ and hence $I(t)$ is also a function of ${\bf r}_{0}(t)$ and
${\bf r}_{0}^{\prime}(t)$. Thus, ${\bf B}_{c}(t,{\bf r})$ is a function of
${\bf r}_{0}(t),{\bf r}_{0}^{\prime}(t),{\bf m}(t),{\bf m}^{\prime}(t)$, ie,
we can express it as

${\bf B}_{c}(t,{\bf r}|{\bf r}_{0}(t),{\bf r}_{0}^{\prime}(t),{\bf m}(t),{\bf
m}^{\prime}(t))$

Thus, we write (1) as

$d{\bf m}(t)/dt=\gamma.{\bf m}(t)\times{\bf B}_{c}(t,{\bf r}_{0}(t)|{\bf
r}_{0}(t),{\bf r}_{0}^{\prime}(t),{\bf m}(t),{\bf m}^{\prime}(t))---(2)$

Next, the force exerted on the magnet by the magnetic field produced by the
coil is given by

${\bf F}(t)=\nabla_{{\bf r}_{0}(t)}({\bf m}(t).{\bf B}_{c}((t,{\bf
r}_{0}(t)|{\bf r}_{0}(t),{\bf r}_{0}^{\prime}(t),{\bf m}(t),{\bf
m}^{\prime}(t))$

which means that the rectilinear equation of motion of the magnet must be
given by

$M{\bf r}_{0}^{\prime\prime}(t)={\bf F}(t)-Mg\hat{z}$

However to be more precise, the term $V=-{\bf m}(t).{\bf B}_{c}(t,{\bf
r}_{0}(t)|{\bf r}_{0}(t),{\bf r}_{0}^{\prime}(t),{\bf m}(t),{\bf
m}^{\prime}(t))$ represents a potential energy of interaction between the
magnet and the reaction field produced by the coil. So its recilinear equation
of motion should be given by

$d/dt\partial L/\partial{\bf r}_{0}^{\prime}(t)-\partial L/\partial{\bf
r}_{0}(t)=0$

where $L$, the Lagrangian is given by

$L=M|{\bf r}_{0}^{\prime}(t)|^{2}/2-V-mgz_{0}(t)$

Thus the rectilinear equation of motion of the coil is given by

$M{\bf r}_{0}^{\prime\prime}(t)+d/dt(\nabla_{{\bf r}_{0}^{\prime}(t)}{\bf
m}(t).{\bf B}_{c}(t,{\bf r}_{0}(t)|{\bf r}_{0}(t),{\bf r}_{0}^{\prime}(t),{\bf
m}(t),{\bf m}^{\prime}(t)))$ $-\nabla_{{\bf r}_{0}(t)}({\bf m}(t).{\bf
B}_{c}(t,{\bf r}_{0}(t)|{\bf r}_{0}(t),{\bf r}_{0}^{\prime}(t),{\bf m}(t),{\bf
m}^{\prime}(t)))+mg\hat{z}={\bf 0}---(3)$

(2) and (3) constitute the complete set of six coupled ordinary differential
equations for the six components of ${\bf m}(t),{\bf r}_{0}(t)$.

###  7.2 Converse of the Shannon coding theorem

Let $A$ be an alphabet. Let $u_{1},...,u_{M}\in\mathbb{A}^{n}$ with

$P_{u_{i}}=P$

where for $u\in A^{n}$, $P_{u}(x)=N(x|u)/n,x\in A$ with $N(x|u)$ denoting the
number of times that $x$ has occurred in $u$. Let $\nu_{x}(y),x\in A,y\in B$
be the discrete memory channel transition probability from the alphabet $A$ to
the alphabet $B$. Define

$Q(y)=\sum_{x}P(x)\nu_{x}(y),y\in B$

let $D_{1},...,D_{M}$ be disjoint sets in $B^{n}$ such that
$\nu_{u_{k}}(D_{k})>1-\epsilon,k=1,2,...,n$, where
$\nu_{u}(v)=\Pi_{k=1}^{n}\nu_{u(k)}(v(k))$ for
$u=(u(1),...,u(n)),v=(v(1),...,v(n))$. Let

$E_{k}=D_{k}\cap T(n,\nu_{u_{k}},\delta),k=1,2,...,M$

where $T(n,f,\delta)$ is the set of $\delta$-Bernoulli typical sequences of
length $n$ having probability distribution $f$. For $u\in A^{n}$ with
$P_{u}=P$, define $T(n,\nu_{u},\delta)$ to consist of all $v\in B^{n}$ for
which

$v_{x}\in T(N(x|u),\nu_{x},\delta)\forall x\in A$

Then we claim that if $v\in T(n,\nu_{u},\delta)$, then

$v\in T(n,Q,\delta\sqrt{a})$

The notation used is as follows. For $x\in A$, and $u\in A^{n},v\in B^{n}$,
define

$v_{x}=(v(k):u(k)=x,k=1,2,...,n)$

Note that $v_{x}\in B^{N(x|u)}$. To prove this claim, we note that for $y\in
B$,

$|N(y|v)-nQ(y)|=|\sum_{x}N(y|v_{x})-n\sum_{x}P(x)\nu_{x}(y)|$
$=|\sum_{x}N(y|v_{x})-\sum_{x}N(x|u)\nu_{x}(y)|\leq\sum_{x}|N(y|v_{x})-N(x|u)\nu_{x}(y)|$
$\leq\sum_{x}\delta\sqrt{N(x|u)\nu_{x}(y)(1-\nu_{x}(y))}$
$\leq\delta\sqrt{a}.\sqrt{\sum_{x}N(x|u)\nu_{x}(y)(1-\nu_{x}(y))}$
$=\delta\sqrt{an}.\sqrt{\sum_{x}P(x)\nu_{x}(y)(1-\nu_{x}(y))}$

Now,

$\sum_{x}P(x)\nu_{x}(y)=Q(y),$
$\sum_{x}P(x)\nu_{x}(y)^{2}\geq(\sum_{x}P(x)\nu_{x}(y))^{2}=Q(y)^{2}$

and therefore,

$|N(y|v)-nQ(y)|\leq\delta\sqrt{an}\sqrt{Q(y)(1-Q(y)}$

proving the claim. For $v\in E_{k}$, we have $v\in T(n,\nu_{u_{k}},\delta)$
and hence by the result just proved, $v\in T(n,Q,\delta\sqrt{a})$.

Now let $v\in T(n,Q,\delta)$.

It follows that since

$Q^{n}(v)=\Pi_{y\in B}Q(y)^{N(y|v)}$

and since

$|N(y|v)-nQ(y)|\leq\delta\sqrt{nQ(y)(1-Q(y))},y\in B$

we have

$nQ(y)-\delta\sqrt{nQ(y)(1-Q(y))}\leq N(y|v)\leq
nQ(y)+\delta\sqrt{nQ(y)(1-Q(y))},y\in B$

and therefore,

$\Pi_{y}Q(y)^{nQ(y)+\delta\sqrt{nQ(y)(1-Q(y))}}\leq
Q_{n}(v)\leq\Pi_{y}Q(y)^{nQ(y)-\delta\sqrt{nQ(y)(1-Q(y))}}$

Thus summing over all $v\in T(n,Q,\delta)$, we get

$2^{-nH(Q)-c\sqrt{n}}\mu(T(n,Q,\delta))\leq Q_{n}(T(n,Q,\delta)\leq
2^{-nH(Q)+c\sqrt{n}}\mu(T(n,Q,\delta))$

where

$c=-\delta\sum_{y}\sqrt{Q(y)(1-Q(y))}log_{2}(Q(y))$

and where $\mu(E)$ denotes the cardinality of a set $E$. In particular,

$\mu(T(n,Q,\delta))\leq 2^{nH(Q)+c.sqrtn}$

It follows that

$\sum_{k=1}^{M}\mu(E_{k})=\mu(\bigcup_{k}E_{k})\leq\mu(T(n,Q,\delta\sqrt{a}))\leq
2^{nH(Q)+c\sqrt{an}}$

On the other hand,

$\nu_{u_{k}}(D_{k})>1-\epsilon$

and by the Chebyshev inequality combined with the union bound,

$\nu_{x,N(x|u)}(T(N(x|u),\nu_{x},\delta))\geq 1-b/\delta^{2}$

and hence

$\nu_{u}(T(n,\nu_{u},\delta))\geq 1-ab/\delta^{2}$

for any $u\in A^{n}$. In particular,

$\nu_{u_{k}}(E_{k})\geq 1-ab/\delta^{2}-\epsilon,k=1,2,...,M$

Further, by the above argument, we have for $v\in T(n,\nu_{u},\delta)$,

$\nu_{x,N(x|u)}(v_{x})\leq 2^{N(x|u)H(\nu_{x})+c\sqrt{n}}$

and hence,

$\nu_{u}(v)\leq 2^{-\sum_{x}N(x|u)H(\nu_{x})-c\sqrt{n}}$
$=2^{-nI(P,\nu)+c\sqrt{n}}$

where

$I(P,\nu)=\sum_{x}P(x)H(\nu_{x}),P=P_{u}$

From this we easily deduce that

$\nu_{u_{k}}(E_{k})\leq 2^{-nI(P,\nu)+c\sqrt{n}}\mu(E_{k}),k=1,2,...,M$

Thus,

$(1-ab/\delta^{2}-\epsilon)2^{nI(P,\nu)-c\sqrt{n}}\leq\mu(E_{k}),k=1,2,...,M$

so that

$M(1-ab/\delta^{2}-\epsilon)2^{nI(P,\nu)-c\sqrt{n}}\leq\sum_{k}\mu(E_{k})\leq
2^{nH(Q)+c\sqrt{n}}$

or equivalently,

$log(M)/n\leq H(Q)-I(P,\nu)+O(1/\sqrt{n})$

which proves the converse of Shannon’s coding theorem.

##  Chapter 8 Problems in electromagnetics and gravitation

The magnetic field produced by a magnet of given magnetic dipole moment in a
background gravitational field taking space-time curvature into account. The
Maxwell equations in curved space-time are

$(F^{\mu\nu}\sqrt{-g})_{,\nu}=J^{\mu}\sqrt{-g}$

or equivalently

$(g^{\mu\alpha}g^{\nu\beta}\sqrt{-g}F_{\alpha\beta})_{,\nu}=J^{\mu}\sqrt{-g}$

Writing

$g_{\mu\nu}(x)=\eta_{\mu\nu}+h_{\mu\nu}(x)$

where $\eta$ is the Minkowski metric and $h$ is a small perturbation, we have
approximately

$g^{\mu\nu}=\eta_{\mu\nu}-h^{\mu\nu},h^{\mu\nu}=\eta_{\mu\alpha}\eta_{\nu\beta}h_{\alpha\beta}$

and

$\sqrt{-g}=1-h/2,h=\eta_{\mu\nu}h_{\mu\nu}$

We have

$F_{\mu\nu}=A_{\nu,\mu}-A_{\mu,\nu}$

and writing

$A_{\mu}=A_{\mu}^{(0)}+A_{\mu}^{(1)}$

where $A{\mu}^{(0)}$ is the zeroth order solution, ie, without gravity and
$A_{\mu}^{(1)}$ the first order correction to the solution, ie, it is a linear
functional of $h_{\mu\nu}$, we obtain using first order perturbation theory

$F_{\mu\nu}=F_{\mu\nu}^{(0)}+F_{\mu\nu}^{(1)}$

where

$F_{\mu\nu}^{(k)}=A_{\nu,\mu}^{(k)}-A_{\mu,\nu}^{(k)},k=0,1$
$g^{\mu\alpha}g^{\nu\beta}\sqrt{-g}=$
$(\eta_{\mu\alpha}-h^{\mu\alpha})(\eta_{\nu\beta}-h^{\nu\beta})(1-h/2)$
$=\eta_{\mu\alpha}\eta_{\nu\beta}-f_{\mu\nu\alpha\beta}$

where

$f_{\mu\nu\alpha\beta}=\eta_{\mu\alpha}\eta_{\nu\beta}h/2+\eta_{\mu\alpha}h^{\nu\beta}$
$+\eta_{\nu\beta}h^{\mu\alpha}$

Then

$g^{\mu\alpha}g^{\nu\beta}\sqrt{-g}F_{\alpha\beta}=$
$\eta_{\mu\alpha}\eta_{\nu\beta}F_{\alpha\beta}^{(0)}$
$+\eta_{\mu\alpha}\eta_{\nu\beta}F_{\alpha\beta}^{(1)}-f_{\mu\nu\alpha\beta}F_{\alpha\beta}^{(0)}$
$=F^{0\mu\nu}+F^{1\mu\nu}$

where

$F^{0\mu\nu}=\eta_{\mu\alpha}\eta_{\nu\beta}F_{\alpha\beta}^{(0)},$
$F^{1\mu\nu}=\eta_{\mu\alpha}\eta_{\nu\beta}F_{\alpha\beta}^{(1)}-f_{\mu\nu\alpha\beta}F_{\alpha\beta}^{(0)}$

Also

$J^{\mu}\sqrt{-g}=J^{0\mu}+J^{1\mu},J^{0\mu}=J^{\mu},J^{1\mu}=-J^{\mu}h/2$

The gauge condition

$(A^{\mu}\sqrt{-g})_{,\mu}=0$

becomes

$0=(g^{\mu\nu}\sqrt{-g}A_{\nu})_{,\mu}=$
$[(\eta_{\mu\nu}-h^{\mu\nu})(1-h/2)A_{\nu}]_{,\mu}$
$=[(\eta_{\mu\nu}-k_{\mu\nu})(A_{\nu}^{(0)}+A_{\nu}^{(1)})]_{,\mu}$

where

$k_{\mu\nu}=h^{\mu\nu}-h\eta_{\mu\nu}/2$

and hence equating coefficients of zeroth and first orders of smallness on
both sides gives

$\eta_{\mu\nu}A_{\nu,\mu}^{(0)}=0,$
$(k_{\mu\nu}A_{\nu}^{(0)})_{,\mu}=\eta_{\mu\nu}A_{\nu,\mu}^{(1)}$

##  Chapter 9 Some more aspects of quantum information theory

###  9.1 An expression for state detection error

Let $\rho,\sigma$ be two states and let their spectral decompositions be

$\rho=\sum_{a}|e_{a}>p(a)<e_{a}|,\sigma=\sum_{a}|f_{a}>q(a)<f_{a}|$

so that $<e_{a}|e_{b}>=\delta(a,b),<f_{a}|f_{b}>=\delta(a,b)$. The relative
entropy between these two states is

$D(\rho|\sigma)=Tr(\rho.(log(\rho)-log(\sigma)))$
$=\sum_{a}p(a)log(p(a))-\sum_{a,b}p(a)log(q_{b})|<e_{a}|f_{b}>|^{2}$
$=\sum_{a,b}(p(a)|<e_{a}|f_{b}>|^{2}(log(p(a)|<e_{a}|f_{b}>|^{2})-log(q_{b}|<e_{a}|f_{b}>|^{2}))$
$=\sum_{a,b}(P(a,b)log(P(a,b))-log(Q(a,b)))=D(P|Q)$

where $P,Q$ are two bivariate probability distributions defined by

$P(a,b)=p(a)|<e_{a}|f_{b}>|^{2},Q(a,b)=q(b)|<e_{a}|f_{b}>|^{2}$

Now let $0\leq T\leq 1$, ie, $T$ is a detection operator. Consider the
detection error probabiity

$2P(\epsilon)=Tr(\rho(1-T))+Tr(\sigma T)$
$=\sum_{a}(p(a)<e_{a}|1-T|e_{a}>-q(a)<f_{a}|T|f_{a}>)$
$=\sum_{a}p(a)(1-<e_{a}|T|e_{a}>)-\sum_{a,b}q(a)|<f_{a}|e_{b}>|^{2}<e_{b}|T|f_{a}>$

Suppose $T$ is a projection, ie, $T^{2}=T^{*}=T$. Then,

$Tr(\rho(1-T))=\sum_{a}p(a)<e_{a}|1-T)|e_{a}>=\sum_{a}p(a)<e_{a}|(1-T)^{2}|e_{a}>$
$=\sum_{a,b}p(a)|<e_{a}|1-T|f_{b}>|^{2}$

Likewise,

$Tr(\sigma T)=\sum_{a,b}q(b)<e_{a}|T|f_{b}>|^{2}$

Hence,

$2P(\epsilon)=\sum_{a,b}(p(a)|<e_{a}|1-T|f_{b}>|^{2}+q(b)<e_{a}|T|f_{b}>|^{2})$
$\geq\sum_{a,b}min(p(a),q(b))(|<e_{a}|1-T|f_{b}>|^{2}+q(b)<e_{a}|T|f_{b}>|^{2})$
$\geq\sum_{a,b}(1/2)min(p(a),q(b))|<e_{a}|f_{b}>|^{2}$

where we have used

$2(|z|^{2}+|w|^{2})\geq|z+w|^{2}$

for any two complex numbers $z,w$. Note that

$\sum_{a,b}(1/2)min(p(a),q(b))|<e_{a}|f_{b}>|^{2}=$ $(1/2)min_{0\leq
t(a,b)\leq
1,a,b=1,2,...,n}\sum_{a,b}[p(a)|<e_{a}|f_{b}>|^{2}(1-t(a,b))+q(b)|<e_{a}|f_{b}>|^{2}t(a,b))]$

because if $u,v$ are positive real numbers, then as $t$ varies over $[0,1]$,
$tu+(1-t)v$ attains is minimum value of $v$ when $t=0$ if $u\geq v$ and $u$
when $t=1$ if $u<v$. In other words,

$min_{t\in[0,1]}(tu+(1-t)v)=min(u,v)$

In other words we have proved that

$2P(\epsilon)\geq(1/2)min_{0\leq t(a,b)\leq
1,a,b=1,2,...,n}\sum_{a,b}[P(a,b)(1-t(a,b))+Q(a,b)t(a,b)]$

The quantity on the rhs is the minimum error probability for the classical
hypothesis testing problem between the two bivariate probability distributions
$P,Q$.

###  9.2 Operator convex functions and operator monotone functions

[1] Let $f$ be operator convex. Then, if $K$ is a contraction matrix, we have

$f(K^{*}XK)\leq K^{*}f(X)K$

for all Hermitian matrices $X$. By contraction we mean that $K^{*}K\leq I$. To
see this define the following positive (ie non-negative) matrices

$L=\sqrt{1-K^{*}K},M=\sqrt{1-KK^{*}}$

Then by Taylor expansion and the identity

$K(K^{*}K)^{m}=(KK^{*})^{m}K,m=1,2,...,$

we have

$KL=MK,LK^{*}=K^{*}M$

Define the matrix

$U=\left(\begin{array}[]{cc}K&-M\\\ L&K^{*}\end{array}\right)$

and observe that since $L^{*}=L,M^{*}=M$ and

$K^{*}K+L^{2}=I,-K^{*}M+LK^{*}=0,-MK+KL=0,M^{2}+KK^{*}=I$

it follows that $U$ is a unitary matrix. Likewise define the unitary matrix

$V=\left(\begin{array}[]{cc}K&M\\\ L&-K^{*}\end{array}\right)$

Also define for two Hermitian matrices $A,B$

$T=\left(\begin{array}[]{cc}A&0\\\ 0&B\end{array}\right),$

Then, we obtain the identity

$(1/2)(U^{*}TU+V^{*}TV)=\left(\begin{array}[]{cc}K^{*}AK+LBL&0\\\
0&KBK^{*}+MAM\end{array}\right)$

Using operator convexity of $f$, we have

$f((1/2)(U^{*}TU+V^{*}TV))\leq(1/2)(f(U^{*}TU)+f(V^{*}TV))=(1/2)(U^{*}f(T)U+V^{*}f(T)V)$

which in view of the above identity, results in

$f(K^{*}AK+LBL)\leq K^{*}f(A)K+Lf(B)L$

and

$f(KBK^{*}+MAM)\leq Kf(B)K^{*}+Mf(A)M$

In particular, taking $B=0$, the first gives

$f(K^{*}AK)\leq K^{*}f(A)K+f(0)L^{2}$

and choosing $A=0$, the second gives

$f(KBK^{*})\leq Kf(B)K^{*}+f(0)M^{2}$

In particular, if in addition to being operator convex, $f$ satisfies
$f(0)\leq 0$, then we get

$f(K^{*}AK)\leq K^{*}f(A)K,f(KBK^{*})\leq Kf(B)K^{*}$

for any contraction $K$. Now we show that if $f$ is operator convex and
$f(0)\leq 0$, then for operators $K_{1},...,K_{p}$ that satisfy

$\sum_{j=1}^{p}K_{j}^{*}K_{j}\leq I$

we have

$f(\sum_{j=1}^{p}K_{j}^{*}A_{j}K_{j})\leq\sum_{j=1}^{p}K_{j}^{*}f(A_{j})K_{j}$

In fact, defining

$K=\left(\begin{array}[]{ccccc}K_{1}&0&...&0\\\ K_{2}&0&...&0\\\
..&..&...&..\\\ K_{p}&0&...&0\end{array}\right)$

we observe that

$K^{*}K=\left(\begin{array}[]{cc}\sum_{j=1}^{p}K_{j}^{*}K_{j}&0\\\
0&0\end{array}\right)\leq I$

ie $K$ is a contraction and hence by the above result

$f(K^{*}TK)\leq K^{*}f(T)K$

Taking

$T=diag[A_{1},A_{2},...,A_{p}]$

we get

$K^{*}TK=\left(\begin{array}[]{cc}\sum_{j=1}^{p}K_{j}^{*}A_{j}K_{j}&0\\\
0&0\end{array}\right)$

and hence,

$\left(\begin{array}[]{cc}f(\sum_{j=1}^{p}K_{j}^{*}A_{j}K_{j})&0\\\
0&f(0)\end{array}\right)$
$\leq\left(\begin{array}[]{cc}\sum_{j=1}^{p}K_{j}^{*}f(A_{j})K_{j}&0\\\
0&0\end{array}\right)$

which results in the desired inequality. Note that from the above discussion,
we also have

$f(\sum_{j}K_{j}B_{j}K_{j}^{*})\leq\sum_{j}K_{j}f(B_{j})K_{j}^{*}$

###  9.3 A selection of matrix inequalities

[1] If $A,B$ are square matrices the eigenvalues of $AB$ are the same as those
of $BA$. If $A$ is non-singular, then this is obvious since

$A^{-1}(AB)A=BA$

Likewise if $B$ is non-singular then also it is obvious since

$B^{-1}(BA)B=AB$

If $A$ is singular, we can choose a sequence $c_{n}\rightarrow 0$ so that
$A+c_{n}I$ is non-singular for every $n$ and then

$(A+c_{n}I)^{-1}(A+c_{n}I)B(A+c_{n}I)=B(A+c_{n}I)\forall n$

Thus, $(A+c_{n}I)B$ has the same eigenvalues as those of $B(A+c_{n}I)$ for
each $n$. Taking the limit $n\rightarrow\infty$, the result follows since the
eigenvalues of $(A+cI)B$ are continuous functions of $c$.

[2] Let $A$ be any square diagonable matrix. Let $\lambda_{1},...,\lambda_{n}$
be its eigenvalues. We have for any unit vector $u$,

$|<u,Au>|^{2}\leq<u,A^{*}Au>$

Let $e_{1},...,e_{n}$ be a basis of unit vectors of $A$ corresponding to the
eigenvalues $\lambda_{1},...,\lambda_{n}$ respectively. Assume that

$|\lambda_{1}>\geq|\lambda_{2}|\geq...\geq|\lambda_{n}|$

Let $M$ be any $k$ dimensional subspace and consider the $n-k+1$ dimensional
subspace $W=span\\{e_{k},e_{k+1}...,e_{n}\\}$. Then $dim(M\cap W)\geq 1$ and
hence we can choose a unit vector $u\in M\cap W$. Then writing
$u=c_{k}e_{k}+...+c_{n}e_{n}$, it follows that

$|<u,Au>|=|c_{k}\lambda_{k}e_{k}+...+c_{n}\lambda_{n}e_{n}|\leq|\lambda_{k}|.(|c_{k}|+...+|c_{n}|)$

Suppose we Gram-Schmidt orthonormalize $e_{k},e_{k+1},...,e_{n}$ in that
order. The resulting vectors are

$w_{k}=e_{k},w_{k+1}\in span(e_{k},e_{k+1}),...,w_{n}\in
span(e_{k},...,e_{n})$

Specifically,

$w_{k+1}=r(k+1,k)e_{k}+r(k+1,k+1)e_{k+1},...,w_{n}=r(n,k)e_{k}+...+r(n,n)e_{n}$

Then we can write

$u=d_{k}w_{k}+...+d_{m}w_{n},|d_{k}|^{2}+...+|d_{n}|^{2}=1,<w_{r},w_{s}>=\delta(r,s),r,s=k,k+1,...,n$

and we find that

$Aw_{k}=\lambda_{k}w_{k},Aw_{k+1}=r(k+1,k)\lambda_{k}e_{k}+r(k+1,k+1)\lambda_{k+1}e_{k+1},$
$Aw_{n}=r(n,k)\lambda_{k}e_{k}+...+r(n,n)\lambda_{n}e_{n}$

Diagonalize $A$ so that

$A=\sum_{k=1}^{n}\lambda_{k}e_{k}f_{k}^{*},f_{k}^{*}e_{j}=\delta(k,j)$

Then,

$<f_{k},Ae_{k}>=\lambda_{k}$

Thus,

$|\lambda_{k}|\leq\parallel f_{k}\parallel.\parallel Ae_{k}\parallel$

Let $A^{\otimes_{a}k}$ denote the $k$-fold tensor product of $A$ with itself
acting on the k fold antisymmetric tensor product of $\mathbb{C}^{n}$. Then,
the eigenvalues of $A^{\otimes_{a}n}$ are $\lambda_{i_{1}}...\lambda_{i_{k}}$
with $n\geq i_{1}>i_{2}>...>i_{k}\geq 1$ and likewise its singular values are
$s_{i_{1}}...s_{i_{k}},n\geq i_{1}>i_{2}>...>i_{k}\geq 1$ where $s_{1}\geq
s_{2}\geq...\geq s_{n}\geq 0$ are the singular values of $A$. Now if $T$ is
any operator whose eigenvalue having maximum magnitude is $\mu_{1}$ and whose
maximum singular value is $s_{1}$, then we have

$|\mu_{1}|=spr(T)=lim_{n}\parallel T^{n}\parallel^{1/n}\leq\parallel
T\parallel=s_{1}$

Applying this result to $T=A^{\otimes_{a}k}$ gives us

$\Pi_{i=1}^{k}|\lambda_{i}|\leq\Pi_{i=1}^{k}s_{i},k=1,2,...,n$

[3] If $A,B$ are such that $AB$ is normal, then

$\parallel AB\parallel\leq\parallel BA\parallel$

In fact since $AB$ is normal, its spectral radius coincides with its spectral
norm and further, since $AB$ and $BA$ have the same eigenvalues, the spectral
radii of $AB$ and $BA$ are the same and finally since the spectral radius of
any matrix is always smaller than its spectral norm ($\parallel
T^{n}\parallel^{1/n}\leq\parallel T\parallel$), the result follows
immediately:

$\parallel AB\parallel=spr(AB)=spr(BA)\leq\parallel BA\parallel$

[1] Show that if $S,T\geq 0$ and $S\leq 1$, then

$1-(S+T)^{-1/2}S(S+T)^{-1/2}\leq 2-2S+4T$

[2] Show that if $S_{k},T_{k},R_{k},k=1,2$ are positive operators in a Hilbert
space such that (a) $[S_{1},S_{2}]=[T_{1},T_{2}]=[[R_{1},R_{2}]=0$ and
$S_{k}+T_{k}\leq R_{k},k=1,2,$, then for any $0\leq s\leq 1$, we have Lieb’s
inequality:

$R_{1}^{1-s}R_{2}^{s}\geq S_{1}^{1-s}S_{2}^{s}+T_{1}^{1-s}T_{2}^{s}$

First we show that this inequality holds for $s=1/2$, then we show that if it
holds for $s=\mu,\nu$, then it also holds for $s=(\mu+\nu)/2$ and that will
complete the proof.

$|<y|S_{1}^{1/2}S_{2}^{1/2}+T_{1}^{1/2}T_{2}^{1/2}|x>|^{2}$
$\leq(|<S_{1}^{1/2}y,S_{2}^{1/2}x>|+|<T_{1}^{1/2}y,T_{2}^{1/2}x>|)^{2}$
$\leq[\parallel S_{1}^{1/2}y\parallel.\parallel
S_{2}^{1/2}x\parallel+\parallel T_{1}^{1/2}y\parallel.\parallel
T_{2}^{1/2}x\parallel]^{2}$ $\leq[\parallel
S_{1}^{1/2}y\parallel^{2}+\parallel T_{1}^{1/2}y\parallel^{2}]$
$\times[\parallel S_{2}^{1/2}x\parallel^{2}+\parallel
T_{2}^{1/2}x\parallel^{2}]$
$=<y,(S_{1}+T_{1})y><x,(S_{2}+T_{2})x>\leq<y,R_{1}y><x,R_{2}x>$

Then, replacing $y$ by $R_{1}^{-1/4}R_{2}^{1/4}x$ and $x$ by
$R_{2}^{-1/4}R_{1}^{1/4}x$ in this inequality gives us (recall that $R_{1}$
and $R_{2}$ are assumed to commute)

$<x|R_{1}^{-1/4}R_{2}^{1/4}(S_{1}^{1/2}S_{2}^{1/2}+T_{1}^{1/2}T_{2}^{1/2})R_{2}^{-1/4}R_{1}^{1/4}|x>$
$\leq<x,R_{1}^{1/2}R_{2}^{1/2}x>$

Now define the matrices

$B=R_{1}^{-1/4}R_{2}^{1/4}(S_{1}^{1/2}S_{2}^{1/2}+T_{1}^{1/2}T_{2}^{1/2},A=R_{2}^{-1/4}R_{1}^{1/4}$

Then it is clear that

$AB=S_{1}^{1/2}S_{2}^{1/2}+T_{1}^{1/2}T_{2}^{1/2}$

is a Hermitian positive matrix because $[S_{1},S_{2}]=[T_{1},T_{2}]=0$ and we
also have the result that the eigenvalues of $AB$ are the same as those of
$BA$. Further, from the above inequality, the $k^{th}$ largest eigenvalue of
$BA$ is $\leq$ the $k^{th}$ largest eigenvalue of $R_{1}^{1/2}R_{2}^{1/2}$. It
follows therefore that the $k^{th}$ largest eigenvalue of the Hermitian matrix
$AB$ is $\leq$ the $k^{th}$ largest eigenvalue of the Hermitian matrix
$R_{1}^{1/2}R_{2}^{1/2}$. However, from this result, we cannot deduce the
desired inequality. To do so, we note that writing

$S_{1}^{1/2}S_{2}^{1/2}+T_{1}^{1/2}T_{2}^{1/2}$

we have proved that

$|<y,Cx>|^{2}\leq<y,R_{1}y><x,R_{2}x>$

This implies

$|<R_{1}^{-1/2}y,CR_{2}^{-1/2}x>|\leq\parallel y\parallel.\parallel
x\parallel$

for all $x,y$ and hence

$|<y,R_{1}^{-1/2}CR_{2}^{-1/2}x>|/\parallel y\parallel.\parallel
x\parallel\leq 1$

for all nonzero $x,y$. Taking supremum over $x,y$ yields

$\parallel R_{1}^{-1/2}CR_{2}^{-1/2}\parallel\leq 1$

and by normality of

$R_{1}^{-1/4}R_{2}^{-1/4}CR_{1}^{-1/4}R_{2}^{-1/4}=$
$(R_{1}^{1/4}R_{2}^{-1/4})(R_{1}^{-1/2}CR_{1}^{-1/4}R_{2}^{-1/4})=PQ$

where

$P=(R_{1}^{1/4}R_{2}^{-1/4}),Q=(R_{1}^{-1/2}CR_{1}^{-1/4}R_{2}^{-1/4})$

we get

$\parallel PQ\parallel\leq\parallel QP\parallel=$ $\parallel
R_{1}^{-1/2}CR_{2}^{-1/2}\parallel\leq 1$

which implies since $PQ$ is positive that

$PQ\leq I$

or equivalently

$(R_{1}^{-1/4}R_{2}^{-1/4}CR_{1}^{-1/4}R_{2}^{-1/4})\leq I$

from which we deduce that (Note that $R_{1}$ and $R_{2}$ commute)

$C\leq R_{1}^{1/2}R_{2}^{1/2}$

proving the claim.

Now suppose it is valid for some $s=\mu,\nu$. Then we have

$R_{1}^{1-\mu}R_{2}^{\mu}\geq
S_{1}^{1-\mu}S_{2}^{\mu}+T_{1}^{1-\mu}T_{2}^{\mu},$

and

$R_{1}^{1-\nu}R_{2}^{\nu}\geq
S_{1}^{1-\nu}S_{2}^{\nu}+T_{1}^{1-\nu}T_{2}^{\nu},$

Then, replacing $R_{1}$ by $R_{1}^{1-\mu}R_{2}^{\mu}$, $R_{2}$ by
$R_{1}^{1-\nu}R_{2}^{\nu}$, $S_{1}$ by $S_{1}^{1-\mu}S_{2}^{\mu}$, $S_{2}$ by
$S_{1}^{1-\nu}S_{2}^{\nu}$, $T_{1}$ by $T_{1}^{1-\mu}T_{2}^{\mu}$ and $T_{2}$
by $T_{1}^{1-\nu}T_{2}^{\nu}$ and noting that these replacements satisfy all
the required conditions of the theorem, gives us

$R_{1}^{1-(\mu+\nu)/2}R_{2}^{(\mu+\nu)/2}\geq
S_{1}^{1-(\mu+\nu)/2}S_{2}^{(\mu+\nu)/2}+T_{1}^{1-(\mu+\nu)/2}T_{2}^{(\mu+\nu)/2},$

ie the result also holds for $s=(\mu+\nu)/2$. Since it holds for $s=0,1/2,1$
we deduce by induction that it holds for all dyadic rationals in $[0,1]$, ie,
for all $s$ of the form $k/2^{n},k=0,1,...,2^{n},n=0,1,...$. Now every real
number in $[0,1]$ is a limit of dyadic rationals and hence by continuity of
the inequality, the theorem has been proved.

An application example:

Remark: Let $T$ be a Hermitian matrix of size $n\times n$. Let $M$ be an
$n-k+1$-dimensional subspace and let $\lambda_{1}\geq...\geq\lambda_{n}$ be
the eigenvalues of $T$ arranged in decreasing order. Then,

$sup_{x\in M}<x,Tx>\geq\lambda_{k}$

and thus

$inf_{dimN=n-k+1}sup_{x\in N}<x,Tx>=\lambda_{k}$

Likewise, if $M$ is a $k$ dimensional subspace, then

$inf_{x\in M}<x,Tx>\leq\lambda_{k}$

and hence

$sup_{dimN=k}inf_{x\in N}<x,Tx>=\lambda_{k}$

Thus,

$\lambda_{k}(AB)=\lambda_{k}(BA)\leq\lambda_{k}(C),k=1,2,...,n$

where

$C=R_{1}^{1/2}R_{2}^{1/2}$

Let $A,B$ be two Hermitian matrices. Choose a subspace $M$ of dimension
$n-k+1$ so that

$\lambda_{k}(A)^{\downarrow}=sup_{x\in M}<x,Ax>$

Then,

$\lambda_{k}(A-B)^{\downarrow}=inf_{dimN=n-k+1}sup_{x\in N}<x,(A-B)x>$ $\leq
sup_{x\in M}<x,(A-B)x>=sup_{x\in M}(<x,Ax>-<x,Bx>)$ $\leq sup_{x\in
M}<x,Ax>-inf_{x\in M}<x,Bx>$ $=\lambda_{k}(A)^{\downarrow}-inf_{x\in M}<x,Bx>$
$\lambda_{k}(A-B)^{\downarrow}\leq\lambda_{k}(A)^{\downarrow}-inf_{x\in
M}<x,Bx>$ $\leq\lambda_{k}(A)^{\downarrow}-\lambda_{n}(B)^{\downarrow}$

Now let $M$ be any $n-k+1$ dimensional subspace such that

$inf_{x\in M}<x,Bx>=\lambda_{n-k+1}(B)^{\downarrow}=\lambda_{k}(B)^{\uparrow}$

Then,

$\lambda_{k}(A-B)^{\downarrow}\leq sup_{x\in M}<x,(A-B)x>\leq sup_{x\in
M}<x,Ax>-inf_{x\in M}<x,Bx>$
$\leq\lambda_{k}(A)^{\downarrow}-\lambda_{k}(B)^{\uparrow}$

This is a stronger result than the previous one.

###  9.4 Matrix Holder Inequality

Now let $A,B$ be two matrices. Consider $(AB)^{\otimes_{a}k}$. its maximum
singular value square is

$\parallel(AB)^{\otimes_{a}k}\parallel^{2}$

which is the maximum eigenvalue of $((AB)^{*}(AB))^{\otimes_{a}k}$. This
maximum eigenvalue equals the product of the first $k$ largest eigenvalues of
$(AB)^{*}(AB)$, ie, the product of the squares of the first $k$ largest
singular values of $AB$. Thus, we can write

$\parallel(AB)^{\otimes_{a}k}\parallel=\Pi_{j=1}^{k}s_{j}(AB)^{\downarrow}$

On the other hand,

$\parallel(AB)^{\otimes_{a}k}\parallel=\parallel A^{\otimes_{a}k}.\parallel
B^{\otimes_{a}k}\parallel$ $\leq\parallel A^{\otimes_{a}k}\parallel.\parallel
B^{\otimes_{a}k}\parallel$

Now, in the same way, $\leq\parallel A^{\otimes_{a}k}\parallel$ equals
$\Pi_{j=1}^{k}s_{j}(A)^{\downarrow}$ and likewise for $B$. Thus we obtain the
fundamental inequality

$\Pi_{j=1}^{k}s_{j}(AB)^{\downarrow}\leq\Pi_{j=1}^{k}s_{j}(A)^{\downarrow}s_{j}(B)^{\downarrow},k=1,2,...,n$

and hence we easily deduce that

$s(AB)\leq s(A).s(B)$

which is actually a shorthand notation for

$\sum_{j=1}^{k}s_{j}(AB)^{\downarrow}\leq\sum_{j=1}^{k}s_{j}(A)^{\downarrow}.s_{j}(B)^{\downarrow},k=1,2,...,n$

In particular, taking $k=n$ gives us for $p,q\geq 1$ and $1/p+1/q=1$,

$Tr(|AB|))=\sum_{j=1}^{n}s_{j}(AB)\leq(\sum_{j}s_{j}(A)^{p})^{1/p}.(\sum_{j}s_{j}(B)^{q})^{1/q}$

Now,

$Tr(|A|^{1/p})=\sum_{j}s_{j}(A)^{1/p}$

and likewise for $B$. Therefore, we have proved the Matrix Holder inequality

$Tr(|AB|)\leq(Tr(|A|^{p}))^{1/p}.(Tr(|B|^{q}))^{1/q}$

for any two matrices $A,B$ such that $AB$ exists and is a square matrix. In
fact, the method of our proof implies immediately that if $\psi(.)$ is any
unitarily invariant matrix norm, then

$\psi(AB)\leq\psi(|A|^{p}).^{1/p}.\psi(|B|^{q})^{1/q}$

This is seen as follows. Since $\psi$ is a unitarily invariant matrix norm,

$\psi(A)=\Phi(s(A))$

where $\Phi()$ is a convex mapping from $\mathbb{R}_{+}^{n}$ into
$\mathbb{R}_{+}$ that satisfies the triangle inequality and in fact all
properties of a symmetric vector norm and also since

$a^{p}/p+b^{q}/q\geq ab,a,b>0$

we get for $t>0$,

$\Phi(x.y)=\phi(tx.y/t)\leq\phi(t^{p}x^{p})/p+\phi(y^{q}/t^{q})/q$ $\leq
t^{p}\phi(x^{p})/p+\phi(y^{q})/qt^{q}$

Now $ut^{p}+v/t^{q}$ is minimized when

$put^{p-1}-qv/t^{q+1}=0$

or equivalently, when

$t^{p+q}=qv/pu$

Substituting this value of $t$ immediately gives

$\Phi(x.y)\leq\Phi(x^{p})^{1/p}.\Phi(y^{q})^{1/q}$

and hence since

$s(A.B)\leq s(A).s(B)$

(in the sense of majorization), it follows that for two matrices $A,B$,

$\psi(A.B)=\Phi(s(A.B))\leq\Phi(s(A).s(B))\leq\Phi(s(A)^{p})^{1/p}.\Phi(s(B)^{q})^{1/q}$
$=\psi(|A|^{p})^{1/p}.\Phi(|B|^{q})^{1/q}$

Note that the eigenvalues (or equivalently the singular values) of $|A|^{p}$
are same as $s(A)^{p}$.

Remark: We are assuming that if $x,y$ are positive vectors such that $x<y$ in
the sense of majorization, then

$\Phi(x)\leq\Phi(y)$

This result follows since if $a<b$ are two positive real numbers, then

$a=tb+(1-t)0$

for $t=a/b\in[0,1]$ and by using the convexity of $\Phi$ and the fact that
$\Phi(0)=0$. If $x<y$, then
$\sum_{i=1}^{k}x_{i}\leq\sum_{i=1}^{k}y_{i},k=1,2,...,n$ and hence writing
$\xi_{k}=\sum_{i=1}^{k}x_{i},\eta_{k}=\sum_{i=1}^{k}y_{i},k=1,2,...,n$, we get
by regarding $\Phi(x)$ as a function of $\xi_{k},k=1,2,...,n$ and noting that
convexity of $\Phi$ in $x$ implies convexity of $\Phi$ in $\xi$ and the above
result that

$\Phi(\xi)=\Phi(\xi_{1},...,\xi_{n})\leq\Phi(\eta_{1},\xi_{2},...,\xi_{n})\leq$
$\Phi(\eta_{1},\eta_{2},\xi_{3},...,\xi_{n})\leq...\leq\Phi(\eta_{1},...,\eta_{n})=\Phi(\eta)$

##  Chapter 10 Lecture Plan for Statistical Signal Processing

Course Instructor:Harish Parthasarathy.

[1] Revision of basic probability theory: Probability spaces, random
variables, expectation, variance, covariance, characteristic function,
convergence of random variables, conditional expectation. (four lectures)

[2] Revision of basic random process theory:Kolmogorov’s existence theorem,
stationary stochastic processes, autocorrelation and power spectral density,
higher order moments and higher order spectra, Brownian motion and Poisson
processes, an introduction to stochastic differential equations. (four
lectures)

[3] Linear estimation theory: Estimation of parameters in linear models and
sequential linear models using least squares and weighted least squares
methods, estimation of parameters in linear models with coloured Gaussian
measurement noise using the maximum likelihood method, estimation of
parameters in time series models (AR,MA,ARMA) using least squares method with
forgetting factor, Estimation of parameters in time series models using the
minimum mean square method when signal correlations are known. Linear
estimation theory as a computation of an orthogonal projection of a Hilbert
space onto a subspace, the orthogonality principle and normal equations. (four
lectures)

[4] Estimation of non-random parameters using the Maximum likelihood method,
estimation of random parameters using the maximum aposteriori method, basic
statistical hypothesis testing, the Cramer-Rao lower bound. (four lectures)

[5] non-linear estimation theory: Optimal nonlinear minimum mean square
estimation as a conditional expectation, applications to non-Gaussian models.
(four lectures)

[6] Finite order linear prediction theory: The Yule-Walker equations for
estimating the AR parameters. Fast order recursive prediction using the
Levinson-Durbin algorithm, reflection coefficients and lattice filters,
stability in terms of reflection coefficients. (four lectures)

[7] The recursive least squares algorithm for parameter estimation linear
models, simultaneous time and order recursive prediction of a time series
using the RLS-Lattice filter. (four lectures)

[8] The non-causal Wiener filter, causal Wiener filtering using Wiener’s
spectral factorization method and using Kolmogorov’s innovations method,
Kolmogorov’s formula for the mean square infinite order prediction error.
(four lectures)

[9] An introduction to stochastic nonlinear filtering: The Kushner-Kallianpur
filter, the Extended Kalman filter approximation in continuous and discrete
time, the Kalman-Bucy filter for linear state variable stochastic models.
(four lectures)

[10] Power spectrum estimation using the periodogram and the windowed
periodogram, mean and variance of the periodogram estimate, high resolution
power spectrum estimation using eigensubspace methods like Pisarenko harmonic
decomposition, MUSIC and ESPRIT algorithms. Estimation of the power spectrum
using time series models. (four lectures)

[11] Some new topics:

[a] Quantum detection and estimation theory, Quantum Shannon coding theory.
(four lectures)

[b] Quantum filtering theory (four lectures)

Total: Forty eight lectures.

[1] Large deviations for the empirical distribution of a stationary Markov
process.

[2] Proof of the Gartner-Ellis theorem.

[3] Large deviations in the iid hypothesis testing problem.