$\displaystyle\leq\sqrt{\mathbb{E}\\!\left[\lVert{\bm{\delta}^{(1)}_{i}}\rVert^{2}\right]}\sigma_{\lVert{\mathbf{w}_{\ell}-\mathbf{v}_{\ell}}\rVert}+\mathbb{E}\\!\left[\lVert{\bm{\delta}^{(1)}_{i}}\rVert\right]\mathbb{E}\\!\left[\lVert{\mathbf{w}_{\ell}-\mathbf{v}_{\ell}}\rVert\right],$

where
$\operatorname{cov}(X,Y)\triangleq\operatorname{{\mathbb{E}}}[(X-\operatorname{{\mathbb{E}}}(X))(Y-\operatorname{{\mathbb{E}}}(Y))]$
is the covariance between $X$ and $Y$ and $\sigma^{2}_{X}$ represents the
variance of the random variable $X$. The last inequality comes from the fact
that $\operatorname{cov}(x,y)\leq\sigma_{x}\sigma_{y}$ and
$\sigma^{2}_{x}\leq\mathbb{E}\\!\left[x^{2}\right]$. Besides this, it holds
from $\lVert{\bm{\delta}^{(1)}_{i}}\rVert^{2}\sim\Gamma_{d}(N_{T},1)$ that
${\mathbb{E}\\!\left[\lVert{\bm{\delta}^{(1)}_{i}}\rVert^{2}\right]}={N_{T}}$.
From this point and the fact that
$\mathbb{E}\\!\left[x\right]\leq\sqrt{\mathbb{E}\\!\left[x^{2}\right]}$, we
can write

$\displaystyle{\mathbb{E}}\big{[}{|\bm{\delta}^{(1)}_{i}(\mathbf{w}_{\ell}-\mathbf{v}_{\ell})|}\big{]}$
$\displaystyle\leq\sqrt{N_{T}}\left(\sigma_{\lVert{\mathbf{w}_{\ell}-\mathbf{v}_{\ell}}\rVert}+\mathbb{E}\\!\left[\lVert{\mathbf{w}_{\ell}-\mathbf{v}_{\ell}}\rVert\right]\right)$
(117) $\displaystyle\overset{(a)}{\leq}\sqrt{N_{T}}\
2\sqrt{\mathbb{E}\\!\left[\lVert{\mathbf{w}_{\ell}-\mathbf{v}_{\ell}}\rVert^{2}\right]}$
$\displaystyle\overset{(b)}{=}{{\mathcal{O}}}({\bar{P}}^{-\alpha_{q}}),$

where $(a)$ comes from the fact that $\sigma_{x}+{\mathbb{E}}[x]\leq
2\sqrt{{\mathbb{E}}[x^{2}]}$ and $(b)$ from Corollary 3. ∎

##  Appendix E Proof of Lemma 1 (Quantizer Consistency)

Let $q\triangleq{\bar{P}}^{-\alpha_{q}}$ be the quantization step size of the
quantizer ${{\mathcal{Q}}}_{u}$. Then, ${{\mathcal{Q}}}_{u}$ is defined such
that, for a scalar value $x\in{{\mathbb{R}}}$,

$\displaystyle{{\mathcal{Q}}}_{u}(x)\triangleq
q\left\lfloor\frac{x}{q}+\frac{1}{2}\right\rfloor.$ (118)

We extend the notation for any complex matrix
$\mathbf{A}\in{{\mathbb{C}}}^{n\times m}$ such that
$\mathbf{A}_{q}={{\mathcal{Q}}}_{u}(\mathbf{A})$ denotes the element-wise
quantization, i.e.,

$\displaystyle(\mathbf{A}_{q})_{i,k}\triangleq{{\mathcal{Q}}}_{u}\big{(}\operatorname{Re}(\mathbf{A}_{i,k})\big{)}+\mathrm{i}{{\mathcal{Q}}}_{u}\big{(}\operatorname{Im}(\mathbf{A}_{i,k})\big{)},$
(119)

where $\operatorname{Re}(x)$ and $\operatorname{Im}(x)$ stand for the real
imaginary part of $x\in{{\mathbb{C}}}$, and $\mathrm{i}\triangleq\sqrt{-1}$.
In this appendix we prove that, for a scalar uniform quantizer
${{\mathcal{Q}}}_{u}$ with $q={\bar{P}}^{-\alpha_{q}}$ and
$\alpha^{(j)}>\alpha_{q}>0$, $\forall j\in{{\mathbb{N}}}_{M}$, it follows that

$\displaystyle\Pr\left({{\hat{\mathbf{H}}}}^{(j)\leftarrow(1)}_{q}\neq{{\hat{\mathbf{H}}}}^{(j)}_{q}\right)=o\left(\frac{1}{\log_{2}(P)}\right),$
(120)

where
${{\hat{\mathbf{H}}}}^{(j)}_{q}={{\mathcal{Q}}}_{u}({{\hat{\mathbf{H}}}}^{(j)})$
and ${{\hat{\mathbf{H}}}}^{(j)\leftarrow(1)}_{q}$ is the MAP estimator of
${{\hat{\mathbf{H}}}}^{(j)}_{q}$ given ${{\hat{\mathbf{H}}}}^{(1)}$. We start
by noting that, by definition of the MAP estimator,

$\displaystyle\Pr\left({{\hat{\mathbf{H}}}}^{(j)\leftarrow(1)}_{q}\neq{{\hat{\mathbf{H}}}}^{(j)}_{q}\right)$
$\displaystyle\leq\Pr\left({{\mathcal{Q}}}_{u}({{\hat{\mathbf{H}}}}^{(1)})\neq{{\hat{\mathbf{H}}}}^{(j)}_{q}\right).$
(121)

Since $\operatorname{Re}({{\hat{\mathbf{H}}}}^{(1)}_{i,k})$ and
$\operatorname{Im}({{\hat{\mathbf{H}}}}^{(1)}_{i,k})$ are i.i.d. for any
$i,k$, it follows that

$\displaystyle\Pr\left({{\mathcal{Q}}}_{u}({{\hat{\mathbf{H}}}}^{(1)})\neq{{\hat{\mathbf{H}}}}^{(j)}_{q}\right)$
$\displaystyle\leq
2KN_{T}\Pr\left({{\mathcal{Q}}}_{u}(\operatorname{Re}({{\hat{\mathbf{H}}}}^{(1)}_{1,1}))\neq{{\mathcal{Q}}}_{u}(\operatorname{Re}({{\hat{\mathbf{H}}}}^{(j)}_{1,1}))\right),$
(122)

where we have selected w.l.o.g. the real part of the (1,1) channel element.
Hence, it is sufficient to obtain the probability of disagreement for
$\operatorname{Re}({{\hat{\mathbf{H}}}}^{(j)}_{1,1})$. For that purpose, we
split each reconstruction level of the quantizer in two parts: The _edge_ of
the cell and the _center_ of the cell. This is done in order to show that, as
$P$ increases, the probability of disagreement vanishes if
${{\hat{\mathbf{H}}}}^{(1)}_{1,1}$ is in the _center_ of the quantization
level and, besides this, the probability that
${{\hat{\mathbf{H}}}}^{(1)}_{1,1}$ is in the _edge_ area also vanishes. We
rigorously show it in the following. Before starting, we introduce the
simplified notation
${\mathrm{h}}^{(j)}\triangleq\operatorname{Re}({{\hat{\mathbf{H}}}}^{(j)}_{1,1})$
to ease the readability. Accordingly, we also introduce the notation
${\mathrm{h}}\triangleq\operatorname{Re}(\mathbf{H}_{1,1})$ and
$\delta\triangleq\operatorname{Re}(\bm{\delta}^{(j)}_{1,1})$ such that
${\mathrm{h}}^{(j)}=\breve{z}^{(j)}{\mathrm{h}}+z^{(j)}\delta^{(j)}$, with
$z^{(j)}={\bar{P}}^{-\alpha^{(j)}}$ and
$\breve{z}^{(j)}\triangleq\sqrt{1-(z^{(j)})^{2}}$. Furthermore, we recall the
notations
$\breve{z}^{(j)}_{\operatorname{inv}}\triangleq\frac{1}{\breve{z}^{(j)}}$ and
$z^{(j)}_{n}\triangleq\frac{z^{(j)}}{\breve{z}^{(j)}}$, introduced in Appendix
B.

###  E-A Egde and center of the reconstruction level

Let $\ell_{n}$ be the $n$-th quantization level of ${{\mathcal{Q}}}_{u}$,
$n\in{{\mathbb{Z}}}$, with $\ell_{0}=0$. Let us define $L_{n}$ as the input
interval that outputs $\ell_{n}$, i.e.,

$\displaystyle L_{n}\triangleq\\{x\mid{{\mathcal{Q}}}_{u}(x)=\ell_{n}\\}.$
(123)

$L_{n}$ has a range $[L^{\min}_{n},L^{\max}_{n})$ such that $|L_{n}|\triangleq
L^{\max}_{n}-L^{\min}_{n}={\bar{P}}^{-\alpha_{q}}$. We split $L_{n}$ in two
areas, the _edge_ area $E_{n}$ and the _center_ area $C_{n}$, depicted in Fig.
8. The _edge_ area is defined as the part of $L_{n}$ that is at most at
distance ${\bar{P}}^{{-c_{e}\alpha_{q}}}$ of the boundary of the cell,with
$c_{e}>1$.

$\displaystyle E_{n}\triangleq\left\\{x\in L_{n}\mid
x-L^{\min}_{n}<{\bar{P}}^{-c_{e}\alpha_{q}}\ \lor\
L^{\max}_{n}-x<{\bar{P}}^{{-c_{e}\alpha_{q}}}\right\\}.$ (124)

The _center_ area is given by

$\displaystyle C_{n}\triangleq\left\\{x\in L_{n}\backslash E_{n}\right\\}.$
(125)

Intuitively, the probability of disagreement is very high if
${\mathrm{h}}^{(1)}$ lies in the edge area $E_{n}$, whereas this probability
vanishes in the central area $C_{n}$. Mathematically, we have that

$\displaystyle\Pr\bigg{(}{{\mathcal{Q}}}_{u}({\mathrm{h}}^{(1)})\neq{{\mathcal{Q}}}_{u}({\mathrm{h}}^{(j)})\bigg{)}\leq\Pr\bigg{(}{\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}E_{n}\bigg{)}+\Pr\bigg{(}{{\mathcal{Q}}}_{u}({\mathrm{h}}^{(1)})\neq{{\mathcal{Q}}}_{u}({\mathrm{h}}^{(j)})\mid{\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}C_{n}\bigg{)}.$
(126)

Let us analyze separately the two probabilities in the right-hand side of
(126).

\begin{overpic}[width=234.15189pt]{./TIT2019_FIG_quant_error.pdf}
{\put(16.0,-2.25){\parbox{433.62pt}{\small$\underbrace{\phantom{Thisisareallyblankspace}}_{L_{n}}$
}}}
{\put(7.4,9.5){\parbox{433.62pt}{\small$\overbrace{\phantom{ccici\,}}^{E_{n-1}}$
}}}
{\put(17.0,9.5){\parbox{433.62pt}{\small$\overbrace{\phantom{ccici\,}}^{E_{n}}$
}}}
{\put(46.5,9.5){\parbox{433.62pt}{\small$\overbrace{\phantom{ccici\;}}^{E_{n}}$
}}}
{\put(25.5,9.2){\parbox{433.62pt}{\small$\overbrace{\phantom{omakespace\,}}^{C_{n}}$
}}}
{\put(65.75,9.2){\parbox{433.62pt}{\small$\overbrace{\phantom{omakespace\,}}^{C_{n+1}}$
}}}
{\put(57.0,9.75){\parbox{433.62pt}{\small$\overbrace{\phantom{ccici\,}}^{E_{n+1}}$
}}}
{\put(87.0,9.75){\parbox{433.62pt}{\small$\overbrace{\phantom{ccici\,}}^{E_{n+1}}$
}}} {\put(14.5,16.25){\parbox{433.62pt}{\small$\ell_{n}$ }}}
{\put(52.6,16.25){\parbox{433.62pt}{\small$\ell_{n+1}$ }}}
{\put(92.6,16.25){\parbox{433.62pt}{\small$\ell_{n+2}$ }}} \end{overpic}
Figure 8: Illustration of a reconstruction level $L_{n}$ of the quantizer and
the two sub-areas in which we divide it: The central area $C_{n}$ and the edge
area $E_{n}$.

###  E-B Probability of belonging to the edge area

Consider an arbitrary quantization level $\ell_{n}$. Let $f^{\max}_{L_{n}}$ be
the maximum value of the pdf of ${\mathrm{h}}^{(1)}$ in
$L_{n}=\\{x\mid{{\mathcal{Q}}}_{u}(x)=\ell_{n}\\}$. It follows that the
probability that ${\mathrm{h}}^{(1)}$ is in $E_{n}$ is upper-bounded by

$\displaystyle\Pr\big{(}{\mathrm{h}}^{(1)}\in E_{n}\big{)}$ $\displaystyle\leq
f^{\max}_{L_{n}}|E_{n}|$ (127)
$\displaystyle=2f^{\max}_{L_{n}}{{\bar{P}}^{{-c_{e}\alpha_{q}}}},$

where $|E_{n}|$ denotes the length of $E_{n}$. The standard normal
distribution has a derivative that is, at most, $1/\sqrt{2\pi e}$. Thus, the
probability of being in $L_{n}$ satisfies

$\displaystyle\Pr\big{(}{\mathrm{h}}^{(1)}\in L_{n}\big{)}$
$\displaystyle\geq\big{(}f^{\max}_{L_{n}}-1/\sqrt{2\pi
e}|L_{n}|\big{)}|L_{n}|$ (128)
$\displaystyle=\big{(}f^{\max}_{L_{n}}-1/\sqrt{2\pi
e}{\bar{P}}^{-\alpha_{q}}\big{)}{{\bar{P}}^{-\alpha_{q}}}.$

Hence, the probability that ${\mathrm{h}}^{(1)}$ is in $E_{n}$, given that it
is in $L_{n}$, satisfies for any $n$ that

$\displaystyle\Pr\big{(}{\mathrm{h}}^{(1)}\in E_{n}\mid L_{n}\big{)}$
$\displaystyle=\frac{\Pr\left({\mathrm{h}}^{(1)}\in
E_{n}\right)}{\Pr\left({\mathrm{h}}^{(1)}\in L_{n}\right)}$ (129)
$\displaystyle\leq\frac{2f^{\max}_{L_{n}}}{\big{(}f^{\max}_{L_{n}}-1/\sqrt{2\pi
e}{\bar{P}}^{-\alpha_{q}}\big{)}}{\bar{P}}^{-(c_{e}-1)\alpha_{q}}.$

Let us define $g_{\max}$ as
$g_{\max}\triangleq\max_{n\in{{\mathbb{Z}}}}\frac{2f^{\max}_{L_{n}}}{\big{(}f^{\max}_{L_{n}}-1/\sqrt{2\pi
e}{\bar{P}}^{-\alpha_{q}}\big{)}}$. Note that $g_{\max}=\Theta(1)$. Hence,
from (129) and the fact that
$\sum_{n\in{{\mathbb{Z}}}}\Pr\big{(}{\mathrm{h}}^{(1)}\in L_{n}\big{)}=1$ we
can write

$\displaystyle\Pr\big{(}{\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}E_{n}\big{)}$
$\displaystyle=\sum_{n\in{{\mathbb{Z}}}}\frac{\Pr\left({\mathrm{h}}^{(1)}\in
E_{n}\right)}{\Pr\left({\mathrm{h}}^{(1)}\in
L_{n}\right)}\Pr\big{(}{\mathrm{h}}^{(1)}\in L_{n}\big{)}$ (130)
$\displaystyle\leq
g_{\max}{\bar{P}}^{-(c_{e}-1)\alpha_{q}}\sum_{n\in{{\mathbb{Z}}}}\Pr\big{(}{\mathrm{h}}^{(1)}\in
L_{n}\big{)}$
$\displaystyle={{\mathcal{O}}}\big{(}{\bar{P}}^{-(c_{e}-1)\alpha_{q}}\big{)}.$

Consequently, it holds that

$\displaystyle\Pr\big{(}{\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}E_{n}\big{)}$
$\displaystyle=o\bigg{(}\frac{1}{\log_{2}(P)}\bigg{)}.$ (131)

###  E-C Probability of disagreement in the center area

From the fact that the minimum distance from any point of $C_{n}$ to the
border of $L_{n}$ is ${\bar{P}}^{-c_{e}\alpha_{q}}$, it holds that

$\displaystyle\Pr\left(\mathcal{Q}_{u}({\mathrm{h}}^{(1)})\neq\mathcal{Q}_{u}({\mathrm{h}}^{(j)})\
\Big{|}\ {\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}C_{n}\right)$
$\displaystyle\leq\Pr\left(\big{|}{{\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}}\big{|}\geq{\bar{P}}^{-c_{e}\alpha_{q}}\
\Big{|}\ {\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}C_{n}\right).$ (132)

Given that, for two events $A,C$, $\Pr(A\mid C)\leq\Pr(A)/\Pr(C)$, it follows
that

$\displaystyle
Pr\left(\big{|}{{\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}}\big{|}\geq{\bar{P}}^{-c_{e}\alpha_{q}}\
\Big{|}\ {\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}C_{n}\right)$
$\displaystyle\leq\,\frac{1}{\Pr\left({\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}C_{n}\right)}\Pr\left(|{\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}|\geq{\bar{P}}^{-c_{e}\alpha_{q}}\right)$
(133)
$\displaystyle\leq\frac{1}{\Pr\left({\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}C_{n}\right)}\
\frac{{{\mathbb{E}}}\left[\big{|}{\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}\big{|}\right]}{{\bar{P}}^{-c_{e}\alpha_{q}}},$

what comes from Markov’s Inequality. In the following, we obtain the
expectation
${{\mathbb{E}}}\left[\big{|}{\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}\big{|}\right]$.
Then,

$\displaystyle{\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}$
$\displaystyle={\mathrm{h}}(\breve{z}^{(1)}-\breve{z}^{(j)})+(z^{(1)}\delta^{(1)}-z^{(j)}\delta^{(j)}).$
(134)

From the assumption of Gaussian variables, it holds that

$\displaystyle{\mathrm{h}}(\breve{z}^{(1)}-\breve{z}^{(j)})\ $ $\displaystyle\
\sim\ {{\mathcal{N}}}\left(0,\ (\breve{z}^{(1)}-\breve{z}^{(j)})^{2}\right),$
(135) $\displaystyle z^{(1)}\delta^{(1)}-z^{(j)}\delta^{(j)}\ $
$\displaystyle\ \sim\ {{\mathcal{N}}}\left(0,\
(z^{(1)})^{2}+(z^{(j)})^{2}\right).$ (136)

Since ${\mathrm{h}}^{(1)}$ is independent of $\delta^{(1)}$ and
$\delta^{(j)}$, it follows that

$\displaystyle{\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}\ \sim\
{{\mathcal{N}}}\left(0,\sigma^{2}_{d}\right),$ (137)

where $\sigma^{2}_{d}$ is given by

$\displaystyle\sigma^{2}_{d}=(\breve{z}^{(1)}-\breve{z}^{(j)})^{2}+(z^{(1)})^{2}+(z^{(j)})^{2}.$
(138)

Substituting the variables for their values yields

$\displaystyle\sigma^{2}_{d}=2\left(1-\sqrt{1-P^{-\alpha^{(1)}}-P^{-\alpha^{(j)}}+P^{-\alpha^{(1)}-\alpha^{(j)}}}\right).$
(139)

Furthermore, if ${\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}$ is drawn from a zero-
mean Normal distribution of variance $\sigma^{2}_{d}$,
$|{\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}|$ is distributed as a half-normal
distribution of mean

$\displaystyle{{\mathbb{E}}}\left[\big{|}{\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}\big{|}\right]$
$\displaystyle=\sigma_{d}\sqrt{\frac{2}{\pi}}.$ (140)

From (139) and the fact that for any $x$ such that $0\leq x\leq 1$ it holds
that $\sqrt{1-x}\geq 1-x$, it follows that

$\displaystyle{{\mathbb{E}}}\left[\big{|}{\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}\big{|}\right]$
$\displaystyle\leq\sqrt{\frac{4}{\pi}\left({P^{-\alpha^{(1)}}+P^{-\alpha^{(j)}}-P^{-\alpha^{(1)}-\alpha^{(j)}}}\right)}$
(141) $\displaystyle={{\mathcal{O}}}({\bar{P}}^{-\alpha^{(j)}}).$

Besides this, it holds from (131) that

$\displaystyle\Pr\Big{(}{\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}C_{n}\Big{)}$
$\displaystyle=1-\Pr\Big{(}{\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}E_{n}\Big{)}$
(142) $\displaystyle=1-{{\mathcal{O}}}({\bar{P}}^{-\alpha_{q}}).$

Both (141) and (142) lead to

$\displaystyle\Pr\left(\mathcal{Q}_{u}({\mathrm{h}}^{(1)})\neq\mathcal{Q}_{u}({\mathrm{h}}^{(j)})\
\Big{|}\ {\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}C_{n}\right)$
$\displaystyle\leq\frac{1}{\Pr\Big{(}{\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}C_{n}\Big{)}}\
\frac{{{\mathbb{E}}}\left[\big{|}{\mathrm{h}}^{(1)}-{\mathrm{h}}^{(j)}\big{|}\right]}{{\bar{P}}^{-c_{e}\alpha_{q}}}$
(143)
$\displaystyle={{\mathcal{O}}}\left({\bar{P}}^{c_{e}\alpha_{q}-\,\alpha^{(j)}}\right)$
$\displaystyle=o\left(\frac{1}{\log_{2}(P)}\right).$

The last inequality is obtained only if $c_{e}\alpha_{q}<\alpha^{(j)}$ for any
$j\in{{\mathbb{N}}}_{M}$. Thus, it follows from (143) that it is necessary to
satisfy that $c_{e}\alpha_{q}<\alpha^{(j)}$ for any $j\in{{\mathbb{N}}}_{M}$.
Since for any $\alpha_{q}<\alpha^{(j)}$ we can find a $c_{e}>1$ such that
$c_{e}\alpha_{q}<\alpha^{(j)}$, any $\alpha_{q}<\alpha^{(j)}$ will satisfy
(143) as long as a correct $c_{e}$ is selected.

###  E-D Assembling probabilities

Recalling (126), we make use of (131) and (143) to show that

$\displaystyle\Pr\bigg{(}\\!{{\mathcal{Q}}}_{u}({\mathrm{h}}^{(1)})\neq{{\mathcal{Q}}}_{u}({\mathrm{h}}^{(j)})\\!\bigg{)}$
$\displaystyle\leq\Pr\bigg{(}{\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}E_{n}\bigg{)}+\Pr\bigg{(}{{\mathcal{Q}}}_{u}({\mathrm{h}}^{(1)})\neq{{\mathcal{Q}}}_{u}({\mathrm{h}}^{(j)})\mid{\mathrm{h}}^{(1)}\in\bigcup_{n\in{{\mathbb{Z}}}}C_{n}\bigg{)}$
(144) $\displaystyle=o\left(\frac{1}{\log_{2}(P)}\right),$

what concludes the proof of Lemma 1. ∎

##  Appendix F Proof of Lemma 5 (Error on Naive Precoder)

In this appendix we show that

$\displaystyle\operatorname{{\mathbb{E}}}\left[\lVert{\mathbf{v}_{i,2}-\mathbf{w}_{i,2}}\rVert^{2}\right]={{\mathcal{O}}}\left(P^{-\alpha_{q}}\right).$
(145)

Then, (145) is straightforwardly generalized for any
$\operatorname{{\mathbb{E}}}\left[\lVert{\mathbf{v}_{i,k}-\mathbf{w}_{i,k}}\rVert^{2}\right]$,
$k\in{{\mathbb{N}}}_{M}\backslash 1$. In order to prove (145), we make use of
the fact that, as presented in Section III-A, we assume that the precoding
vectors and matrices can be expressed as a combination of summations,
products, and generalized inverses of the channel estimate. Note that with the
previous operations, it is also possible to compute divisions and norms of the
channel estimate coefficients.

First, note that both $\mathbf{w}_{i,2}$ and $\mathbf{v}_{i,2}$ are obtained
following the same algorithm but based on different information (input).
Specifically, $\mathbf{w}_{i,2}$ is computed on the basis of
${{\hat{\mathbf{H}}}}^{(2)}_{q}={{\mathcal{Q}}}_{u}({{\hat{\mathbf{H}}}}^{(2)})$,
where ${{\mathcal{Q}}}_{u}$ is a scalar uniform quantizer with quantization
step $q={\bar{P}}^{-\alpha_{q}}$, and $\mathbf{v}_{i,2}$ on the basis of
${{\hat{\mathbf{H}}}}^{(1)}$. Similarly as in the previous appendix, let
${\mathrm{h}}^{(2)}_{q}$ (resp. ${\mathrm{h}}^{(j)}$ and ${\mathrm{h}}$)
denote the real or imaginary part of an arbitrary element of the matrix
${{\hat{\mathbf{H}}}}^{(2)}_{q}$ (resp. ${{\hat{\mathbf{H}}}}^{(j)}$ and
$\mathbf{H}$). Let us define ${\mathrm{h}}^{(2)}_{\varsigma}$ as

$\displaystyle{\mathrm{h}}^{(2)}_{\varsigma}\triangleq{\mathrm{h}}^{(2)}+\varsigma_{q},$
(146)

where $\varsigma_{q}$ is distributed as a binary symmetric distribution with
points $[-q,\,q]$, independent of the other variables, such that
$\sigma_{\varsigma_{q}}^{2}=q^{2}$. Note that the error
${\mathrm{h}}^{(2)}_{q}-{\mathrm{h}}^{(1)}$ has smaller or equal variance than
${\mathrm{h}}^{(2)}_{\varsigma}-{\mathrm{h}}^{(1)}={\mathrm{h}}^{(2)}-{\mathrm{h}}^{(1)}+\varsigma_{q}$.
Hence, we can assume that $\mathbf{w}_{i,2}$ is computed on the basis of the
estimate ${\mathrm{h}}^{(2)}_{\varsigma}$ as increasing the error variance can
only hurt. Consequently, the error
$\xi\triangleq{\mathrm{h}}^{(2)}_{\varsigma}-{\mathrm{h}}^{(1)}$ has a
variance

$\displaystyle\sigma_{\xi}^{2}$
$\displaystyle\leq\sigma^{2}_{d}+\sigma_{\varsigma_{q}}^{2}+2\sigma_{d}\sigma_{\varsigma_{q}}$
(147) $\displaystyle={{\mathcal{O}}}(P^{-\alpha_{q}}),$

where $\sigma^{2}_{d}$ is given in (139). Therefore, we can write that

$\displaystyle{\mathrm{h}}^{(2)}_{\varsigma}={\mathrm{h}}^{(1)}+{\bar{P}}^{-\alpha_{q}}\delta_{\xi},$
(148)

where $\delta_{\xi}$ is a variable with variance $\Theta(1)$ and bounded
density. We continue by showing that the error variance scaling remains being
at most $\Theta(P^{-\alpha_{q}})$ after applying addition, product, inverse or
pseudo-inverse operations. Afterward, based on those results, we prove (145).

###  F-A Error in the addition

Let $\mathrm{a}^{(2)}_{\xi}$, $\mathrm{b}^{(2)}_{\xi}$, be distributed as
(148), i.e.,
$\mathrm{a}^{(2)}_{\xi}\triangleq\mathrm{a}^{(1)}+{\bar{P}}^{-\alpha_{q}}\delta^{\mathrm{a}}_{\xi}$,
$\mathrm{b}^{(2)}_{\xi}\triangleq\mathrm{b}^{(1)}+{\bar{P}}^{-\alpha_{q}}\delta^{\mathrm{b}}_{\xi}$,
where $\delta^{\mathrm{a}}_{\xi},\delta^{\mathrm{b}}_{\xi}$, are variables
with variance $\Theta(1)$ and bounded density. It is easy to see that, for any
$\mathrm{a}^{(2)}_{\xi}$, $\mathrm{b}^{(2)}_{\xi}$,

$\displaystyle\mathrm{a}^{(2)}_{\xi}+\mathrm{b}^{(2)}_{\xi}$
$\displaystyle=\mathrm{a}^{(1)}+{\bar{P}}^{-\alpha_{q}}\delta^{\mathrm{a}}_{\xi}+\mathrm{b}^{(1)}+{\bar{P}}^{-\alpha_{q}}\delta^{\mathrm{b}}_{\xi}$
(149)
$\displaystyle=\mathrm{a}^{(1)}+\mathrm{b}^{(1)}+{\bar{P}}^{-\alpha_{q}}(\delta^{\mathrm{a}}_{\xi}+\delta^{\mathrm{b}}_{\xi}).$

This implies that the error variance of the sum is also
${{\mathcal{O}}}(P^{-\alpha_{q}})$ as (147).

###  F-B Error in the product

In a similar way, it follows that

$\displaystyle\mathrm{a}^{(2)}_{\xi}\mathrm{b}^{(2)}_{\xi}$
$\displaystyle=\left(\mathrm{a}^{(1)}+{\bar{P}}^{-\alpha_{q}}\delta^{\mathrm{a}}_{\xi}\right)\left(\mathrm{b}^{(1)}+{\bar{P}}^{-\alpha_{q}}\delta^{\mathrm{b}}_{\xi}\right)$
(150)
$\displaystyle=\mathrm{a}^{(1)}\mathrm{b}^{(1)}+{\bar{P}}^{-\alpha_{q}}\left(\mathrm{a}^{(1)}\delta^{\mathrm{b}}_{\xi}+\mathrm{b}^{(1)}\delta^{\mathrm{a}}_{\xi}+{\bar{P}}^{-\alpha_{q}}\delta^{\mathrm{a}}_{\xi}\delta^{\mathrm{b}}_{\xi}\right),$

what implies that the product also maintains the scaling of the variance as
${{\mathcal{O}}}(P^{-\alpha_{q}})$. Moreover, as the sum and product of
matrices is a composition of sums and products of its coefficients, the result
extends to any two matrices of suitable dimension.

###  F-C Error in the inverse

Let us first assume that $\mathbf{A}^{(2)}_{\xi}$ and $\mathbf{A}^{(1)}$ are
square matrices with full rank with probability one, and with coefficients
following (148). We can then write that

$\displaystyle(\mathbf{A}^{(2)}_{\xi})^{-1}$
$\displaystyle=\left(\mathbf{A}^{(1)}+{\bar{P}}^{-\alpha_{q}}\bm{\Delta}^{\mathbf{A}}_{\xi}\right)^{-1}$
(151)
$\displaystyle=(\mathbf{A}^{(1)})^{-1}-{\bar{P}}^{-\alpha_{q}}\,(\mathbf{A}^{(1)})^{-1}\bm{\Delta}^{\mathbf{A}}_{\xi}\left(\mathbf{A}^{(1)}+{\bar{P}}^{-\alpha_{q}}\bm{\Delta}^{\mathbf{A}}_{\xi}\right)^{-1},$

which is obtained from the Woodbury matrix identity[49]. Hence, the error
variance of the inverse is again ${{\mathcal{O}}}(P^{-\alpha_{q}})$. Once that
it is proved that the inverse operation generates an error with variance
${{\mathcal{O}}}(P^{-\alpha_{q}})$, we extend it for the Moore–Penrose inverse
(pseudo-inverse) $(\mathbf{A}^{(2)}_{\xi})^{\dagger}$. We assume (as
throughout the rest of the document) that each sub-matrix has maximum rank,
i.e.,

$\displaystyle\operatorname{rank}\left(\mathbf{A}^{(2)}_{\xi}\in{{\mathbb{C}}}^{N\times
M}\right)=\min(N,M).$ (152)

Let us assume that $\mathbf{A}^{(2)}_{\xi}$ is full row-rank matrix, i.e.,
$N\leq M$. Under this assumption, the pseudo-inverse is given by

$\displaystyle(\mathbf{A}^{(2)}_{\xi})^{\dagger}=(\mathbf{A}^{(2)}_{\xi})^{{\mathrm{H}}}\left(\mathbf{A}^{(2)}_{\xi}(\mathbf{A}^{(2)}_{\xi})^{{\mathrm{H}}}\right)^{-1}.$
(153)

The case in which $\mathbf{A}^{(2)}_{\xi}$ is full column-rank matrix ($N\geq
M$) will follow the same steps and thus we omit it. It follows from (150) that
$\mathbf{A}^{(2)}_{\xi}(\mathbf{A}^{(2)}_{\xi})^{{\mathrm{H}}}=\mathbf{A}^{(1)}(\mathbf{A}^{(1)})^{{\mathrm{H}}}+{\bar{P}}^{-\alpha_{q}}\bm{\Delta}_{\text{eq}}$,
where $\bm{\Delta}_{\text{eq}}$ has variance $\Theta(1)$. This, together with
(151), implies that

$\displaystyle\left(\mathbf{A}^{(2)}_{\xi}(\mathbf{A}^{(2)}_{\xi})^{{\mathrm{H}}}\right)^{-1}=\left(\mathbf{A}^{(1)}(\mathbf{A}^{(1)})^{{\mathrm{H}}}\right)^{-1}+{\bar{P}}^{-\alpha_{q}}\bm{\Delta}^{\prime}_{\text{eq}},$
(154)

and by applying again (150) it holds

$\displaystyle(\mathbf{A}^{(2)}_{\xi})^{\dagger}=(\mathbf{A}^{(1)})^{{\mathrm{H}}}\left(\mathbf{A}^{(1)}(\mathbf{A}^{(1)})^{{\mathrm{H}}}\right)^{-1}+{\bar{P}}^{-\alpha_{q}}\bm{\Delta}^{{}^{\prime\prime}}_{\text{eq}},$
(155)

where $\bm{\Delta}^{\prime}_{\text{eq}}$ and
$\bm{\Delta}^{{}^{\prime\prime}}_{\text{eq}}$ have variance $\Theta(1)$. As
explained in [45], under the assumption that $\hat{\mathbf{X}}$ is a full row-
rank matrix, any generalized inverse may be expressed as
$\hat{\mathbf{X}}^{-}=\hat{\mathbf{X}}^{\dagger}+\mathbf{P}_{\bot}\mathbf{U}$.
Hence, similar result could be obtained for a broad set of generalized
inverse.

###  F-D Error variance of the difference of precoders

The centralized precoder $\mathbf{v}_{i,2}$ is based on
${{\hat{\mathbf{H}}}}^{(1)}$, i.e.,
$\mathbf{V}={{\mathcal{V}}}({{\hat{\mathbf{H}}}}^{(1)})$. The distributed
precoder at TX 2 is based on its own quantized CSIT
${{\hat{\mathbf{H}}}}^{(2)}_{q}$, then $\mathbf{w}_{i,2}$ is obtained from
$\mathbf{W}={{\mathcal{V}}}({{\hat{\mathbf{H}}}}^{(2)}_{q})$. Based on the
previous results and the definition of linear precoders, it follows that we
can write the distributed precoder based on the CSIT of TX $2$
(${{\hat{\mathbf{H}}}}^{(2)}_{q}$) as

$\displaystyle\mathbf{w}_{i,2}=\mathbf{v}_{i,2}+{\bar{P}}^{-\alpha_{q}}\mathbf{e}_{\mathbf{w}},$
(156)

where
$\operatorname{{\mathbb{E}}}\big{[}\lVert{\mathbf{e}_{\mathbf{w}}}\rVert^{2}\big{]}={{\mathcal{O}}}(1)$.
Consequently,

$\displaystyle\operatorname{{\mathbb{E}}}\left[\lVert{\mathbf{v}_{i,2}-\mathbf{w}_{i,2}}\rVert^{2}\right]$
$\displaystyle=\operatorname{{\mathbb{E}}}\left[\lVert{\mathbf{v}_{i,2}-(\mathbf{v}_{i,2}+{\bar{P}}^{-\alpha_{q}}\mathbf{e}_{\mathbf{w}})}\rVert^{2}\right]$
(157)
$\displaystyle=P^{-\alpha_{q}}\operatorname{{\mathbb{E}}}\left[\lVert{\mathbf{e}_{\mathbf{w}}}\rVert^{2}\right]$
$\displaystyle={{\mathcal{O}}}\left(P^{-\alpha_{q}}\right).$

Moreover, since
$\operatorname{{\mathbb{E}}}[\lVert{\mathbf{x}}\rVert]\leq\sqrt{\operatorname{{\mathbb{E}}}[\lVert{\mathbf{x}}\rVert^{2}]}$,
it follows that

$\displaystyle\operatorname{{\mathbb{E}}}\left[\lVert{\mathbf{v}_{i,j}-\mathbf{w}_{i,j}}\rVert\right]$
$\displaystyle={{\mathcal{O}}}\left({\bar{P}}^{-\alpha_{q}}\right),$ (158)

what concludes the proof of Lemma 5. ∎

## References

  * [1] M. Costa and A. E. Gamal, “The capacity region of the discrete memoryless interference channel with strong interference (Corresp.),” _IEEE Trans. Inf. Theory_ , vol. 33, no. 5, pp. 710–711, Sep. 1987.
  * [2] G. Caire and S. Shamai (Shitz), “On the achievable throughput of a multiantenna Gaussian Broadcast Channel,” _IEEE Trans. Inf. Theory_ , vol. 49, no. 7, pp. 1691–1706, 2003.
  * [3] A. Lozano, A. M. Tulino, and S. Verdú, “High-SNR power offset in multiantenna communication,” _IEEE Trans. Inf. Theory_ , vol. 51, no. 12, pp. 4134–4151, Dec 2005.
  * [4] B. Hassibi and M. Sharif, “Fundamental Limits in MIMO Broadcast Channels,” _IEEE Journal on Selected Areas in Communications_ , vol. 25, no. 7, pp. 1333–1344, Sep. 2007.
  * [5] N. Jindal and A. Goldsmith, “Dirty-paper coding versus TDMA for MIMO Broadcast Channels,” _IEEE Trans. Inf. Theory_ , vol. 51, no. 5, pp. 1783–1794, May 2005.
  * [6] V. R. Cadambe and S. A. Jafar, “Interference alignment and degrees of freedom of the K-user interference channel,” _IEEE Trans. Inf. Theory_ , vol. 54, no. 8, pp. 3425–3441, Aug. 2008.
  * [7] P. Viswanath and D. N. C. Tse, “Sum capacity of the vector Gaussian Broadcast Channel and uplink-downlink duality,” _IEEE Trans. Inf. Theory_ , vol. 49, no. 8, pp. 1912–1921, Aug. 2003.
  * [8] H. Weingarten, Y. Steinberg, and S. Shamai (Shitz), “The capacity region of the gaussian multiple-input multiple-output broadcast channel,” _IEEE Trans. Inf. Theory_ , vol. 52, no. 9, pp. 3936–3964, Sep. 2006.
  * [9] R. Etkin, D. Tse, and H. Wang, “Gaussian interference channel capacity to within one bit,” _IEEE Trans. Inf. Theory_ , vol. 54, no. 12, pp. 5534–5562, Dec. 2008.
  * [10] M. Kountouris, R. de Francisco, D. Gesbert, D. T. M. Slock, and T. Salzer, “Multiuser diversity - multiplexing tradeoff in mimo broadcast channels with limited feedback,” in _Proc. Asilomar Conf. on Signals, Systems and Computers_ , Oct 2006, pp. 364–368.
  * [11] A. S. Motahari and A. K. Khandani, “Capacity Bounds for the Gaussian Interference Channel,” _IEEE Trans. Inf. Theory_ , vol. 55, no. 2, pp. 620–643, Feb 2009.
  * [12] H. Huh, A. M. Tulino, and G. Caire, “Network MIMO with linear zero-forcing beamforming: Large system analysis, impact of channel estimation, and reduced-complexity scheduling,” _IEEE Trans. Inf. Theory_ , vol. 58, no. 5, pp. 2911–2934, May 2012.
  * [13] E. Piovano and B. Clerckx, “Optimal DoF Region of the K-User MISO BC With Partial CSIT,” _IEEE Commun. Lett._ , vol. 21, no. 11, pp. 2368–2371, Nov 2017.
  * [14] A. G. Davoodi and S. A. Jafar, “Aligned Image Sets Under Channel Uncertainty: Settling Conjectures on the Collapse of Degrees of Freedom Under Finite Precision CSIT,” _IEEE Trans. Inf. Theory_ , vol. 62, no. 10, pp. 5603–5618, Oct 2016.
  * [15] M. Maddah-Ali and D. Tse, “Completely stale transmitter channel state information is still very useful,” _IEEE Trans. Inf. Theory_ , vol. 58, no. 7, pp. 4418–4431, Jul. 2012.
  * [16] J. Xu, J. G. Andrews, and S. A. Jafar, “MISO Broadcast Channels with delayed finite-rate feedback: Predict or observe?” _IEEE Trans. on Wireless Commun._ , vol. 11, no. 4, pp. 1456–1467, Apr. 2012.
  * [17] S. Yang, M. Kobayashi, D. Gesbert, and X. Yi, “Degrees of freedom of time correlated MISO Broadcast Channel with delayed CSIT,” _IEEE Trans. Inf. Theory_ , vol. 59, no. 1, pp. 315–328, Jan. 2013.
  * [18] A. Vahid, M. A. Maddah-Ali, and A. S. Avestimehr, “Approximate Capacity Region of the MISO Broadcast Channels With Delayed CSIT,” _IEEE Trans. on Commun._ , vol. 64, no. 7, pp. 2913–2924, July 2016.
  * [19] O. Simeone, O. Somekh, H. V. Poor, and S. Shamai (Shitz), “Downlink multicell processing with limited-backhaul capacity,” _EURASIP Journal on Advances in Signal Processing_ , May 2009.
  * [20] M. Jaber, M. A. Imran, R. Tafazolli, and A. Tukmanov, “5g backhaul challenges and emerging research directions: A survey,” _IEEE Access_ , vol. 4, pp. 1743–1766, 2016.
  * [21] P. Popovski, J. J. Nielsen, C. Stefanovic, E. d. Carvalho, E. Strom, K. F. Trillingsgaard, A. Bana, D. M. Kim, R. Kotaba, J. Park, and R. B. Sorensen, “Wireless access for ultra-reliable low-latency communication: Principles and building blocks,” _IEEE Network_ , vol. 32, no. 2, pp. 16–23, March 2018.
  * [22] M. Bennis, M. Debbah, and H. V. Poor, “Ultrareliable and low-latency wireless communication: Tail, risk, and scale,” _Proceedings of the IEEE_ , vol. 106, no. 10, pp. 1834–1853, Oct 2018.
  * [23] R. Radner, “Team decision problems,” _The Annals of Mathematical Statistics_ , 1962.
  * [24] S. Grandhi, R. Vijayan, and D. Goodman, “Distributed power control in cellular radio systems,” _IEEE Trans. on Commun._ , vol. 42, no. 234, pp. 226–228, Feb 1994.
  * [25] B. L. Ng, J. S. Evans, S. V. Hanly, and D. Aktas, “Distributed downlink beamforming with cooperative base stations,” _IEEE Trans. Inf. Theory_ , vol. 54, no. 12, pp. 5491–5499, 2008.
  * [26] A. G. Dimakis, S. Kar, J. M. F. Moura, M. G. Rabbat, and A. Scaglione, “Gossip algorithms for distributed signal processing,” _Proceedings of the IEEE_ , vol. 98, no. 11, pp. 1847–1864, Nov 2010.
  * [27] V. Aggarwal, Y. Liu, and A. Sabharwal, “Sum capacity of interference channels with a local view: Impact of distributed decisions,” _IEEE Trans. on Inf. Theory_ , vol. 58, no. 3, pp. 1630–1659, 2012.
  * [28] A. Vahid and R. Calderbank, “Two-User Erasure Interference Channels With Local Delayed CSIT,” _IEEE Trans. Inf. Theory_ , vol. 62, no. 9, pp. 4910–4923, Sept 2016.
  * [29] M. A. Maddah-Ali and U. Niesen, “Fundamental limits of Caching,” _IEEE Trans. Inf. Theory_ , vol. 60, no. 5, pp. 2856–2867, May 2014.
  * [30] J. Zhang and P. Elia, “Fundamental Limits of Cache-Aided Wireless BC: Interplay of Coded-Caching and CSIT Feedback,” _IEEE Trans. Inf. Theory_ , vol. 63, no. 5, pp. 3142–3160, May 2017.
  * [31] M. A. Hail, M. Amadeo, A. Molinaro, and S. Fischer, “Caching in named data networking for the wireless internet of things,” in _2015 International Conference on Recent Advances in Internet of Things (RIoT)_ , April 2015, pp. 1–6.
  * [32] S. Wang, X. Zhang, Y. Zhang, L. Wang, J. Yang, and W. Wang, “A survey on mobile edge networks: Convergence of computing, caching and communications,” _IEEE Access_ , vol. 5, pp. 6757–6779, 2017.
  * [33] P. de Kerret and D. Gesbert, “Degrees of freedom of the network MIMO channel with distributed CSI,” _IEEE Trans. Inf. Theory_ , vol. 58, no. 11, pp. 6806–6824, Nov. 2012.
  * [34] A. Bazco-Nogueras, P. de Kerret, D. Gesbert, and N. Gresset, “Distributed CSIT does not reduce the Generalized DoF of the 2-user MISO Broadcast Channel,” _IEEE Wireless Commun. Lett._ , vol. 8, no. 3, pp. 685–688, June 2019.
  * [35] ——, “On the Degrees-of-Freedom of the K-user Distributed Broadcast Channel,” 2020, accepted to IEEE Trans. Inf. Theory.
  * [36] P. de Kerret, A. Bazco, and D. Gesbert, “Enforcing coordination in network MIMO with unequal CSIT,” in _2016 50th Asilomar Conference on Signals, Systems and Computers_ , Nov 2016, pp. 39–43.
  * [37] S. Shamai and S. Verdú, “The impact of frequency-flat fading on the spectral efficiency of CDMA,” _IEEE Trans. Inf. Theory_ , vol. 47, no. 4, pp. 1302–1327, May 2001.
  * [38] J. Lee and N. Jindal, “High SNR analysis for MIMO Broadcast Channels: Dirty paper coding versus linear precoding,” _IEEE Trans. Inf. Theory_ , vol. 53, no. 12, pp. 4787 – 4792, Dec. 2007.
  * [39] N. Jindal, “MIMO Broadcast Channels with finite-rate feedback,” _IEEE Trans. Inf. Theory_ , vol. 52, no. 11, pp. 5045–5060, Nov. 2006.
  * [40] A. Bazco-Nogueras, L. Miretti, P. de Kerret, D. Gesbert, and N. Gresset, “Achieving vanishing rate loss in decentralized Network MIMO,” in _Proc. IEEE International Symposium on Information Theory (ISIT)_ , 2019, pp. 1457–1461.
  * [41] D. E. Knuth, “Big Omicron and Big Omega and Big Theta,” _SIGACT News_ , vol. 8, no. 2, pp. 18–24, Apr. 1976. [Online]. Available: http://doi.acm.org/10.1145/1008328.1008329
  * [42] G. Hardy, E. Wright, D. Heath-Brown, and J. Silverman, _An Introduction to the Theory of Numbers_ , ser. Oxford mathematics. Oxford University Press, 2008. [Online]. Available: https://books.google.fr/books?id=d3wpAQAAMAAJ
  * [43] A. G. Davoodi, B. Yuan, and S. A. Jafar, “GDoF Region of the MISO BC: Bridging the Gap Between Finite Precision and Perfect CSIT,” _IEEE Trans. Inf. Theory_ , vol. 64, no. 11, pp. 7208–7217, Nov 2018.
  * [44] A. Bazco Nogueras, “Fundamental limits and algorithms in decentralized and cooperative wireless networks,” Ph.D. dissertation, Sorbonne Université, 11 2019. [Online]. Available: http://www.eurecom.fr/publication/6121
  * [45] A. Wiesel, Y. C. Eldar, and S. Shamai (Shitz), “Zero-forcing precoding and generalized inverses,” _IEEE Trans. Signal Process._ , vol. 56, no. 9, pp. 4409–4418, 2008.
  * [46] C. T. K. Ng, D. Gunduz, A. J. Goldsmith, and E. Erkip, “Distortion minimization in Gaussian layered broadcast coding with successive Refinement,” _IEEE Trans. Inf. Theory_ , vol. 55, no. 11, pp. 5074–5086, 2009.
  * [47] O. E. Ayach and R. W. Heath, “Interference alignment with analog channel state feedback,” _IEEE Trans. Wireless Commun._ , vol. 11, no. 2, pp. 626–636, Feb. 2012.
  * [48] I. Atzeni and D. Gesbert, “Cooperative MIMO Precoding with Distributed CSI: A Hierarchical Approach,” in _2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC)_ , June 2018, pp. 1–5.
  * [49] H. V. Henderson and S. R. Searle, “On deriving the inverse of a sum of matrices,” _Siam Review_ , vol. 23, no. 1, pp. 53–60, Jan. 1981.


[1]Jeffery Painter [1]GlaxoSmithKline, Durham, NC, USA 2]Tech Mahindra, Plano,
TX, USA 3]GlaxoSmithKline, Upper Providence, PA, USA 4]GlaxoSmithKline,
Brentford, Middelsex, UK 5]London School of Hygiene and Tropical Medicine,
London, UK

# Bridging the Gap in Drug Safety Data Analysis: Large Language Models for SQL
Query Generation

<EMAIL_ADDRESS>Venkateswara Rao Chalamalasetti
<EMAIL_ADDRESS>Raymond Kassekert
<EMAIL_ADDRESS>Andrew Bate<EMAIL_ADDRESS>*  [  [  [  [

###### Abstract

Pharmacovigilance (PV) is essential for drug safety, primarily focusing on
adverse event monitoring. Traditionally, accessing safety data required
database expertise, limiting broader use. This paper introduces a novel
application of Large Language Models (LLMs) to democratize database access for
non-technical users. Utilizing OpenAI’s GPT-4, we developed a chatbot that
generates structured query language (SQL) queries from natural language,
bridging the gap between domain knowledge and technical requirements. The
proposed application aims for more inclusive and efficient data access,
enhancing decision making in drug safety. By providing LLMs with plain
language summaries of expert knowledge, our approach significantly improves
query accuracy over methods relying solely on database schemas. The
application of LLMs in this context not only optimizes PV data analysis,
ensuring timely and precise drug safety reporting—a crucial component in
adverse drug reaction monitoring—but also promotes safer pharmacological
practices and informed decision making across various data intensive fields.

###### keywords:

Pharmacovigilance, SQL Query Generation, Large Language Models (LLMs), Drug
Safety, Natural Language Processing (NLP)

##  1 Introduction

Drug safety, or pharmacovigilance (PV), encompasses the rigorous and ongoing
assessment of the safety of medications and vaccines given to individuals as
part of standard healthcare practices. It plays a crucial role in safeguarding
patient well being and in deepening our comprehension of any issues that arise
during the development and regular use of pharmaceuticals in healthcare
settings. PV is fundamental in identifying, assessing, and minimizing the
risks associated with the use of drugs, aiming to ensure that their benefits
significantly outweigh any risks.

Central to PV are extensive databases that compile individual case safety
reports (ICSRs), crucial for assessing drug safety and reporting to regulatory
bodies. Beninger elucidates PV’s role in risk identification and the
development of strategies to mitigate these risks, highlighting its intricate
and multidimensional nature [1].

However, navigating these databases to generate precise SQL queries is a
complex task, requiring specialized expertise due to their intricate
structure. This complexity, as noted by Brass and Goldberg, arises from the
mismatch between the graph-based nature of ontologies and the tabular
structure of relational databases [2], often leaving users struggling to
precisely articulate their search criteria. The prevalence of SQL
misapplication, especially in healthcare, underscores the pressing need for
enhanced query generation methods [3].

Query development in PV is further complicated by the highly regulated nature
of data, which imposes strict access controls and demands significant
computational resources for querying. Moreover, translating the requirements
of safety scientists into SQL involves not just technical skill, but also a
deep understanding of the scientific context, with potential for errors even
among experts [4].

Addressing these challenges, this study explores the application of Large
Language Models (LLMs), specifically OpenAI’s GPT-4[5], to facilitate database
querying through natural language. This novel approach leverages a retrieval-
augmented generation (RAG) framework, combined with a business context
document, to enhance the LLM’s ability to produce SQL queries that meet
specific business constraints, thus improving query accuracy and domain-
specific requirement compliance. Our methodology significantly advances PV
data analysis by improving the timeliness and precision of drug safety
reporting, a crucial component in adverse drug reaction surveillance.

LLMs, particularly those developed on the transformative Transformer model
introduced in 2017, have significantly influenced artificial intelligence and
natural language processing [6]. These models offer a promising solution to
the enduring challenges of database interaction in PV. By making complex
database queries accessible to a broader audience, our work not only
streamlines the PV data analysis process, but potentially facilitates wider
access to data across various domains.

##  2 Background

The process of developing queries within PV databases illustrates the
intricate blend of technical expertise and scientific acumen required.
Translating the nuanced requests of safety scientists into precise SQL queries
poses a notable challenge, with a considerable margin for error even among
seasoned professionals [4]. Historically, tools like Query-by-Example (QBE)
have offered simplified means for crafting queries [7], yet they fall short
when faced with complex or nuanced querying needs [8].

The rise and increasing recognition of LLMs in text-to-SQL generation have
significantly propelled their use in database management, notably enhancing
the generation of efficient and accurate SQL code [9, 10, 11]. Building upon
our previous exploration into utilizing LLMs for safety documentation
searches, which utilized a RAG framework alongside a Chat-GPT interface, we’ve
set a foundation for this current study [12]. In this research, we introduce
an innovative approach aimed at refining text-to-SQL conversions for PV
databases through the integration of a business context document. This
document distills complex database knowledge from PV systems experts into
accessible, plain language and, when combined with LLMs, markedly enhances the
models’ capacity to craft SQL queries that adhere closely to specific business
needs. This method addresses and surmounts the challenges associated with
relying solely on database metadata for query generation.

Recent assessments have cast a critical eye on the effectiveness of LLMs in
text-to-SQL tasks, noting performance gaps between proprietary models like
GPT-4 and open-source alternatives, especially in processing complex queries
and semantic understanding [10, 13, 14]. This observation underscores the
importance of supplementing LLMs with detailed contextual information to
enhance query precision within the PV sector.

Central to PV are extensive safety databases, crucial for regulatory
compliance and monitoring drug safety [15]. Relying on technical teams for
query formulation can introduce delays, underscoring the urgency for
innovative solutions that empower safety scientists with direct data access.
Our work builds on the advancements in LLM technology, providing an intuitive
interface that enables non-technical users to perform database queries,
potentially enhancing PV data analysis and reporting processes.

While LLMs enhanced by business context documents hold promise for improving
SQL query generation, the need for database experts remains crucial. They are
essential for monitoring system performance, addressing errors, and managing
non-systematic changes, such as updates to the database schema. In fields like
PV, where accuracy is paramount, these experts play a vital role in ensuring
the LLM’s outputs align with the database’s actual structure. Their
involvement is indispensable for adapting to and mitigating the impacts of
dynamic changes within the database environment.

In summary, our study leverages the progressive capabilities of LLMs,
presenting a novel approach to address the challenges in PV database querying.
By integrating LLM technology with a business context document, we aim to
narrow the divide between technical complexity and domain expertise,
democratizing access to crucial drug safety information.

##  3 Methods

Our study aims to transform natural language queries (NLQs) into syntactically
precise OracleTM SQL queries, leveraging LLMs within a RAG framework, enriched
by a detailed business context. Our study unfolded in three distinct phases to
assess the capability of an LLM in SQL query generation under varying
conditions of database knowledge provision.

###  3.1 Experimental Phases

This study was conducted in three phases, each designed to assess different
aspects of LLM performance in translating NLQs into SQL. The details of each
phase will be elaborated below. Phase 1 established baseline performance
metrics using an exhaustive database schema. Phase 2 evaluated the
effectiveness of a business context document, which provided plain language
summaries of the database structure. Phase 3 examined whether refining the
approach from Phase 1 by focusing only on essential tables could enhance or
match the effectiveness demonstrated in Phase 2, thereby assessing the added
value of using a business context document in the process.

Phase 1 aimed to assess the LLM’s performance when provided with exhaustive
background knowledge of the database schema. This was implemented using a
detailed 290-page PDF that included every table and column definition within
the database, complemented by database comments that explained the function
and significance of each column.

Phase 2 introduced a novel approach by utilizing a plain language summary of
the database structure, rather than the comprehensive 290-page PDF used in
Phase 1. This business context document, prepared by a database expert, not
only detailed the structural elements like tables, columns, and fields but
also provided contextual insights into their relevance within the operational
framework of the PV database system.

Phase 3 further honed the LLM’s input by strategically narrowing down the
original 290-page document used in Phase 1 to include only the tables that
were identified as essential in the business context document. This refinement
aimed at eliminating potential confusion caused by irrelevant or conflicting
schema information from the comprehensive database documentation. This focused
approach was designed to enhance the LLM’s ability to concentrate on the most
pertinent data, thereby improving the effectiveness of query generation. The
resultant streamlined document, reduced to a concise 29-page PDF, contained
only 36 table definitions and represented about 10% of the original Phase 1
document’s size. It included all necessary data elements to comprehensively
answer test test set of NLQs. This phase underscored the effectiveness of
using a curated subset of the database schema to optimize query accuracy and
relevancy. Additionally, by comparing the performance of this refined subset
with the results obtained using the business context document from Phase 2, we
aimed to evaluate whether a more focused schema could perform comparably or
better in generating precise SQL queries. Preliminary results indicated that
while the refined Phase 3 approach did reduce confusion and streamline query
processing, the business context document still provided superior context and
guidance, leading to more accurate SQL generation to be addressed in the
results below.

###  3.2 NLQ Selection

Before the experimental investigation commenced, we curated 60 NLQs from the
historical user request logs, ensuring a broad representation of query
complexity for a thorough assessment of the LLM’s performance. This set of
queries, maintained consistently across all experimental phases, spans from
simple inquiries to more complex data retrieval tasks.

To facilitate a uniform testing environment for each phase, a structured
prompt was introduced to guide the LLM towards generating syntactically
accurate SQL queries while minimizing the generation of unfounded responses.
This approach aimed to standardize the query generation process, allowing for
a fair comparison of the LLM’s performance under different conditions of
database knowledge provision. The prompt provided to the chatbot was as
follows:

“You are an Oracle SQL expert. Given a question, generate a syntactically
correct Oracle SQL query. Avoid querying non-existent columns and pay close
attention to column-table associations. For keywords in the WHERE clause,
ensure case-insensitive data comparison, for example, ‘upper(STATE_NAME) =
upper(’deleted’)‘. If you are unable to generate the SQL query, please state
that you cannot create the query without additional information or context, do
not attempt to make anything up.”

A vector-based retrieval strategy was consistently applied across all phases,
leveraging embeddings from the text-embedding-ada-002
model111https://platform.openai.com/docs/guides/embeddings/embedding-models.
The database knowledge provided to the LLM varied across the phases: Phase 1
involved a comprehensive database schema, Phase 2 utilized a concise business
context document, and Phase 3 incorporated a refined schema focusing on the
essential tables identified in Phase 2. Each phase employed a character text
splitter (chunk size: 1,000 characters; overlap: 200 characters; separator:
‘\n‘) to facilitate the processing of input text through the GPT-4 model, as
shown in Figure 1.

Figure 1: LLM chatbot User Interface

This structured approach ensured that each phase of the experiment was
initiated with a clear, consistent framework, allowing us to evaluate the
LLM’s SQL generation capabilities across a variety of informational contexts
and database knowledge levels. The distinctions between each phase are
elaborated upon in the sections that follow.

###  3.3 Phase 1 Design Details

Phase 1 of our study, outlined in the process flowchart in Figure 2, initiated
with the extraction of the Oracle database schema. This extraction included
details such as tables, columns, and their descriptive comments, which were
compiled into a context document. Subsequently, this document was converted
into a PDF to act as a comprehensive reference material. To prepare for the
experiment’s execution phase, the context document underwent a process to
create vector embeddings from the database schema definitions.

Figure 2: Phase 1 Experimental Design

For the experiment, each NLQ was processed by the chatbot, supplemented with
our specifically crafted prompt to guide the LLM’s response generation. The
outcomes were then cataloged for a detailed evaluation by our database expert.

The SQL queries generated during this phase were subject to a thorough review
by SQL administrators, who assessed their precision and relevance. Based on
this evaluation, each query’s outcome was categorized as Passed, Partial, or
Fail, reflecting the degree of accuracy in addressing the intended query
requirements.

###  3.4 Phase 2 Design Details

Phase 2 closely paralleled the first, with key modifications illustrated in
Figure 3. Notably, this phase integrated a newly created business context
document, a resource developed by a database expert. This document, absent
before this experiment, encapsulated the expert’s understanding of the
database’s structure and its relevance to user queries, honed through years of
experience. Following an initial draft, the document was iteratively refined,
informed by a review of the NLQs designated for LLM processing, as delineated
in Steps 1-3 of the accompanying diagram. After its completion, the business
context document was processed in a manner analogous to the database schema
PDF utilized in Phase 1, with the chatbot subsequently evaluating each NLQ and
the resultant SQL outputs undergoing expert review and classification (Passed,
Failed, or Partial Pass) as detailed in Steps 4-7.

Figure 3: Phase 2 Experimental Design

A distinctive feature of this phase, introduced in Step 8, was the provision
for iterative refinement of the business context document. This step was
activated when the LLM failed to generate accurate SQL for a given NLQ,
leveraging feedback from the chatbot to progressively enhance the document’s
accuracy and comprehensiveness. This iterative loop, executed approximately 10
times, was instrumental in evolving the business context document, ultimately
culminating in a refined version poised for presentation in our final results.
The iterative enhancement of the business context document emerged as a
pivotal component, with each iteration targeting the bridging of knowledge
gaps identified through benchmark query assessments.

###  3.5 Phase 3 Design Details

Phase 3 closely mirrored the setup of Phase 1, with the primary distinction
being the utilization of a significantly condensed set of tables for the
database schema export, as delineated in the initial steps of the Phase 1
methodology. This adjustment aimed to refine the LLM’s focus and potentially
enhance its query generation accuracy by limiting the scope to only the tables
outlined within the business context document. Beyond this strategic reduction
in schema complexity, the approach to processing NLQs and the subsequent
evaluation of the generated SQL code remained consistent. Each query’s outcome
was documented and analyzed by a database expert, ensuring a consistent
assessment of the LLM’s performance under these modified conditions.

###  3.6 Classification of Generated SQL Code

Each NLQ processed through our chatbot was evaluated by a database expert who
categorized the resultant SQL code into one of three classes: pass, fail, or
partial pass. A “pass” indicated that the LLM-generated SQL code was valid and
executable by the Oracle SQL interpreter without modifications, and it
produced the anticipated results akin to those of a manually crafted query. A
“fail” was noted when the LLM explicitly stated its inability to generate a
query given the context, produced unexecutable SQL code, or generated
syntactically correct SQL that did not yield the expected results.

A “partial pass” was assigned based on the expert’s judgment that the SQL code
needed only minor modifications to become valid or to address minor issues,
such as incorrect table names used in scenarios where tables had similarly
named columns, leading to ambiguity.

This classification approach, particularly the identification of specific
errors such as misidentified table names, led us to the development of the
Phase 3 part of this study . This phase aimed to assess whether focusing the
LLM’s scope exclusively on the tables essential for our test set of NLQs would
enhance its performance, compared to the broader database schema used in Phase
1.

###  3.7 Constructing the Business Context Document

As noted above, the creation of the business context document was initiated
during phase 2 of our experiment. This document requires capturing the domain
specialist knowledge in unstructured text form. While there is not yet a
benchmark of the best approach to accomplishing this, for our experiment we
focused on the tables most frequently required to answer a scientific question
of the global safety database.

This process helps to extricate the necessary tables and business logic that
must be described to create a generalizable LLM based database expert capable
of answering any new queries.

An example of the contents of our business context document is highlighted
below.

This process was repeated for 36 database tables which encompass primary
features of a case safety report, along with supporting tables related to code
sets and dictionaries and system configuration tables necessary for supporting
a wide range of example queries for testing.

In addition, specific instructions and definitional statements known to the PV
systems experts were added to the business context document such as:

Definition “New case”: A new case includes cases that have not yet been
assigned to any database user (e.g., Intake Specialist or Data Entry
Specialist) as well as cases that have been assigned, but not yet accepted by
the user.

The development of the business context document is an iterative process, and
as such, the level of effort spent on the creation of this document, we
believe, will yield more positive results from the LLMs capability in
effectively converting NLQs into valid and executable SQL code. The iterative
process helps to highlight knowledge gaps, prompting successive refinements.
This process can be repeated until a set of benchmark queries when prompted to
the LLM attain satisfactory performance levels. It is noted that this is
potentially a limiting factor in deploying a similar solution for other
databases, as it requires one with expert knowledge of the system to help
develop such a document. Updates to database structures or schemas in the
future could also affect the utility of such a system, and should be
considered as to their impact in a production system for reliability.

Excerpt of the Safety Database Context Document Purpose: The purpose of this
context document is to equip the Large Language Model (LLM) with the knowledge
and intricacies of the global Safety Database. This will enable accurate and
efficient SQL generation for data retrieval and analysis.
Database Overview: The global Safety Database is a comprehensive
pharmacovigilance software solution that stores and manages data related to
drug safety and adverse event reporting.
Introduction: This document provides guidance on generating SQL related to
business configurations in the Global Safety database system. It emphasizes
the tables, relationships, and common querying patterns associated with these
configurations.
Data Tables and Relationships: The global safety database contains the
following tables: 1. Table name PRODUCT_FAMILY contains company product data,
where the column identified as FAMILY_ID is the primary key, the column
identified as NAME contains the product family name, and the column identified
as DELETED contains the date a record was deleted.
If the DELETED column is NULL, then the record is not deleted.
The column PRODUCT_GROUP_ID is a foreign key to link records matching on the
same column from the table PRODUCT_GROUP.

###  3.8 System Design and Implementation

LLMs demonstrate proficiency in answering queries, but they face challenges
when required to extract and interpret information from specific contextual or
business documents. Priming, or training, LLMs with document-specific
knowledge can significantly improve their ability to provide informed and
contextually relevant responses. Research by Brown et al. [16] and Xie et al.
[17] highlights LLMs’ adaptability through fine-tuning processes [18],
underscoring their versatility across a range of applications.

To leverage this adaptability, we implemented a framework using the RAG model,
which employs vectorization techniques to enhance LLMs’ capacity for accurate,
data-validated responses [19] [20]. The RAG model enriches LLMs by integrating
vector-based data representations, augmenting their understanding and response
generation [21]. While Text-to-SQL generation is a niche area within RAG
applications [22], our focus is on generating contextually informed SQL
statements.

Our system utilizes OpenAI’s GPT-4, hosted on the Azure
cloud222https://azure.microsoft.com/ platform in a secure, sandboxed
environment, and is designed to seamlessly integrate with the LLM APIs and
supporting libraries such as LangChain333https://www.langchain.com/. This
ensures the generated SQL queries from natural language inputs are not only
accurate and efficient but also contextually relevant, facilitating a bridge
between complex database systems and users lacking SQL expertise.

Figure 4: System Architectural Diagram

Central to our framework is the business context document, visualized in the
bottom left of Figure 4 and highlighted in yellow. It is segmented and
processed through the text-embedding-ada-002 model to produce vector
embeddings. These vectors enable the GPT-4 model to access a database of
contextually pertinent information, enhancing response accuracy.

User queries undergo a similar processing routine (top right of the figure),
supporting a similarity search against the vector database to guide SQL
generation. When a user submits a natural language query (NLQ), a similar
embedding process is applied to convert the query into vector embeddings,
which are used to search for relevant data chunks in the vector database. The
search results, in conjunction with the user query, are then forwarded to the
GPT-4 model, which auto generates the SQL query. Following this, the generated
SQL queries undergo thorough expert review and validation before being
systematically cataloged in the “SQL Index” for future reference.

By employing advanced LLMs and the RAG framework, our system aims to improve
the efficiency of data retrieval, making database interactions more
straightforward and accessible for non-technical users within the PV domain.

##  4 Results

Our experiment sought to understand the impact of leveraging LLMs with a
business context document on the complexity of SQL queries generated from
NLQs. We developed a complexity scoring algorithm to provide an objective
measure of SQL query complexity. This method, while straightforward, offers a
reproducible way to evaluate SQL complexity, acknowledging that complexity
assessment can vary widely in methodology [23].

The scoring algorithm outlined in Algorithm 1 assigns points based on several
criteria: time estimated to manually create the query, the number of tables
involved, the presence of joins, group by clauses, order by clauses, and
aggregation functions. This scoring system aims to quantify the inherent
complexity of each SQL query in an unbiased manner.

Algorithm 1 Compute the SQL Complexity Score

1:$\text{score}\leftarrow 0$

2:$\text{score}\leftarrow time\\_to\\_create$

3:$\text{score}\leftarrow\text{score}+number\\_of\\_tables$

4:$\text{score}\leftarrow\text{score}+number\\_of\\_joins$

5:$\text{score}\leftarrow\text{score}+number\\_of\\_where\\_clauses$

6:if $\text{has\\_group\\_by}=\text{True}$ then

7: $\text{score}\leftarrow\text{score}+1$

8:end if

9:if $\text{has\\_order}=\text{True}$ then

10: $\text{score}\leftarrow\text{score}+1$

11:end if

12:if $\text{has\\_aggregation}=\text{True}$ then

13: $\text{score}\leftarrow\text{score}+1$

14:end if

15:return $score$

Following the application of our scoring algorithm to the SQL queries
generated for each NLQ, we performed a distribution analysis of the resulting
complexity scores. Figure 5 illustrates this distribution. We employed a
systematic approach to categorize the complexity of SQL queries into “low”,
“medium”, or “high”. This categorization was based on their scoring
percentile: “low” for scores beneath the 25th percentile, “medium” for scores
within the interquartile range (IQR), and “high” for scores above the 75th
percentile. This method provides a clear, objective means to classify SQL
query complexity, incorporating both the subjective expertise of a database
professional regarding time to manually craft the query and objective measures
like the count of tables required.

Figure 5: Boxplot of SQL Complexity Scores

Incorporating the complexity scoring algorithm, our analysis of the 60 NLQs
resulted in the following classifications: 17 queries were deemed “low”
complexity, 31 were categorized as “medium”, and 12 fell into the “high”
complexity category.

As detailed in the methods section, we categorized the LLM’s SQL code
generation into three outcomes: ’pass’ for accurate SQL production, ’partial
success’ for codes requiring slight modifications, and ’fail’ for incorrect or
absent responses. Initially, with the database schema alone, the LLM showed an
8.3% pass rate444“Pass rate” refers to the proportion of natural language
queries (NLQs) that were successfully translated into SQL queries by the LLM,
as manually classified by a database expert. The expert tested each generated
SQL query directly within the database system. If the output was equivalent to
that of a manually crafted SQL query, the query was classified as “pass.” The
pass rate thus represents the percentage of NLQs that resulted in accurately
generated SQL queries by the LLM. Similarly, the “partial pass” and “fail”
rates are computed using the same method, indicating the proportion of NLQs
that resulted in SQL queries needing minor adjustments or failing to produce
correct results, respectively., with 78.3% failures, demonstrating a
significant challenge in generating queries without additional context (Table
1).

Table 1: LLM Performance with DB Schema Only | NLQ Complexity |  |
---|---|---|---
Result | Low | Medium | High | Total | Percent
Pass | 3 | 2 | 0 | 5 | (8.3%)
Fail | 11 | 13 | 23 | 47 | (78.3%)
Partial Pass | 3 | 2 | 3 | 8 | (13.3%)
Total | 17 | 17 | 24 | 60 |
|  |  |  |  |

Integrating the business context document markedly improved the LLM’s
performance, achieving a pass rate above 78%, as shown in Table 2. This
underscores the significant role of the business context in enhancing query
accuracy for complex inquiries.

Table 2: LLM Performance with Business Context Document | NLQ Complexity |  |
---|---|---|---
Result | Low | Medium | High | Total | Percent
Pass | 16 | 16 | 15 | 47 | (78.3%)
Fail | 0 | 0 | 5 | 5 | (8.3%)
Partial Pass | 1 | 1 | 6 | 8 | (13.3%)
Total | 17 | 17 | 26 | 60 |
|  |  |  |  |

Statistical analysis via Fisher’s Exact Test highlighted a substantial
improvement, with a shift from a p-value of 0.2163 without the business
context to 0.0186 with it, indicating a statistically significant performance
boost when queries are enriched with contextual information.

The enhancements were evident across all query complexities, suggesting the
effectiveness of contextual priming in optimizing LLM SQL query generation.
Despite some remaining challenges with complex queries, these findings
strongly support further development and refinement of business context
documents for intuitive database interactions and broader LLM applications in
data-centric fields.

Phase 3 showed slight improvements over Phase 1 by limiting the LLM’s focus to
only essential tables, reducing the failure rate from 78% to 50% and slightly
increasing the pass rate. However, a significant number of queries still
required additional adjustments, as seen in Table 3.

Table 3: Phase 3 LLM Performance with Narrowed Schema Definition | NLQ Complexity |  |
---|---|---|---
Result | Low | Medium | High | Total | Percent
Pass | 4 | 2 | 0 | 6 | (10%)
Fail | 6 | 8 | 16 | 30 | (50%)
Partial Pass | 7 | 7 | 10 | 24 | (40%)
Total | 17 | 17 | 26 | 60 |
|  |  |  |  |

A Fisher’s Exact Test in Phase 3 yielded a p-value of 0.1063, suggesting that
narrowing the database schema to essential tables modestly improved outcomes
but did not match the success achieved with the business context document.
This result emphasizes that while schema optimization can yield some benefits,
the comprehensive insights provided by the business context document are more
critical in enhancing LLM’s query generation accuracy.

These findings underscore the importance of developing detailed business
context documents for LLM frameworks, not only to enhance SQL query accuracy
but also to extend the application of LLMs in complex, data-rich environments.

##  5 Discussion

This study underscores the potential of context-enriched LLMs in facilitating
SQL query generation from NLQs within PV and other data-intensive domains. By
integrating OpenAI’s GPT-4 model with a business context document, we have
enhanced the model’s ability to generate syntactically precise and
contextually relevant SQL queries. This approach demonstrates a promising path
towards democratizing access to complex databases and improving the process’s
intuitiveness and efficiency for data querying.

Our findings reveal a marked improvement in SQL query generation accuracy when
the LLM is augmented with contextual knowledge. The introduction of a business
context document led to a notable increase in performance, achieving a success
rate exceeding 78% for a broad spectrum of query complexities. This stark
enhancement, from an initial 8.3% success rate with the database schema alone
to over 78% with the business context document, underscores the critical role
of context in bridging the gap between natural language and SQL code. Despite
some improvements in phase 3 with a narrowed database schema, the
comprehensive context provided by the business document proved indispensable
for achieving higher accuracy.

While our research marks a significant stride in making SQL querying more
accessible, thereby enabling non-technical users to harness data-driven
insights for more inclusive and efficient decision-making, it’s essential to
interpret our findings with caution. Our approach, characterized by its
adaptability and ease of use, undeniably democratizes data access, especially
in fields where the complexity of data and the diversity of users present
formidable challenges. However, the efficacy of the business context document
has been assessed with a relatively small set of NLQs and within the confines
of a single, albeit intricate, enterprise database. The scalability and
applicability of our findings to other databases remain uncertain. The
creation and integration of a business context document are critical factors
that could influence the reproducibility and effectiveness of our method in
varied database environments. Future research should aim to validate these
results across broader datasets and diverse database architectures to fully
understand the potential and limitations of our approach.

However, further advancements are necessary. While our system demonstrates
substantial capabilities, it also identifies key areas for additional research
and development. Handling complex queries and resolving ambiguous user intents
continue to pose challenges, indicating the necessity for continued efforts to
refine the model’s understanding and adaptability. Moreover, concerns related
to scalability and the implementation of such systems within large, dynamic
enterprise environments highlight the need for future investigations to
strengthen the robustness and applicability of our methodology.

In conclusion, this study offers a novel paradigm for employing LLMs in
database management, showcasing significant advancements in making PV data
analysis more intuitive and accessible. The integration of LLMs with a
business context document not only streamlines the SQL query generation
process but also opens the door to broader applications in various sectors. As
we continue to harness the burgeoning capabilities of LLMs, the horizon for
data interaction and analysis widens, promising a future where insights from
complex databases are within reach for a more extensive array of users,
thereby enhancing our collective ability to make informed decisions in an
increasingly data-driven world.

##  6 Acknowledgments

This research was supported in part by additional team members who helped with
testing and development of the architectural framework to support context
based LLMs, including Marcin Karwowski (GSK, Poland), Christie Roshan (GSK,
UK) and Richard Barlow (GSK, USA), all members of the PV Systems team in
Global Safety.

##  7 Declarations

Data availability The datasets generated during and/or analyzed during the
current study are available from the corresponding author upon reasonable
request.

Competing interests The authors have no competing interests to declare that
are relevant to the content of this article.

Conflict of interests This manuscript has not been submitted to, nor is under
review at, another journal or other publishing venue.

Funding GlaxoSmithKline Biologicals SA covered all costs associated with the
conduct of the study and the development of the manuscript and the decision to
publish the manuscript. JP, RK and AB are employees, and VC is a contractor,
of the GSK group of companies. JP, RK and AB hold shares and/or options in the
GSK group of companies.

## References

  *   * Beninger [2018] Beninger, P.: Pharmacovigilance: an overview. Clinical therapeutics 40(12), 1991–2004 (2018)
  * Brass and Goldberg [2006] Brass, S., Goldberg, C.: Semantic errors in SQL queries: A quite complete list. Journal of Systems and Software 79(5), 630–644 (2006)
  * Sivarajkumar et al. [2024] Sivarajkumar, S., Mohammad, H.A., Oniani, D., Roberts, K., Hersh, W., Liu, H., He, D., Visweswaran, S., Wang, Y.: Clinical information retrieval: A literature review. Journal of Healthcare Informatics Research, 1–40 (2024)
  * Taipalus et al. [2018] Taipalus, T., Siponen, M., Vartiainen, T.: Errors and complications in SQL query formulation. ACM Transactions on Computing Education (TOCE) 18(3), 1–29 (2018)
  * OpenAI [2024] OpenAI: GPT-4 Technical Report (2024). https://arxiv.org/abs/2303.08774
  * Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)
  * Zloof [1977] Zloof, M.M.: Query-by-example: A data base language. IBM systems Journal 16(4), 324–343 (1977)
  * Ramakrishnan and Gehrke [2002] Ramakrishnan, R., Gehrke, J.: Database Management Systems. McGraw-Hill, Inc., New York, NY (2002)
  * Dauphin et al. [2017] Dauphin, Y.N., Fan, A., Auli, M., Grangier, D.: Language modeling with gated convolutional networks. In: International Conference on Machine Learning, pp. 933–941 (2017). PMLR
  * Li et al. [2023] Li, J., Hui, B., Qu, G., Li, B., Yang, J., Li, B., Wang, B., Qin, B., Cao, R., Geng, R., et al.: Can LLM Already Serve as A Database Interface? A Big Bench for Large-Scale Database Grounded Text-to-SQLs. arXiv preprint arXiv:2305.03111 (2023)
  * Gao et al. [2023] Gao, D., Wang, H., Li, Y., Sun, X., Qian, Y., Ding, B., Zhou, J.: Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation. arXiv preprint arXiv:2308.15363 (2023)
  * Painter et al. [2023] Painter, J.L., Mahaux, O., Vanini, M., Kara, V., Roshan, C., Karwowski, M., Chalamalasetti, V.R., Bate, A.: Enhancing Drug Safety Documentation Search Capabilities with Large Language Models: A User-Centric Approach. 2023 International Conference on Computational Science and Computational Intelligence (CSCI) (2023). CSCI
  * Roberson et al. [2024] Roberson, R., Kaki, G., Trivedi, A.: Analyzing the Effectiveness of Large Language Models on Text-to-SQL Synthesis. arXiv preprint arXiv:2401.12379 (2024)
  * Zhang et al. [2024] Zhang, Q., Dong, J., Chen, H., Li, W., Huang, F., Huang, X.: Structure Guided Large Language Model for SQL Generation. arXiv preprint arXiv:2402.13284 (2024)
  * Bhangale et al. [2017] Bhangale, R., Vaity, S., Kulkarni, N.: A day in the life of a pharmacovigilance case processor. Perspectives in Clinical Research 8(4), 192 (2017)
  * Brown et al. [2020] Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems 33, 1877–1901 (2020)
  * Xie et al. [2021] Xie, S.M., Raghunathan, A., Liang, P., Ma, T.: An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXiv:2111.02080 (2021)
  * Radford et al. [2019] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)
  * Lewis et al. [2020] Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., et al.: Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems 33, 9459–9474 (2020)
  * Topsakal and Akinci [2023] Topsakal, O., Akinci, T.C.: Creating large language model applications utilizing langchain: A primer on developing LLM apps fast. In: International Conference on Applied Engineering and Natural Sciences, vol. 1, pp. 1050–1056 (2023)
  * IBM [2023] IBM: What is retrieval-augmented generation? https://research.ibm.com/blog/retrieval-augmented-generation-RAG. [Online; accessed 01-Apr-2024] (2023)
  * Zhao et al. [2024] Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Cui, B.: Retrieval-Augmented Generation for AI-Generated Content: A Survey. arXiv preprint arXiv:2402.19473 (2024)
  * Simon and Pataki [2010] Simon, M., Pataki, N.: SQL Code Complexity Analysis. In: Proceedings of the 8th International Conference of Applied Informatics (2010)