#  $n$-Best Kernel Approximation in Reproducing Kernel Hilbert Spaces

Tao Qian

###### Abstract.

By making a seminal use of the maximum modulus principle of holomorphic
functions we prove existence of $n$-best kernel approximation for a wide class
of reproducing kernel Hilbert spaces of holomorphic functions in the unit
disc, and for the corresponding class of Bochner type spaces of stochastic
processes. This study thus generalizes the classical result of $n$-best
rational approximation for the Hardy space and a recent result of $n$-best
kernel approximation for the weighted Bergman spaces of the unit disc. The
type of approximations have significant applications to signal and image
processing and system identification, as well as to numerical solutions of the
classical and the stochastic type integral and differential equations.

Email<EMAIL_ADDRESS>ORCID: 0000-0002-8780-9958
Macau University of Science and Technology, (Macau Center for Mathematical
Sciences), Macau, China
Funded by The Science and Technology Development Fund, Macau SAR (File no.
0123/2018/A3)

Keywords: Weighted Bergman space, Weighted Hardy space, Maximum Modulus
Principle, $n$-best rational approximation, $n$-best kernel approximation,
Bochner space today

MSC 2020: 41A50; 46E22; 30H10; 30H20

##  1\. Introduction

A main form of application of mathematical analysis is approximation by basic
functions of the underlying space. Various forms and topics of polynomial and
rational approximations have been studied, including convergence models,
capacity and rates, existence and uniqueness of best approximation, as well as
algorithms, etc. See for instance the selected list of the literature [63, 19,
14, 54, 12, 30, 56, 28, 10, 11, 9, 18, 67, 8, 58, 55] and the references
therein. The present study will concentrate in approximation of reproducing
kernel Hilbert spaces (RKHSs) of complex holomorphic functions, the latter
being related to $Z$-transforms of system transfer functions. In RKHSs the
most natural basic functions are the parameterized reproducing kernels. Tasks
of signal and image processing are based on effective reconstruction of a
given signal or image. To measure reconstruction efficiency, among the most
commonly used, there are two dual models. One is, for a previously given
$\epsilon$, to determine the smallest integer $n$ such that the difference,
measured in the underlying space norm, between the given function and some
$n$-linear combination of the basic functions is already dominated by
$\epsilon.$ The second model is, for a given resource limitation represented
by a natural integer $n,$ to find an $n$-tuple of parameterized basic
functions and an $n$-tuple of coefficients such that the $n$-linear
combination that they compose gives rise to the best possible approximation to
the given function. The second model is abbreviated as $n$-_best
approximation_ or more briefly $n$-_best problem_. The present paper restricts
to study the second model, and only the existence part of the $n$-best
solutions. The corresponding algorithm part for actually finding one or all
$n$-best solutions, as a consequence of the technical results of the existence
proof, will be separately studied.

We will be based on the general concept reproducing kernel Hilbert space
(RKHS). In 1930, to study the partial differential equation

$\frac{\partial^{2}u}{\partial x^{2}}+\frac{\partial^{2}u}{\partial
y^{2}}+\alpha(x,y)\frac{\partial u}{\partial x}+\beta(x,y)\frac{\partial
u}{\partial y}+\gamma(x,y)u=0,$

where $\alpha(x,y),\beta(x,y),\gamma(x,y)\in C^{2}(\Omega),\Omega$ is a
bounded region, $\alpha(x,y),\beta(x,y),\gamma(x,y)\in C^{2}(\Omega),\Omega$
are all real-analytic functions, S. Bergman proposed the reproducing kernel
concept and gave the related formulas. The theoretical study of reproducing
kernel may be divided into two stages. Of which the first started from J.
Mercer ([36]), at the beginning of the 20th century, who in his studies of
integral equations brought up the concept positive definite kernel:

$\sum_{i,j=1}^{n}K(y_{i},y_{j})\xi_{i}\xi_{j}\geq 0.$

The second stage is the development by E. H. Moore ([37]), around 1930’s, who
proved that every positive Hermitian matrix induces a Hilbert space that has a
kernel function $K(x,y)$ enjoying the property

$f(y)=\langle f,K(\cdot,y)\rangle.$

The same phenomenon was observed by S. Bochner in the convolution kernel form
connected to Fourier theory ([7]). Around 1940’s the most popular reproducing
kernels were the Bergman type ones. Bergman developed the idea of S. Zaremba
([70]) to solve boundary value problems by using reproducing kernels, showing
that reproducing kernels are effective tools to solve elliptic boundary value
problems.

Combined with various science and engineering objects there have developed new
theories and algorithms, including signal processing ([3]), system
identification ([4, 39, 40, 48]), scholastics processing ([25, 44, 51, 66]),
estimation theory ([22, 42]), wavelet transform ([57]), reproducing kernel
particle method ([26, 24, 23, 27, 17, 61]), the moving least-square
reproducing kernel method ([32, 33]), multi-scale reproducing kernel particle
methods ([31]), etc., with ample applications.

F. M. Larkin ([38]) and M. M. Chawla ([13]) studied the approximation aspect.
Formats of interpolation with reproducing kernel have been used to numerical
solutions of partial differential equations and integral equations. The latest
reproducing kernel approximation methodology, called adaptive Fourier
decomposition (AFD), uses the maximal energy extraction principle similar to
what is in greedy algorithm or matching pursuit ([41, 59]). Greedy algorithms
are based on general Hilbert space theory with a dictionary ([60]). The AFD
methods, originated from analytic positive frequency decomposition, validate
attainability of the best suited parameters. Technically, AFDs are based on
delicate mathematical analysis, and, in particular, allow repeating selection
of parameters through defining multiple kernels, when necessary. The technical
treatment is a blend of functional analysis, complex analysis and harmonic
analytic. Recent studies on approximation in Hardy spaces, including the
latest $n$-best and stochastic AFD approximations, may be found in a sequence
of articles [45, 43, 1, 2, 15, 16, 47, 62, 44, 51]. Some early studies of
adaptive Fourier decomposition in Bergman and weighted Bergman spaces are
given in [49, 50]. Celebrating results for the $n$-best type approximation in
weighted Bergman spaces are presented in [52].

Recent studies given by Ball et al.’s papers ([5, 6]) show that approximations
of Hilbert spaces of holomorphic functions have intimate connections with
system identification and in particular with time-variant linear systems. In
the Hardy space case there holds the Sz.-Nagy-Foias model theory for $C_{\cdot
0}$ contraction operators. The model theory combined with the Burling-Lax
theorem addresses a correspondence between any two of the four kinds of
objects: shift invariant subspaces, operator-valued inner functions,
conservative discrete-time input/state/output linear system, and $C_{\cdot 0}$
Hilbert-space contraction operators. The studies of [5] and [6] extend such
correspondence to weighted Bergman and weighted Hardy spaces. Under such frame
work and via the $Z$-transform of the system, $n$-best approximation in each
of the mentioned spaces determines the optimal shift invariant subspaces for
effective and efficient system identification.

To complete the introduction for science and engineering motivations and
involvements, we at last, but not least, mention that there have been
forerunner but recent developments on the stochastic $n$-best model as cited
in [44, 51, 66]. Stochastic $n$-best approximation of our general setting will
be given in §5. Stochastic AFD offers a new approach to stochastic processes,
including solutions of stochastic partial differential equations. It, in
particular, stands as an alternative method to the Karhunen-Loéve
decomposition together with several advantages. Further developments along
this direction are to be reported in separate and forthcoming papers.

Next, we introduce the related preliminary knowledge and raise the $n$-best
problem in our general setting. The Hardy space is defined

$\mathbb{H}^{2}({\bf D})=\\{f:{\bf D}\to{\bf{C}}\ :\ f\ {\rm is\ analytic\
in}\ {\bf D}\ {\rm and}\ \|f\|_{\mathbb{H}^{2}}=\sup_{0\leq
r<1}\int_{0}^{2\pi}|f(r{\rm e}^{it})|^{2}<\infty\\}.$

As a fundamental result, functions in the Hardy space have non-tangential
boundary limits a.e. on the unit circle $\partial{\bf D}$. The Hardy space is
isometric to the function space ${\mathbb{H}^{2}}_{\partial{\bf D}}$
consisting of the non-tangential limiting functions. One of the alternative
definitions of the space ${\mathbb{H}^{2}}_{\partial{\bf D}}$ is

${\mathbb{H}^{2}}_{\partial{\bf D}}=\\{f:\partial{\bf D}\to{\bf{C}}\ :\ f\in
L^{2}(\partial{\bf D}),\quad f({\rm e}^{it})=\sum_{k=0}^{\infty}c_{k}{\rm
e}^{ikt},\quad\sum_{k=0}^{\infty}|c_{k}|^{2}<\infty\\}.$

${\mathbb{H}^{2}}_{\partial{\bf D}}$ is a closed subspace of the Hilbert space
$L^{2}(\partial{\bf D})$ equipped with the inner product

(1.1) $\displaystyle\langle f,g\rangle=\frac{1}{2\pi}\int_{0}^{2\pi}f({\rm
e}^{it})\overline{g({\rm e}^{it})}dt.$

For a given positive integer $n,$ an ordered pair of polynomials $(p,q)$ is
called an $n$-_admissible pair_ if $p$ and $q$ are co-prime, $q\neq 0$ in
${\bf D},$ and both the degrees of $p$ and $q$ do not exceed $n.$ The famous
$n$-_best rational approximation problem_ in $\mathbb{H}^{2}({\bf D})$ is as
follows: For $f\in\mathbb{H}^{2}({\bf D}),$ find an $n$-admissible ordered
pair $(\tilde{p},\tilde{q})$ such that

(1.2) $\displaystyle\|f-\frac{\tilde{p}}{\tilde{q}}\|_{\mathbb{H}^{2}({\bf
D})}=\inf\\{\|f-\frac{p}{q}\|_{\mathbb{H}^{2}({\bf D})}\ :\ (p,q)\ {\rm is\
an}\ n{\mbox{-}admissible\ pair}\\}.$

Closely related to rational approximation there exist studies on what is
called Takenaka-Malmquist (TM) system, or rational orthogonal system:

(1.3)
$\displaystyle\\{B_{a_{1}a_{2}...a_{n}}(z)\\}_{n=1}^{\infty}=\left\\{\frac{\sqrt{1-|a_{n}|^{2}}}{1-\bar{a}_{n}z}\prod_{k=1}^{n-1}\frac{z-a_{k}}{1-\bar{a}_{k}z}\right\\}_{n=1}^{\infty},\quad
a_{1},\cdots,a_{n},\cdots,\in{\bf D},$

where $B_{a_{1}a_{2}...a_{n}}(z)=e_{a_{n}}(z)\phi_{a_{1}a_{2}...a_{n-1}}(z),$
where $e_{a_{n}}$ is the normalized Szegö kernel at $a_{n},$

$e_{a_{n}}(z)=\frac{\sqrt{1-|a_{n}|^{2}}}{1-\overline{a}_{n}z},$

and the _canonical Blaschke product_ with zeros $a_{1},\cdots,a_{n-1},$

(1.4)
$\displaystyle\phi_{a_{1}a_{2}...a_{n-1}}(z)=\prod_{k=1}^{n-1}\frac{z-a_{k}}{1-\bar{a}_{k}z}.$

There further holds the relation, for $a\in\bf D,$

(1.5) $\displaystyle
e_{a}(z)=\frac{\sqrt{1-|a|^{2}}}{1-\overline{a}z}=\frac{k_{a}}{\|k_{a}\|},$

where

$k_{a}(z)=\frac{{1}}{1-\overline{a}z}\quad{\rm
and}\quad\|k_{a}\|=\frac{1}{\sqrt{1-|a|^{2}}}$

are, respectively, the reproducing kernel of $\mathbb{H}^{2}({\bf D})$ and the
normalizing constant making $\|e_{a}\|=1.$

The above formulated $n$-best rational approximation problem (1.2) is, in
essence, equivalent to the following $n$-_best Blaschke form approximation_
problem ([46, 47]): Let $n$ be a given positive integer. If $f$ itself is not
an $m$-Blaschke form for some $m<n,$ find a set of $n$ parameters
$a_{1},\cdots,a_{n},$ all in ${\bf D},$ such that

$\displaystyle\|f-\sum_{k=1}^{n}\langle f,B_{a_{1}\cdots a_{k}}\rangle
B_{a_{1}\cdots a_{k}}\|$ $\displaystyle=$
$\displaystyle\inf\\{\|f-\sum_{k=1}^{n}\langle f,B_{b_{1}\cdots b_{k}}\rangle
B_{b_{1}\cdots b_{k}}\|\ :\ b_{1},\cdots,b_{n}\ \in{\bf D}\\}.$

The formulation of the problem allows multiplicity of the zeros $a_{k}$’s. A
solution of (1) will be referred as an $``n$-best Blaschke form
approximation”([47, 46]). Regardless unimodular constants (see Lemma 3.2
below), an $n$-TM system is the Gram-Schmidt orthogonalization of a set of $n$
Szegö kernels, or multiple kernels (formulated in (2.11) below) when the
parameters are with multiplicities. See explanations in §2). The $n$-best
Blaschke form approximation is, again, equivalent with the $n$-_best kernel
approximation_ formulated as: Find $(a_{1},\cdots,a_{n})\in{\bf D}^{n}$ and
$(c_{1},\cdots,c_{n})\in{\bf{C}}^{n}$ such that

(1.6)
$\displaystyle\|f-\sum_{j=1}^{n}c_{j}\tilde{k}_{a_{j}}\|=\inf\\{\|f-\sum_{j=1}^{n}c^{\prime}_{j}{k}_{a^{\prime}_{j}}\|:a^{\prime}_{1},\cdots,a^{\prime}_{n}\
\in{\bf D}$ $\displaystyle{\rm are\ distinct},{\rm
and}c^{\prime}_{1},\cdots,c^{\prime}_{n}\ \in\bf{C}\\},$

where $\tilde{k}_{a_{j}}$ are multiple kernels (see (2.11)).

Considerable amount of studies have been devoted to the above problem with the
three equivalent forms. See [63, 64, 11, 9, 47, 46, 39, 52]. Amongst,
researchers have obtained several new proofs for existence of a solution to
(1.2). The motivation of exploring new proofs of the existence, including that
of the author himself’s, would be at least two-folder: (i) The known existence
proofs for the Hardy space case do not seem to be adaptable to prove existence
of an $n$-best approximation in any non-Hardy spaces, including weighted
Bergman spaces and weighted Hardy spaces. In fact, before the work [52]
whether there is a solution to the problem in any non-Hardy space was unknown;
and (ii) On top of the existence, an ultimate algorithm of finding even one
$n$-best solution of (1.2) has yet to be sought: The commonly adopted
empirical algorithms are all local that cannot theoretically avoid the
possibility of sinking into a local minimum. See [9, 11, 45, 43, 49, 53] and
the references therein. As an extension of the traditional Fourier method,
both the n-best and the repeated one-by-one types have been found to have
effective applications in signal and image processing and system
identification ([48, 35, 39]).

The question for $n$-best kernel approximation can be raised in general RKHSs,
or even in Hilbert spaces with a dictionary. The recently published new proof
of existence of $n$-best approximation in the Hardy space case ([65]) uses the
maximum modulus principle of holomorphic functions as a crucial technical
trick. Inspired by this complex analysis method, through proving some
necessary new pointwise estimations of the kernels of the involved zero- and
Blaschke-weighted spaces, the study [52] was able to prove existence of the
$n$-best kernel approximation of all the weighted Bergman spaces
$\mathbb{A}^{2}_{\alpha}({\bf D}),-1<\alpha<\infty.$

The study in the present paper is directly motivated by the recent new proofs
of the existence on the Hardy space ([62]) and one on the weighted Bergman
spaces ([52]). We achieve a clever and concise proof for the existence of the
$n$-best kernel approximation for a large class of reproducing kernel Hilbert
spaces, that is in particular strictly larger than that of the Bergman ones,
enclosing all the weighted Hardy spaces.

Based on further analysis of the orthogonalization projection operator
$Q_{a_{1}a_{2}...a_{k}}$ and factorization of higher order generalized
backwardd shift operators $Q_{a_{1}a_{2}...a_{k}}/\phi_{a_{1}\cdots{a_{m-1}}}$
(see §3), the present paper is able to avoid use of the pointwise estimations
of the reproducing kernels of the involved zero spaces and the Blaschke
weighted spaces. For RKHSs more general than the Bergman ones such kernel
estimations may be impossible. As a result, we are able to assert existence of
the $n$-best problem for a class of RKHSs more general than the weighted Hardy
spaces. Precisely, we can declare existence of solutions of the $n$-best
kernel approximation for all RKHSs of holomorphic functions in $\bf D$ that
satisfy the following three conditions (see Theorem 2.1):

(i) The reproducing kernel $K(z,w)$ enjoys _the analyticity condition_ : When
$w\in\bf D$ is fixed, $K(z,w)$ is analytic for $z$ in a neighbourhood of the
closed unit disc $\overline{\bf D},$ and, when $z\in\bf D$ is fixed, $K(z,w)$
is anti-analytic for $w$ in a neighbourhood of $\overline{\bf D};$

(ii) The kernel $K(z,w)$ satisfies _the infinite-norm-property at the
boundary_ , that is,

(1.7) $\displaystyle\lim_{w\to\partial\bf D}\|K_{w}\|=\infty;$

and,

(iii) $K(z,w)$ satisfies _the uniformly boundedness condition_

(1.8) $\displaystyle\frac{|K_{w}(z)|}{\|K_{w}\|^{2}}\leq C_{\mathcal{H}},\quad
w,z\in\bf D,$

where $C_{\mathcal{H}}$ is a constant depending on the space.

There is an ordered sub-family, $\mathbb{H}_{W_{\beta}}({\bf
D}),-\infty<\beta<\infty,$ called the Hardy-Sobolev spaces, within the family
of weighted Hardy spaces ([5, 6], also see §4). The index range $\beta<0$
corresponds to the weighted Bergman spaces including the standard Bergman
space case for $\beta=-1$ whose $n$-best existence results are proved in [52].
The $\beta=0$ case corresponds to the Hardy space [65]. The $n$-best existence
results for the Hardy-Sobolev spaces for $0<\beta\leq 1$ ($\beta=1$
corresponds to the Dirichlet space) are obtained as a consequence of the main
result of this paper (see §4). The Hardy-Sobolev spaces for $\beta>1$ do not
fall into the category of the RKHSs considered in the main theorem of this
paper, but we show that they are governed by the Sobolev Embedding Theorem
(also see [50]).

The writing plan of the paper is as follows. In §2 we discuss in detail the
Gram-Schmidt orthogonalization of reproducing kernels that induces the concept
multiple kernels. It is in terms of the multiple kernel concept that the
$n$-best problem is precisely formulated. In §3 our main result, Theorem 2.1,
on existence of the $n$-best approximation of a wide class of RKHSs is proved
through a number of technical lemmas in relation to orthogonal projections and
analysis of the involved reproducing kernels. In §4, as examples of using the
main result Theorem 2.1, we give re-proofs of the Hardy and the weighted
Bergman space results of [65] and [52], and to prove, using the unified
method, the $n$-best existence result for the range $-\infty<\beta\leq 1$ of
the Hardy-Sobolev spaces $\mathbb{H}_{W_{\beta}}({\bf D}),$ in which the
results for the range $(-\infty,1]$ are known to be equivalent to the weighted
Bergman and the Hardy space cases. The results for the range $\beta\in(0,1]$
are new as applications of Theorem 2.1. We include a remark in §4 concerning
the range $\beta\in(1,\infty)$ through invoking the Sobolev Embedding theorem
that provides complete understanding to the $n$-best issue for the whole range
$-\infty<\beta<\infty.$ In §5 we extend Theorem 2.1 to the stochastic signal
case based on the Bochner type Hilbert space setting. For the existing related
studies in the stochastic signal direction we refer to [44] and [51]. To the
end of §5 we include a remark for the impact of this study on obtaining
algorithms in finding the $n$-best solutions.

##  2\. Main Theorem

Let $\mathcal{H}$ be a RKHS of holomorphic functions in $\bf D$ with
reproducing kernel $K_{w},w\in\bf D,$ satisfying the conditions (i),(ii),(iii)
set in §1. Throughout the paper $n$ is a fixed positive integer. Let
$Z_{k}=(a_{1},\cdots,a_{k}),1\leq k\leq n,$ be an ordered $k$-tuple of complex
numbers in $\bf D$ allowing multiplicity.

Denote by $l(a_{k})$ the multiple of $a_{k}$ in the $k$-tuple
$(a_{1},\cdots,a_{k}),k\leq n.$ Denote

(2.9)
$\displaystyle\tilde{K}_{a_{k}}(z)=\left[\left(\frac{d}{d\overline{w}}\right)^{l(a_{k})-1}K_{w}(z)\right]_{w=a_{k}}.$

We will call $\tilde{K}_{a_{k}}$ the multiple reproducing kernel corresponding
to $(a_{1},\cdots,a_{k})$. It is easy to show, for $f$ being in the
holomorphic function space, there holds

(2.10) $\displaystyle\langle
f,\tilde{K}_{a_{k}}\rangle=f^{(l(a_{k})-1)}(a_{k}).$

The consecutive derivatives of the kernel function correspond to repeating use
of kernel parameters.

In the Hardy space case, for instance, $k_{a}(z)=\frac{1}{1-\overline{a}z},$
and

(2.11)
$\displaystyle\tilde{k}_{a_{k}}(z)=\left[\left(\frac{d}{d\overline{w}}\right)^{l(a_{k})-1}k_{w}(z)\right]_{w=a_{k}}=\frac{l!\overline{a}^{l}}{(1-\overline{a}z)^{l+1}}.$

In general cases, let $(a_{1},\cdots,a_{n})$ be any $n$-tuple of complex
numbers in $\bf D.$ Denote by $(E_{a_{1}},\cdots,E_{a_{1}\cdots a_{m}})$ the
Grand-Schmidt orthonormalization of
$(\tilde{K}_{a_{1}},\cdots,\tilde{K}_{a_{m}}),\ m=1,\cdots,n,$ given by

(2.12) $\displaystyle E_{a_{1}\cdots
a_{m}}(z)=\frac{\tilde{K}_{a_{m}}(z)-\sum_{l=1}^{m-1}\langle\tilde{K}_{a_{m}},E_{a_{1}\cdots
a_{l}}\rangle E_{a_{1}\cdots
a_{l}}(z)}{\sqrt{\|\tilde{K}_{a_{m}}\|^{2}-\sum_{l=1}^{m-1}|\langle\tilde{K}_{a_{m}},E_{a_{1}\cdots
a_{l}}\rangle|^{2}}}.$

We will denote the orthogonal projection of $f$ into the linear subspace $X$
by $P_{X}(f).$ The projection into the orthogonal complement of $X$ is denoted
$Q_{X}=I-P_{X}.$ In particular, denote by $P_{a_{1}\cdots a_{m}}$ the
orthogonal projection from $\mathcal{H}$ to ${\rm
span}\\{\tilde{K}_{a_{1}},\cdots,\tilde{K}_{a_{m}}\\},$ and by $Q_{a_{1}\cdots
a_{m}}=I-P_{a_{1}\cdots a_{m}},$ the projection into the orthogonal complement
subspace of ${\rm span}\\{\tilde{K}_{a_{1}},\cdots,\tilde{K}_{a_{m}}\\}.$ It
is recognized that $Q_{a_{1}a_{2}...a_{k}}$ corresponds to the Gram-Schmidt
process, precisely,

(2.13) $\displaystyle
E_{a_{1}a_{2}...a_{k}}=\frac{Q_{a_{1}a_{2}...a_{k-1}}(\tilde{K}_{a_{k}})}{\|Q_{a_{1}a_{2}...a_{k-1}}(\tilde{K}_{a_{k}})\|}.$

We have been using the notation $\\{B_{a_{1}a_{2},...a_{k}}\\}_{k=1}^{\infty}$
for the TM system in the Hardy space case. Now we use
$\\{E_{a_{1}a_{2}...a_{k}}\\}_{k=1}^{\infty}$ for the Gram-Schmidt
orthonormalization of the multiple reproducing kernels
$\\{\tilde{K}_{a_{k}}\\}_{k=1}^{\infty}$ in $\mathcal{H}$ given by (2.12) and
(2.13). In the classical Hardy space case they are essentially the same, as,
in fact, $E_{a_{1}a_{2}...a_{k}}=c_{k}B_{a_{1}a_{2}...a_{k}},$ where $c_{k}$
are unimodular constants, $k=1,\cdots,$ (see Lemma 3.2 and the relevant
references). None of the $E_{a_{1}a_{2}...a_{k}}$ of any holomorphic Hilbert
spaces other than the Hardy space seem to have such nice construction: the
orthonormalization of the Szegö kernel $k_{a_{k}}$ with respect to the span of
$\\{B_{a_{1}a_{2}...a_{j-1}}\\}_{j=1}^{k-1},$ which is
$B_{a_{1}a_{2}...a_{k}},$ is just the product of the added normalized Szegö
kernel $e_{a_{k}}$ with the the canonical Blaschke
$\phi_{a_{1}a_{2}...a_{k-1}}.$ This extraordinary property, together with the
complex unimodular property of Blaschke products on the circle, as well as the
equivalent norm property restricted to the circle, offer decisive conveniences
in developing the Hardy space theory in contrast with that of the non-Hardy
space cases. The $n$-best kernel approximation problem in the general context
is formulated as follows. Let $f\in\mathcal{H}.$ Whether there exist, and if
yes, how to find computationally $a_{1},\cdots,a_{n},$ all in $\bf D,$ such
that

(2.14) $\displaystyle d_{f}=d_{f}(n)\triangleq\|f-P_{a_{1}\cdots
a_{n}}f\|=\inf\\{\|f-P_{b_{1}\cdots b_{n}}f\|\ :\ (b_{1}\cdots b_{n})\in{\bf
D}^{n}\\}?$

We will prove the following

######  Theorem 2.1.

Suppose that $\mathcal{H}$ is a RKHS of holomorphic functions satisfying (i),
(ii), and (iii), and $n$ is a positive integer. Then there must hold one of
the following two cases: (1) $f$ is a linear combination of
$\tilde{K}_{b_{1}},\cdots,\tilde{K}_{b_{m_{1}}}$ for some $m_{1}$-tuple
$(b_{1},\cdots,b_{m_{1}})\in{\bf D}^{m_{1}},m_{1}\leq n;$ or (2) there exists
an $n$-tuple $(a_{1},\cdots,a_{n})\in{\bf D}^{n}$ such that (2.14) holds for a
positive infimum $d_{f}>0,$ that is,

(2.15) $\displaystyle d_{f}$ $\displaystyle=$
$\displaystyle\|f-\sum_{l=1}^{n}\langle f,E_{a_{1}\cdots a_{l}}\rangle
E_{a_{1}\cdots a_{l}}\|$ $\displaystyle=$
$\displaystyle\inf\\{\|f-\sum_{l=1}^{n}\langle f,E_{b_{1}\cdots b_{l}}\rangle
E_{b_{1}\cdots b_{l}}\|\ :\ (b_{1},\cdots,b_{n})\in{\bf D}^{n}\\}.$

We note that $(a_{1},\cdots,a_{n})\in{\bf D}^{n}$ gives rise to equality
(2.15) if and only if

(2.16) $\displaystyle\sum_{l=1}^{n}|\langle f,E_{a_{1}\cdots
a_{l}}\rangle|^{2}=\sup\\{\sum_{l=1}^{n}|\langle f,E_{b_{1}\cdots
b_{l}}\rangle|^{2},(b_{1},\cdots,b_{n})\in{\bf D}^{n}\\}.$

##  3\. Proof of the Theorem

We will use the notation $f_{a_{1}a_{2}...a_{k}}=Q_{a_{1}a_{2}...a_{k}}f,$
where $a_{j},j=1,\cdots,k,$ are allowed to repeat.

######  Lemma 3.1.

If $a_{j}$ is among $a_{1},\cdots,a_{k},$ then
$f_{a_{1}a_{2}...a_{k}}(a_{j})=0,$ including the multiplicity.

Proof. The proof is straightforward if $a_{1},\cdots,a_{k}$ are all different,
that is $l(a_{j})=1,j=1,\cdots,k.$ In the case, owing to the self-adjoint
property of the projection operators and the orthogonality gained from G-S
process, for $a_{j}$ being among $a_{1},\cdots,a_{k},$

$\displaystyle f_{a_{1}a_{2}...a_{k}}(a_{j})$ $\displaystyle=$
$\displaystyle\langle Q_{a_{1}a_{2}...a_{k}}f,K_{a_{j}}\rangle$
$\displaystyle=$ $\displaystyle\langle
f,Q_{a_{1}a_{2}...a_{k}}K_{a_{j}}\rangle$ $\displaystyle=$
$\displaystyle\langle f,(I-P_{a_{1}a_{2}...a_{k}})K_{a_{j}}\rangle$
$\displaystyle=$ $\displaystyle\langle f,0\rangle$ $\displaystyle=$
$\displaystyle 0.$

Let $a_{j}$ have multiplicity $l(a_{j})>1,$ and
$a_{s_{1}}=a_{s_{2}}=\cdots=a_{s_{l(a_{j})}}=a_{j}.$ For
$m=1,\cdots,l(a_{j}),$ in view of (2.10),

$\displaystyle\left(\frac{d}{dz}\right)^{m-1}[f_{a_{1}a_{2}...a_{k}}](a_{j})$
$\displaystyle=$
$\displaystyle\langle\left(\frac{d}{dz}\right)^{m-1}Q_{a_{1}a_{2}...a_{k}}f,K_{a_{j}}\rangle$
$\displaystyle=$ $\displaystyle\langle
f,Q_{a_{1}a_{2}...a_{k}}\left(\frac{d}{d\overline{w}}\right)^{m-1}K_{w}(z)|_{w=a_{j}}\rangle$
$\displaystyle=$ $\displaystyle\langle
f,(I-P_{a_{1}a_{2}...a_{k}})\tilde{K}_{a_{j}}\rangle$ $\displaystyle=$
$\displaystyle 0.$

So, $f_{a_{1}a_{2}...a_{k}}$ has $l(a_{j})$-multiple zero at $a_{j}.$

$\hfill\square$

The new crucial concepts of this methodology include higher order generalized
backward shift operator

$\frac{Q_{a_{1}\cdots{a_{m-1}}}(f)(z)}{\phi_{a_{1}\cdots{a_{m-1}}}(z)}$

and its factorization (see Lemma 3.2 below). The order-1 generalized backward
shift operator $Q_{a}/[\frac{z-a}{1-\overline{a}z}]$ applied to a test
function $f$ gives rise to what we call reduced remainder ([45]) playing a
central role in the formulation of AFD.

######  Lemma 3.2.

Let $a_{1},\cdots,a_{m}$ be complex numbers in $\bf D$ allowing multiplicity.
Then

(1) In the general $\mathcal{H}$ space setting the Gram-Schmidt
orthonormalization

(3.17) $\displaystyle E_{a_{1}\cdots
a_{m}}(z)=\frac{Q_{a_{1}\cdots{a_{m-1}}}({K}_{a_{m}})(z)}{\|Q_{a_{1}\cdots{a_{m-1}}}({K}_{a_{m}})\|}$
$\displaystyle=$
$\displaystyle\frac{{\tilde{K}}_{a_{m}}(z)-\sum_{l=1}^{m-1}\langle{\tilde{K}}_{a_{m}},E_{a_{1}\cdots
a_{l}}\rangle E_{a_{1}\cdots
a_{l}}(z)}{\|{\tilde{K}}_{a_{m}}-\sum_{l=1}^{m-1}\langle{\tilde{K}}_{a_{m}},E_{a_{1}\cdots
a_{l}}\rangle E_{a_{1}\cdots a_{l}}\|}.$

(2) In the Hardy space case, the last function (3.17) is equal to ${\rm
e}^{ic}e_{a_{m}}(z)\phi_{a_{1}\cdots a_{m-1}}(z)={\rm e}^{ic}B_{a_{1}\cdots
a_{m}}(z),$ where

${\rm
e}^{ic}=\frac{\overline{\phi}_{a_{1}\cdots{a_{m-1}}}(a_{m})}{|\overline{\phi}_{a_{1}\cdots{a_{m-1}}}(a_{m})|},$

and $e_{a_{m}},\phi_{a_{1}\cdots{a_{m-1}}}$ and $B_{a_{1}\cdots a_{m}}$ are
respectively defined in (1.5), (1.4) and (1.3). Moreover,

(3.18) $\displaystyle\langle f,E_{a_{1}\cdots a_{m}}\rangle E_{a_{1}\cdots
a_{m}}=\langle f,B_{a_{1}\cdots a_{m}}\rangle B_{a_{1}\cdots a_{m}},$

and

(3.19) $\displaystyle\langle f,E_{a_{1}\cdots
a_{m}}\rangle=\frac{Q_{a_{1}\cdots{a_{m-1}}}(f)(a_{m})}{\phi_{a_{1}\cdots{a_{m-1}}}(a_{m})}\sqrt{1-|a_{m}|^{2}}.$

(3) In the general $\mathcal{H}$ space setting, the higher order generalized
shift operators (the reduced remainders) may be factorized, that is

(3.20)
$\displaystyle\frac{Q_{a_{1}\cdots{a_{m-1}}}(f)(z)}{\phi_{a_{1}\cdots{a_{m-1}}}(z)}=\left(\frac{Q_{a_{m-1}}}{\phi_{a_{m-1}}}\circ\cdots\circ\frac{Q_{a_{1}}}{\phi_{a_{1}}}\right)(f)(z).$

Proof. For proofs of (1) and (2) we refer to [44]. We now prove (3). For
$k>1,$ denote by $g_{k}$ the _ $k$-reduced remainder_ ([45])

$g_{k+1}(z)=\frac{g_{k}(z)-\langle g_{k},E_{a_{k}}\rangle
E_{a_{k}}(z)}{\phi_{a_{k}}(z)}=\left(\frac{Q_{a_{k}}}{\phi_{a_{k}}}\right)(g_{k})(z),$

where $g_{1}=f.$ Inductively there holds

$g_{k+1}(z)=\left(\frac{Q_{a_{k}}}{\phi_{a_{k}}}\circ\frac{Q_{a_{k-1}}}{\phi_{a_{k-1}}}\right)(g_{k-1})(z)=\left(\frac{Q_{a_{k}}}{\phi_{a_{k}}}\circ\frac{Q_{a_{k-1}}}{\phi_{a_{k-1}}}\circ\cdots\frac{Q_{a_{1}}}{\phi_{a_{1}}}\right)(f)(z).$

On the other hand, the AFD formulation given in [45] implies

$Q_{a_{1}\cdots a_{k}}f=g_{k+1}\prod_{j=1}^{k}\phi_{a_{j}}.$

We thus have

$g_{k+1}=\frac{Q_{a_{1}\cdots
a_{k}}f}{\prod_{j=1}^{k}\phi_{a_{j}}}=\left(\frac{Q_{a_{k}}}{\phi_{a_{k}}}\circ\frac{Q_{a_{k-1}}}{\phi_{a_{k-1}}}\circ\cdots\frac{Q_{a_{1}}}{\phi_{a_{1}}}\right)(f).$

$\hfill\square$

######  Remark 3.3.

It is recognized that the operator $\frac{Q_{a}}{\phi_{a}}$ is the generalized
backward shift operator defined in [45]. Repeating use of the operator yields
the reduced remainders $\frac{Q_{a_{1}\cdots
a_{k}}f}{\prod_{j=1}^{k}\phi_{a_{j}}}.$ The following lemma shows that the
reduced remainders of a function bounded by $M$ are still bounded with
explicit bounds in terms of $M$ and the parameters $a_{1},\cdots,a_{k}$
involved. The result plays a crucial role in the proof of the main result of
the paper.

######  Lemma 3.4.

Let $f$ be an analytic function in an open neighbourhood of $\overline{\bf D}$
and

$|f(z)|\leq M$

for some $M>0$ on $\overline{\bf D}.$ Then for any sequence
$a_{1},a_{2},...,a_{k}\in\bf D$ the reduced remainder functions

$\frac{f_{a_{1}a_{2}...a_{k}}(z)}{\phi_{a_{1}a_{2}...a_{k}}(z)}$

are analytic in an open neighbourhood of $\overline{\bf D}$ with the bounds
over $\overline{\bf D}:$

(3.21)
$\displaystyle\left|\frac{f_{a_{1}a_{2}...a_{k}}(z)}{\phi_{a_{1}a_{2}...a_{k}}(z)}\right|\leq
M(1+C_{\mathcal{H}})^{k},\quad z\in\overline{\bf D},$

where $C_{\mathcal{H}}$ is the constant in (1.8).

Proof. For $a_{1}\in\bf D$ and $z\in\partial\bf D,$

$\displaystyle|f_{a_{1}}(z)|$ $\displaystyle=$ $\displaystyle|f(z)-\langle
f,E_{a_{1}}\rangle E_{a_{1}}(z)|$ $\displaystyle\leq$
$\displaystyle|f(z)|+|f(a_{1})|\frac{|K_{a_{1}}(z)|}{\|K_{a_{1}}\|^{2}}$
$\displaystyle\leq$ $\displaystyle|f(z)|+|f(a_{1})|C_{\mathcal{H}}$
$\displaystyle\leq$ $\displaystyle M(1+C_{\mathcal{H}}).$

Since the zero of $\phi_{a_{1}}$ is a zero of $Q_{a_{1}}f,$
$\frac{Q_{a_{1}}f}{\phi_{a_{1}}}$ is a holomorphic function in an open
neighbourhood of $\overline{\bf D}.$ The maximum modulus principle over
$\overline{\bf D}$ gives

$\left|\frac{Q_{a_{1}}f(w)}{\phi_{a_{1}}(w)}\right|\leq\max\\{|f_{a_{1}}(z)|\
:\ z\in{\partial\bf D}\\}\leq M(1+C_{\mathcal{H}})$

for all $w\in\overline{\bf D}.$ By invoking the result of (iii), Lemma 3.2,
and repeating $k$ times the above estimation for
$\frac{Q_{a_{1}}f}{\phi_{a_{1}}},$ we obtain the bounds claimed by the lemma.
$\hfill\square$

We first prove Theorem 2.1 for $n=1.$ The goal of $1$-best approximation
amounts to finding $a_{1}\in\bf D$ such that

$|\langle f,E_{a_{1}}\rangle|=\frac{|f(a_{1})|}{\|K_{a_{1}}\|}$

attains its global maximum over all possible choices of the parameter in $\bf
D.$ This is what we called Maximal Selection Principle (MSP) in the previous
related studies ([45, 49, 50]). The proof is divided into two steps:

Step 1: We show that Boundary Vanishing Condition (BVC) holds in the case,
that is,

(3.22) $\displaystyle\lim_{|a|\to 1-}|\langle f,E_{a}\rangle|=0.$

BVC is proved here by using a density argument: We note that the given
function $f\in\mathcal{H}$ may be approximated within an error $\epsilon>0$ by
a bounded holomorphic function $g$ as a linear combination of some
parameterized reproducing kernels. Therefore, by the Cauchy-Schwarz
inequality, we have

$|\langle f,E_{a}\rangle|\leq|\langle f-g,E_{a}\rangle|+|\langle
g,E_{a}\rangle|\leq\|f-g\|+\frac{|g(a)|}{\|K_{a}\|}\leq\epsilon+\frac{|g(a)|}{\|K_{a}\|}.$

As a consequence of (1.7) and boundedness of $g$, the BVC (3.22) is concluded.

Step 2: If $f$ is not identical with the zero function there exists $b\in\bf
D$ such that $|\langle f,E_{b}\rangle|>0.$ Denote $|\langle
f,E_{b}\rangle|=\delta.$ The argument in Step 1 amounts that there exists
$0<r_{1}<1$ such that $|a|>r_{1}$ implies $|\langle f,E_{a}\rangle|<\delta/2.$
Therefore,

(3.23) $\displaystyle\max\\{|\langle f,E_{a}\rangle|\ :\ |a|\leq
r_{1}\\}=\sup\\{|\langle f,E_{a}\rangle|\ :\ a\in\bf D\\}.$

By invoking the theorem of global maximum attainability of a continuous
function on a compact set, (3.23) shows that the global maximum of $|\langle
f,E_{a}\rangle|$ is attainable inside $\bf D$.

Now we turn directly to the general $n>1$ case. To begin with the proof we
assume that $f$ itself is not expressible as a linear combination of $m_{1}$
reproducing kernels for $m_{1}<n.$ Based on the definition of supreme, one can
find a sequence of $n$-tuples with mutually distinct and non-zero components
(these can always be done owing to continuity of the inner product, and can at
least simplify the notation), $(a_{1}^{(l)},\cdots,a_{n}^{(l)}),l=1,2,\cdots,$
that corresponds to a sequence of $n$-tuples of reproducing kernels
$(K_{a_{1}^{(l)}},\cdots,K_{a_{n}^{(l)}}),$ such that the norms of the
projections $P_{a_{1}^{(l)}\cdots a_{n}^{(l)}}(f)$ tends to the supreme
(2.16). Since $(a_{1}^{(l)},\cdots,a_{n}^{(l)})\in\overline{\bf D}^{n},$
through a Bolzano-Weierstrass compact argument, we may assume, without loss of
generality, that the sequence of the $n$-tuples
$(a_{1}^{(l)},\cdots,a_{n}^{(l)})$ itself converges to
$(a_{1},\cdots,a_{n})\in\overline{\bf D}^{n}.$ If we have $a_{1},\cdots,a_{n}$
all in ${\bf D},$ then we are done due to continuity of the inner product,
although may involve multiple kernels when multiplicities occur. This gives
rise to the case (1) of the Theorem 2.1 for $m_{1}=n$ when $d_{f}=0;$ and the
case (2) when $d_{f}>0.$

Now we show that if not all the limiting points $a_{1},\cdots,a_{n}$ are
located within ${\bf D},$ then we have the case (1) for some $m_{1}<n,$ being
contrary with our priori assumption. Assume that at least one of
$a_{1},\cdots,a_{n}$ are on the boundary $\partial\bf D.$ Since the
projections $P_{a_{1}^{(l)}\cdots a_{n}^{(l)}}(f)$ and $Q_{a_{1}^{(l)}\cdots
a_{n}^{(l)}}(f)$ are irrelevant with the order, by re-ordering, when
necessary, we may assume without loss of generality that
$a_{1},\cdots,a_{m_{1}}$ are in $\bf D,$ and $a_{m_{1}+1},\cdots,a_{n}$ are on
$\partial\bf D,$ where $m_{1}<n,$ and in particular,
$\lim_{l\to\infty}|a_{n}^{(l)}|=1.$ In the case we will show

(3.24) $\displaystyle\lim_{l\to\infty}|\langle
f,E_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n}}\rangle|=0,$

regardless the locations of $a^{(l)}_{k},k=1,\cdots,n-1$ and $l=1,2,\cdots$.
If (3.24) can be proved, by repeating the same argument $n-m_{1}$ times we
result in that the latter $n-m_{1}$ terms of the $l$-sequence of the
$n$-tuples all have no contribution. We claim that $d_{f}>0$ cannot hold. If
it were $d_{f}>0,$ then it could be further reduced, contrary with $d_{f}$
being infimum. But, $d_{f}$ cannot be zero either, for in such case we got
that $f$ is a linear combination of $m_{1}<n$ multiple reproducing kernels,
contrary with our assumption. Thus, all that remain to be proved is (3.24). By
using the same density argument as we prove the case $n=1$ we may assume that
$f$ itself is an analytic function in a neighbourhood of the closed unit disc
$\overline{\bf D}$ with a bound $M.$

For any zero set $Z$ possibly with multiplicities we use the general notation
$K_{Z}(z,a)$ for the reproducing kernel at $a$ of the zero space
$\mathcal{H}_{Z},$ where

$\mathcal{H}_{Z}=\\{f\in\mathcal{H}\ :\ f\ {\rm vanishes\ at\ points\ in}\ Z\
{\rm including\ multiplicities}\\}.$

The space $\mathcal{H}_{Z}$ uses the same inner product as $\mathcal{H}.$ We
denote by $\mathcal{H}_{\phi_{Z}}$ the Hilbert space

$\mathcal{H}_{\phi_{Z}}=\\{f:{\bf D}\to{\bf{C}}\ :\ f\ {\rm is\ analytic},\
\|f\phi_{Z}\|_{\mathcal{H}}<\infty\\},$

where $\phi_{Z}$ is the canonical Blaschke product generated by the elements
of $Z$ including multiplicities. The inner product of $\mathcal{H}_{\phi_{Z}}$
is denoted as $\langle\cdot,\cdot\rangle_{\mathcal{H}_{\phi_{Z}}}.$ The
reproducing kernel of $\mathcal{H}_{\phi_{Z}}$ is denoted $K_{{\phi_{Z}}}.$ In
this paper we only need to treat zero sets $Z$ with finite points. Note that
$\mathcal{H}\subset\mathcal{H}_{\phi_{Z}},$ and
$\|f\|_{\mathcal{H}_{\phi_{Z}}}\leq\|f\|_{\mathcal{H}}.$

The next two lemmas follow similar idea in [29, 68, 69, 20, 52].

######  Lemma 3.5.

For any finite zero set $Z,$ by denoting $K_{Z}(z,w)$ the reproducing kernel
of the zero space $\mathcal{H}_{Z},$ there holds

(3.25) $\displaystyle
K_{Z}(z,w)=\phi_{Z}(z)K_{\phi_{Z}}(z,w)\overline{\phi_{Z}(w)}$

and

(3.26)
$\displaystyle\|K_{Z}(\cdot,w)\|_{\mathcal{H}}\leq\|K_{\phi_{Z}}(\cdot,w)\|_{\mathcal{H}_{\phi_{Z}}},$

where $\phi_{Z}$ is the canonical Blaschke product defined by $Z,$
${\mathcal{H}_{\phi_{Z}}}$ is the $|\phi_{Z}|^{2}$-weighted $\mathcal{H}$
space, $K_{\phi_{Z}}$ is its reproducing kernel. As a consequence, the
normalized reproducing kernel is

(3.27)
$\displaystyle\frac{K_{Z}(z,w)}{\|K_{Z}(\cdot,w)\|}=\frac{\overline{\phi_{Z}(w)}}{|\phi_{Z}(w)|}\frac{\phi_{Z}(z)K_{\phi_{Z}}(z,w)}{K^{1/2}_{\phi_{Z}}(w,w)}.$

Proof. We note that $K_{Z}(z,w)$ has zero set $Z$ for the variable $z$ when
$w$ is fixed in a neighbourhood of $\overline{\bf D};$ and zero set $Z$ for
the variable $w$ when $z$ is fixed in a neighbourhood of $\overline{\bf D}.$
Therefore, $\phi_{Z}^{-1}(z)K_{Z}(z,w)\overline{\phi}_{Z}^{-1}(w)$ is analytic
for $z$ in a neighbourhood of $\overline{\bf D},$ and anti-analytic for $w$ in
a neighbourhood of $\overline{\bf D}.$ Let $f\in\mathcal{H}_{\phi_{Z}}.$ In
the case $f\phi_{Z}\in\mathcal{H}_{Z}.$ We have

$\displaystyle\langle
f,\phi^{-1}_{Z}(\cdot)K_{{Z}}(\cdot,w)\overline{\phi}^{-1}_{Z}(w)\rangle_{\mathcal{H}_{\phi_{Z}}}$
$\displaystyle=$ $\displaystyle\phi^{-1}_{Z}(w)\langle
f\phi_{Z},K_{{Z}}(\cdot,w)\rangle_{\mathcal{H}_{Z}}$ $\displaystyle=$
$\displaystyle\phi^{-1}_{Z}(w)\langle
f\phi_{Z},K_{{Z}}(\cdot,w)\rangle_{\mathcal{H}_{Z}}$ $\displaystyle=$
$\displaystyle\phi^{-1}_{Z}(w)f(w)\phi_{Z}(w)$ $\displaystyle=$ $\displaystyle
f(w).$

Therefore, $\mathcal{H}_{\phi_{Z}}$ is a RKHS. Due to uniqueness of
reproducing kernel, its kernel $K_{\phi_{Z}}(z,w)$ satisfies the relation
(3.25). To prove (3.26) we have

$\displaystyle\|K_{Z}(\cdot,w)\|^{2}_{\mathcal{H}}$ $\displaystyle=$
$\displaystyle K_{Z}(w,w)$ $\displaystyle=$
$\displaystyle\phi_{Z}(w)K_{\phi_{Z}}(w,w)\overline{\phi_{Z}(w)}$
$\displaystyle=$ $\displaystyle K_{\phi_{Z}}(w,w)|\phi_{Z}(w)|^{2}$
$\displaystyle\leq$ $\displaystyle K_{\phi_{Z}}(w,w)$ $\displaystyle=$
$\displaystyle\|K_{\phi_{Z}}(\cdot,w)\|^{2}_{\mathcal{H}_{\phi_{Z}}}.$

The relation (3.27) is just by dividing $K_{Z}(z,w)$ with
$\|K_{Z}(\cdot,w)\|=\sqrt{K_{Z}(w,w)}$ and invoking (3.25).

$\hfill\square$

######  Lemma 3.6.

For any Hilbert space $\mathcal{H}$ with reproducing kernel $K$ and any
$a\in\bf D$ there hold

(3.28) $\displaystyle K(a,a)=\sup\\{|f(a)|^{2}\ :\ f\in\mathcal{H},\|f\|\leq
1\\}$

and

(3.29) $\displaystyle K(a,a)\leq K_{\phi_{Z}}(a,a).$

Proof. Recall that $E_{a}(z)=K(z,a)/\|K_{a}\|.$ On one hand, $\|E_{a}\|=1.$ On
the other hand, for any $f$ satisfying $\|f\|=1,$ using the Cauchy-Schwarz
inequality,

$|f(a)|^{2}=|\langle f,K_{a}\rangle|^{2}\leq\|K_{a}\|^{2}=K(a,a).$

So, $E_{a}$ is a solution for the extremal problem. Using this argument also
to $\mathcal{H}_{\phi_{Z}}$ and $K_{\phi_{Z}}$, we obtain

$\displaystyle K(a,a)$ $\displaystyle=$ $\displaystyle\sup\\{|f(a)|^{2}\ :\
f\in\mathcal{H},\|f\|_{\mathcal{H}}\leq 1\\}$ $\displaystyle\leq$
$\displaystyle\sup\\{|f(a)|^{2}\ :\
f\in\mathcal{H}_{\phi_{Z}},\|f\|_{\mathcal{H}_{\phi_{Z}}}\leq 1\\}$
$\displaystyle=$ $\displaystyle K_{{\phi_{Z}}}(a,a),$

as desired. $\hfill\square$

Now we proceed with the main technical step of the proof. Denote by
$Z_{n-1}^{(l)}$ the $l$-level zero set
$(a^{(l)}_{1},a^{(l)}_{2},...,a^{(l)}_{n-1}).$ With the above preparations we
have

$\displaystyle\langle
f,E_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n}}\rangle_{\mathcal{H}}$
$\displaystyle=$ $\displaystyle\langle
f_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}},E_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n}}\rangle_{\mathcal{H}}\qquad\left(E_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n}}=\frac{Q_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}}(K_{a^{(l)}_{n}})}{\|Q_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}}(K_{a^{(l)}_{n}}\|}=\frac{Q^{2}_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}}(K_{a^{(l)}_{n}})}{\|Q_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}}(K_{a^{(l)}_{n}}\|}\right)$
$\displaystyle=$ $\displaystyle\left\langle
f_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}},\frac{K_{Z_{n-1}^{(l)}}(\cdot,a^{(l)}_{n})}{\|K_{Z_{n-1}^{(l)}}(\cdot,a^{(l)}_{n})\|}\right\rangle_{\mathcal{H}}\qquad\left(Q_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}}(K_{a^{(l)}_{n}})=K_{Z_{n-1}^{(l)}}(\cdot,a^{(l)}_{n})\right)$
$\displaystyle=$
$\displaystyle\frac{\overline{\phi_{Z^{(l)}_{n-1}}(a^{(l)}_{n})}}{|\phi_{Z^{(l)}_{n-1}}(a^{(l)}_{n})|}\langle
f_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}},K_{\phi_{Z_{n-1}^{(l)}}}(a^{(l)}_{n},a^{(l)}_{n})^{-1/2}\phi_{Z^{(l)}_{n-1}}K_{\phi_{Z_{n-1}^{(l)}}}(\cdot,a^{(l)}_{n})\rangle_{\mathcal{H}}\qquad\left({\rm
Lemma}\ \ref{zero space kernel}\right)$ $\displaystyle=$
$\displaystyle\frac{\overline{\phi_{Z^{(l)}_{n-1}}(a^{(l)}_{n})}}{|\phi_{Z^{(l)}_{n-1}}(a^{(l)}_{n})|}\left\langle\frac{f_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}}}{\phi_{Z^{(l)}_{n-1}}},|\phi_{Z^{(l)}_{n-1}}|^{2}K_{\phi_{Z_{n-1}^{(l)}}}(\cdot,a^{(l)}_{n})\right\rangle_{\mathcal{H}}\frac{1}{\sqrt{K_{\phi_{Z_{n-1}^{(l)}}}(a^{(l)}_{n},a^{(l)}_{n})}}\qquad\left({\rm
Lemma}\ \ref{zero space kernel}\right)$ $\displaystyle=$
$\displaystyle\frac{\overline{\phi_{Z^{(l)}_{n-1}}(a^{(l)}_{n})}}{|\phi_{Z^{(l)}_{n-1}}(a^{(l)}_{n})|}\left\langle\frac{f_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}}}{\phi_{Z^{(l)}_{n-1}}},K_{\phi_{Z_{n-1}^{(l)}}}(\cdot,a^{(l)}_{n})\right\rangle_{\mathcal{H}_{\phi_{Z^{(l)}_{n-1}}}}\frac{1}{\sqrt{K_{\phi_{Z_{n-1}^{(l)}}}(a^{(l)}_{n},a^{(l)}_{n})}}$
$\displaystyle=$
$\displaystyle\frac{\overline{\phi_{Z^{(l)}_{n-1}}(a^{(l)}_{n})}}{|\phi_{Z^{(l)}_{n-1}}(a^{(l)}_{n})|}\frac{f_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}}(a^{(l)}_{n})}{\phi_{Z^{(l)}_{n-1}}(a^{(l)}_{n})}\frac{1}{\sqrt{K_{\phi_{Z_{n-1}^{(l)}}}(a^{(l)}_{n},a^{(l)}_{n})}}\quad\left({\rm
Lemma}\ \ref{zero space kernel}\ {\rm and\ reproducing\ kernel\
property}\right).$

To conclude the theorem it is sufficient to show that the above quantity tends
to zero along with ${\bf D}\ni a^{(l)}_{n}\to a_{n}\in\partial\bf D$ uniformly
in $a^{(l)}_{1},\cdots,a^{(l)}_{n-1}\in\bf D$ for $l=1,2\cdots$ It then
suffices to prove

$1^{o}.$

$\frac{f_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n-1}}(a^{(l)}_{n})}{\phi_{Z_{n-1}^{(l)}}(a^{(l)}_{n})}$

is bounded uniformly in $a_{1}^{(l)},\cdots,a_{n-1}^{(l)}$ and
$a_{n}^{(l)},l=1,2,\cdots;$ and

$2^{o}.$

$\lim_{l\to\infty}K_{\phi_{Z_{n-1}^{(l)}}}(a^{(l)}_{n},a^{(l)}_{n})=\infty$

uniformly in $a_{1}^{(l)},\cdots,a_{n-1}^{(l)},l=1,2,\cdots$

Now we show assertion $1^{o}.$ First by Lemma 3.1 the function

$g_{Z^{(l)}_{n-1}}(z)=\frac{f_{a^{(l)}_{1}\cdots
a^{(l)}_{n-1}}(z)}{\phi_{Z^{(l)}_{n-1}}(z)}$

is analytic in a neighbourhood of $\overline{\bf D}.$ By invoking the maximum
modulus principle for one complex variable in $\overline{\bf D},$ Lemma 3.4,
as well as the fact that all finite Blaschke products are of modulus $1$ on
the boundary $\partial{\bf D},$ we have

$\displaystyle\max\\{|g_{Z^{(l)}_{n-1}}(z)|\ :\ z\in\overline{\bf D}\\}$
$\displaystyle=$ $\displaystyle\max\\{|g_{Z^{(l)}_{n-1}}(\zeta)|\ :\
\zeta\in\partial{\bf D}\\}$ $\displaystyle=$
$\displaystyle\max\\{\left|f_{a^{(l)}_{1}\cdots a^{(l)}_{n-1}}({\rm
e}^{it})\right|\ :\ t\in\partial{\bf D}\\}$ $\displaystyle\leq$ $\displaystyle
M(1+C_{\mathcal{H}})^{n-1},$

concluding the uniform boundedness claimed of $1^{o}.$ The assertion $2^{o}$
is a consequence of the condition (1.7) and Lemma 3.6.

##  4\. Applications

###  4.1. The classical Hardy space

By taking $\mathcal{H}=\mathbb{H}^{2}({\bf D}),$ we are with the inner product
(1.1), and the reproducing kernel $K(z,w)=k_{w}(z)=\frac{1}{1-\overline{w}z}.$
Since

$K(a,a)=\frac{1}{1-|a|^{2}}\to\infty\ {\rm as}\ |a|\to 1,\ {\rm and}\
\frac{|K_{a}(z)|}{K(a,a)}=\frac{1-|a|^{2}}{|1-\overline{a}z|}\leq 2,$

the conditions (1.7) and (1.8) are satisfied. We hence have existence of the
$n$-best approximation.

###  4.2. The Bergman spaces

Let $\mathcal{H}$ be the weighted Bergman spaces with the definition and
notation

$\displaystyle\mathbb{A}^{2}_{\alpha}({\bf D})=\\{f:{\bf D}\to{\bf C}\ |\ f\
{\rm is\ holomorphic\ in\ \bf D},{\rm and}\
\|f\|_{\mathbb{A}^{2}_{\alpha}({\bf D})}^{2}=\int_{\bf
D}|f(z)|^{2}dA_{\alpha}<\infty\\},$

where $\alpha\in(-1,\infty),dA_{\alpha}=(1+\alpha)(1-|z|^{2})^{\alpha}dA(z),$
and $dA=\frac{dxdy}{\pi},z=x+iy,$ is the normalized area measure of the unit
disc. The inner product of $\mathbb{A}^{2}_{\alpha}({\bf D})$ is defined as

$\langle f,g\rangle_{\mathbb{A}^{2}_{\alpha}(\bf D)}=\int_{\bf
D}f(z)\overline{g(z)}dA_{\alpha}.$

In the sequel we sometimes write $\|\cdot\|_{\mathbb{A}^{2}_{\alpha}({\bf
D})}$ and $\langle\cdot,\cdot\rangle_{\mathbb{A}^{2}_{\alpha}({\bf D})}$
briefly as $\|\cdot\|$ and $\langle\cdot,\cdot\rangle,$ and
$\mathbb{A}^{2}_{\alpha}({\bf D})$ as $\mathbb{A}^{2}_{\alpha}.$

$\mathbb{A}^{2}_{\alpha}$ is a RKHS with reproducing kernel

$k_{a}^{\alpha}(z)=\frac{1}{(1-\overline{a}z)^{2+\alpha}}.$

By invoking the reproducing kernel property we have

(4.30)
$\displaystyle\|k_{a}^{\alpha}\|^{2}=k_{a}^{\alpha}(a)=\frac{1}{(1-|a|^{2})^{2+\alpha}}.$

This shows that the condition (1.7) holds for all $\mathbb{A}^{2}_{\alpha}.$ A
simple computation gives

$\frac{k_{a}^{\alpha}(z)}{{k_{a}^{\alpha}(a)}}=\frac{{(1-|a|^{2})^{2+\alpha}}}{(1-\overline{a}z)^{2+\alpha}}\leq
2^{2+\alpha}.$

Hence, the reproducing kernel satisfies the condition (1.8). Therefore, an
$n$-best approximation exists in all the weighted Bergman spaces. This is a
re-proof of the main result of [52].

###  4.3. The weighted Hardy spaces

Let $W(k)$ be a sequence of non-negative numbers satisfying
$\lim_{k\to\infty}W(k)^{\frac{1}{k}}\geq 1$ ([34]). Denote by
$\mathbb{H}_{W}({\bf D})$ the $W$-weighted Hardy $\mathbb{H}^{2}$-space
defined by

$\mathbb{H}_{W}({\bf D})=\\{f\ :\ {\bf D}\to{\bf{C}}\ :\
f(z)=\sum_{k=0}^{\infty}c_{k}z^{k},\ z\in{\bf
D},\|f\|_{\mathbb{H}_{W}}=\sum_{k=0}^{\infty}W(k)|c_{k}|^{2}<\infty\\}.$

We will be considering an ordered sequence of $W$-weighted Hardy spaces
defined by the weights $W_{\beta}(k)=(1+k)^{\beta},-\infty<\beta<\infty.$ This
class of function spaces is a generalization of the Hardy and the weighted
Bergman spaces. In fact, $\mathbb{H}_{W_{0}}({\bf D})=\mathbb{H}^{2}({\bf
D}),$ and $\mathbb{H}_{W_{\beta}}({\bf D})=\mathbb{A}^{2}_{\alpha}({\bf
D}),\alpha=-\beta-1,\beta<0\ (\alpha>-1),$ $\mathbb{H}_{W_{-1}}({\bf D})$ is
the standard Bergman, and $\mathbb{H}_{W_{1}}({\bf D})$ is the Dirichlet space
in $\bf D.$ The spaces $\mathbb{H}_{W_{\beta}}({\bf D})$ are, as a matter of
fact, equivalent with the Hardy-Sobolev spaces $W^{\frac{\beta}{2},2}.$ From
the last two subsections we know that the spaces $\mathbb{H}_{W_{\beta}}({\bf
D}),\beta\leq 0,$ have $n$-best approximation. We now extend the result to
$0<\beta\leq 1.$

The inner product of $\mathbb{H}_{W_{\beta}}({\bf D}),-\infty<\beta<\infty,$
is

$\langle f,g\rangle=\sum_{k=0}^{\infty}(k+1)^{\beta}c_{k}\overline{d}_{k},$

where $c_{k}$ and $d_{k}$ are, respectively, the coefficients of the Taylor
expansions of $f$ and $g.$ From this it can be directly verified that the
reproducing kernel of $\mathbb{H}_{W_{\beta}}({\bf D})$ is

$k^{\beta}_{a}(z)=\sum_{k=0}^{\infty}\frac{(z\overline{a})^{k}}{(k+1)^{\beta}}.$

The function

(4.31) $\displaystyle
k^{\beta}_{a}(a)=\sum_{k=0}^{\infty}\frac{|a|^{2k}}{(k+1)^{\beta}}$

is an increasing function in $|a|,$ and for any large $N,$

$\underline{\lim}_{|a|\to 1-}k^{\beta}_{a}(a)\geq\lim_{|a|\to
1-}\sum_{k=0}^{N}\frac{|a|^{2k}}{(k+1)^{\beta}}=\sum_{k=0}^{N}\frac{1}{(k+1)^{\beta}}.$

Therefore, for all $\beta\leq 1,$

$\lim_{|a|\to 1-}k^{\beta}_{a}(a)=\infty,$

verifying (1.7). Next we show that the weighted Hardy spaces kernels
$k^{\beta}_{a}$ satisfy the condition (1.8). This requires to prove that the
function $\frac{|k^{\beta}_{a}(z)|}{k^{\beta}_{a}(a)}$ is uniformly bounded in
$a,z\in{\bf D}.$ The following estimation uses the well know technique for
summing up series of positive decreasing entries: If $f$ is a positive
decreasing function integrable over $(0,\infty),$ then

$\int_{1}^{\infty}f(t)dt\leq\sum_{k=1}^{\infty}f(k)\leq\int_{0}^{\infty}f(t)dt.$

The estimation amounts to numerically comparing some elementary integrals.
Denote by $|a|=r<1.$ Then

$\displaystyle\frac{|k^{\beta}_{a}(z)|}{k^{\beta}_{a}(a)}$ $\displaystyle=$
$\displaystyle\frac{|\sum_{k=0}^{\infty}\frac{(\overline{a}z)^{k}}{(1+k)^{\beta}}|}{\sum_{k=0}^{\infty}\frac{|a|^{2k}}{(1+k)^{\beta}}}$
$\displaystyle\leq$
$\displaystyle\frac{\sum_{k=0}^{\infty}\frac{r^{k}}{(1+k)^{\beta}}}{\sum_{k=0}^{\infty}\frac{r^{2k}}{(1+k)^{\beta}}}$
$\displaystyle\leq$
$\displaystyle\frac{\int_{0}^{\infty}\frac{r^{x}}{(1+x)^{\beta}}dx}{\int_{1}^{\infty}\frac{r^{2x}}{(1+x)^{\beta}}dx}$
$\displaystyle=$
$\displaystyle\frac{\int_{0}^{\infty}\frac{r^{x}}{(1+x)^{\beta}}dx}{2^{\beta-1}\int_{2}^{\infty}\frac{r^{x}}{(2+x)^{\beta}}dx}\qquad({\rm
change\ of\ variable})$ $\displaystyle=$
$\displaystyle\frac{\int_{0}^{\infty}\frac{r^{x}}{(1+x)^{\beta}}dx}{2^{\beta-1}\left(\int_{0}^{\infty}\frac{r^{x}}{(2+x)^{\beta}}dx-\int_{0}^{2}\frac{r^{x}}{(2+x)^{\beta}}dx\right)}\qquad(\frac{\infty}{\infty}\
{\rm type\ when\ }\ r\to 1-)$ $\displaystyle\leq$
$\displaystyle\frac{\int_{0}^{\infty}\frac{r^{x}}{(1+x)^{\beta}}dx}{2^{\beta-2}\int_{0}^{\infty}\frac{r^{x}}{(1+x)^{\beta}}\frac{(1+x)^{\beta}}{(2+x)^{\beta}}dx}\qquad({\rm
if}\ r\geq\ {\rm some}\ r_{0}\in(0,1))$ $\displaystyle\leq$ $\displaystyle
4.\qquad({\rm mean-value\ theorem\ of\ integration})$

For $r\leq r_{0}$ the estimated quantity also has a uniform bound. Hence, for
$\beta\leq 1,$ the spaces $\mathbb{H}_{W_{\beta}}({\bf D})$ satisfy the three
conditions (i), (ii), and (iii). By invoking Theorem 2.1 the $n$-best
approximation problems have solutions in those spaces.

######  Remark 4.1.

(For the spaces $\mathbb{H}_{W_{\beta}}({\bf D}),\beta>1$) The spaces
$\mathbb{H}_{W_{\beta}}({\bf D}),\beta>1,$ do not fall into the category
governed by Theorem 2.1, as, owing to (4.31), the condition (1.7) is not
satisfied. Recall the Sobolev Embedding theorem asserting that $W^{k,p}\subset
C^{r,\alpha}$ if $m<pk,\frac{1}{p}-\frac{k}{m}=-\frac{r+\alpha}{m},m$ is the
dimension. In our case $m=1,p=2,k=\frac{\beta}{2},$ and, in particular,
$\beta=pk>1.$ It hence concludes that the functions in the spaces
$\mathbb{H}_{W_{\beta}}({\bf D}),\beta>1,$ are all continuously extendable to
the closed unit disc, and the norm square
$\|k^{\beta}_{a}\|^{2}=k^{\beta}_{a}(a)$ given by (4.31) does not have
singularity for $|a|=1.$ Based on these, as well as continuity of the inner
product, we conclude existence of $n$-best approximations of the spaces for
$\beta>1.$

The results of this section are summarized as

######  Theorem 4.2.

For all Hardy-Sobolev spaces $\mathbb{H}_{W_{\beta}}({\bf
D}),-\infty<\beta<\infty,$ there exist solutions to the $n$-best kernel
approximation problem.

##  5\. Stochastic $n$-best Approximation

Let $(\Omega,\mathcal{F},d\mathbb{P})$ be a probability space, and, as in the
previous sections, $\mathcal{H}$ be a RKHS of analytic functions in $\bf D$
with reproducing kernel $K_{w},w\in\bf D.$ We will be studying stochastic
signals $f(z,{\xi}),$ where $\xi\in\Omega$ and $z\in\bf D:$ We assume that for
a.s. $\xi\in\Omega,f(\cdot,\xi)$ is a function in $\mathcal{H};$ and, for a.e.
$z\in{\bf D},f(z,\cdot)$ is a random variable. We will use the notation
$f_{\xi}(z)=f(z,\xi).$ Associated with the probability space and the RKHS we
define a Bochner type space ([21])

(5.32) $\displaystyle L^{2}(\mathcal{H},\Omega)=\\{f:{\bf
D}\times\Omega\to{\bf{C}}\ :\
\|f\|^{2}_{L^{2}(\mathcal{H},\Omega)}=E_{\xi}\|f_{\xi}\|^{2}_{\mathcal{H}}<\infty\\},$

where $E_{\xi}$ denotes the expectation, and precisely,

$E_{\xi}\|f_{\xi}\|^{2}_{\mathcal{H}}=\int_{\Omega}\|f_{\xi}\|^{2}_{\mathcal{H}}d\mathbb{P}(\xi).$

We often use the simplified notation $\mathcal{N}=L^{2}(\mathcal{H},\Omega).$

The following theorem generalizes the existence result for stochastic $n$-best
approximation for the Hardy space (see [51]) to the RKHSs satisfying the
conditions (i), (ii), and (iii) as assumed in Theorem 2.1.

######  Theorem 5.1.

Let $\mathcal{H}$ be a RKHS of analytic functions in the unit disc satisfying
the conditions (i), (ii), and (iii) in §1, and $\Omega$ a probability space.
Let $\mathcal{N}=L^{2}(\mathcal{H},\Omega)$ be the associated Bochner type
space as above defined. Let $f$ be any non-zero random signal. Then for any
positive integer $n,$ either of the following two cases holds: (1) For some
$1\leq m_{1}\leq n,$ there exists an $m_{1}$-tuple of constant parameters
$(a_{1},\cdots,a_{m_{1}})\in{\bf D}^{m_{1}}$ such that $f$ is identical with
the orthogonal expansion

(5.33) $\displaystyle f(z,\xi)=\sum_{k=1}^{m_{1}}\langle
f_{\xi},E_{k}\rangle_{\mathcal{H}}E_{k}(z);$

or (2) There exists an $n$-tuple of constant parameters
$(a_{1},\cdots,a_{n})\in{\bf D}^{n}$ such that

(5.34) $\displaystyle\|f-\sum_{k=1}^{n}\langle
f_{\xi},E_{k}\rangle_{\mathcal{H}}E_{k}\|_{\mathcal{N}}$

attains its positive infimum over all possible $n$-orthonormal systems
$\\{E_{k}\\},$ where in both cases, $\\{E_{k}\\}_{k=1}^{m}$ is the orthonormal
system generated by $(\tilde{K}_{a_{1}},\cdots,\tilde{K}_{a_{m}}),1\leq m\leq
n.$

The proof of Theorem 2.1 of [51] for the stochastic Hardy space case cannot be
directly adopted, for, in the present case no density argument based on the
boundary value of the given function on $\partial\bf D$ is available. In §3 we
established, by using a new technical method, the pointwise convergence result
(3.24) which is crucial to in proving Theorem 5.1.

Proof. In the proof of Theorem 2.1 we already show that, for each $\xi$
outside an event in $\Omega$ of probability zero, there holds uniformly in
$a^{(l)}_{1},a^{(l)}_{2},...,a^{(l)}_{n-1},l=1,2,\cdots,$ that

$\lim_{l\to\infty}|\langle
f_{\xi},E_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n}}\rangle_{\mathcal{H}}|^{2}=0.$

By using the Cauchy-Schwarz inequality for the space $\mathcal{H}$ we have a
dominating function of the function sequence on the left hand side:

$|\langle
f_{\xi},E_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n}}\rangle_{\mathcal{H}}|^{2}\leq\|f_{\xi}\|^{2}\in
L^{1}(\Omega).$

Then the Lebesgue dominated convergence theorem be invoked to conclude

(5.35) $\displaystyle\lim_{l\to\infty}E_{\xi}|\langle
f_{\xi},E_{a^{(l)}_{1}a^{(l)}_{2}...a^{(l)}_{n}}\rangle_{\mathcal{H}}|^{2}=0$

uniformly in $a^{(l)}_{1},a^{(l)}_{2},...,a^{(l)}_{n-1},l=1,2,\cdots.$ Based
on this the contradiction argument used in proving Theorem 2.1 of [51] may be
adopted to conclude the theorem.$\hfill\square$

######  Remark 5.2.

(Impact to Algorithm) This present paper only treats the existence aspect of
the $n$-best problem. Based on the obtained estimates, however, a mathematical
algorithm to actually get a solution is now on its way. We now cite the
crucial step to reduce the problem to a global optimization one of a
differential function defined in a compact set. Separate studies will be
devoted to the computation aspect. To have an $n$-best approximation algorithm
we are under the assumption that the given function $f$ is not expressible by
any $m$-linear combination of multiple kernels for $m\leq n-1.$ This implies
that $d_{f}(n-1)>0,$ and there exists an $n$-tuple
$(b_{1},\cdots,b_{n})\in{\bf D}^{n}$ such that for some $\epsilon>0$

(5.36) $\displaystyle\|f-P_{b_{1}\cdots b_{n}}f\|=d_{f}(n-1)-\epsilon.$

By using (3.24) one can find $\delta>0$ such that if $|a_{n}|>1-\delta,$ then

$\|f-P_{a_{1}\cdots a_{n-1}a_{n}}f\|>d_{f}(n-1)-\epsilon$

for any $a_{1},\cdots,a_{n-1}$ in $\bf D.$ Since $P_{a_{1}\cdots
a_{n-1}a_{n}}f$ is symmetric in $a_{1},\cdots,a_{n},$ we conclude

$\|f-P_{a_{1}\cdots a_{n-1}a_{n}}f\|>d_{f}(n-1)-\epsilon$

whenever $|a_{k}|>1-\delta$ for some $k=1,\cdots,n.$ Owing to the observation
(5.36) we have $d_{f}(n)\leq d_{f}(n-1)-\epsilon.$ When the infimum attains at
$(\tilde{a}_{1},\cdots,\tilde{a}_{n-1},\tilde{a}_{n}),$ that is,

$\|f-P_{\tilde{a}_{1}\cdots\tilde{a}_{n-1}\tilde{a}_{n}}f\|=d_{f}(n),$

there holds $|\tilde{a}_{k}|\leq 1-\delta$ for all $k=1,\cdots,n.$ This
concludes that the global minimum value $d_{f}(n)$ is only attainable in the
compact set $\overline{(1-\delta)\bf D}^{n}.$ In view of the relation (5.35)
the same conclusion holds for the stochastic case.

## Acknowledgement

The author wishes to express his sincere thanks to Y.B. Wang, K.H. Zhu and W.
Qu for the previous works done with the author, as well as for their
inspiriting discussions on this and related subjects.

## References

  * [1] D. Alpay, F. Colombo, T. Qian, I. Sabadini, Adaptive orthonormal systems for matrix-valued functions, Proceedings of the American Mathematical Society, 2017, 145(5): 2089-2106.
  * [2] D. Alpay, F. Colombo, T. Qian, and I. Sabadini, Adaptative Decomposition: The Case of the Drury-Arveson Space, Journal of Fourier Analysis and Applications, 2017, 23(6): 1426-1444.
  * [3] D. Alpay, H. Dym, On a new class of realization formulas and their application, Linear algebra and its applications, 1996, 241: 3-84.
  * [4] H. Akcay, B. Ninness, Orthonormal basis functions for modelling continuous-time system, Signal Processing, 1999, 77(3): 261-274.
  * [5] J. A. Ball, V. Bplotnikov, Weighted Begman spaces: shift invariant subspaces and input/state/output linear systems, Integral Equations and Operator Theory, 2013, 76(3): 301-356.
  * [6] J. A. Ball, V. Bplotnikov, Weighted Hardy spaces: shift invariant and coinvariant subspaces, linear systema and operator model theory, arXiv:1405.2974 [math.CA].
  * [7] S. Bochner, Vorlesungen Ueber Fouriersche Integrale, Leipzig, 1932.
  * [8] H.H. Bauschke, P.L. Combettes, D.R. Luke, Finding best approximation pairs relative to two closed convex sets in Hilbert spaces, Journal of Approximation theory, 2004, 127(2): 178-192.
  * [9] L. Baratchart, M. Cardelli, M. Olivi, Identification and rational $L^{2}$ approximation-a gradient algorithm, Automatica, 1991, 27(2): 413-417.
  * [10] D.W. Byun, S. Saitoh, Best Approximation in Reproducing Kernel Hilbert Spaces, Second International Colloquium on Numerical Analysis, 1994: 55.
  * [11] L. Baratchart, E.B. Staff, F. Wielonsky, A criterion for uniqueness of a critical point in $H^{2}$ rational approximation, Journal d′Analyse Math$\acute{e}$matique, 1996, 70(1): 225-266.
  * [12] N.I. Chernykh, Best approximation of periodic functions by trigonometric polynomials in $L^{2}$, Mathematical notes of the Academy of Sciences of the USSR, 1967, 2(5): 803-808.
  * [13] M.M. Chawla, V. Kaul, Optimal rules with polynomial precision for Hilbert spaces possessing reproducing kernel functions, Numerische Mathematik, 1974, 22(3): 207-218.
  * [14] E.W. Cheney, H.L. Loeb, Generalized Rational Approximation, Journal of the Society for Industrial $\&$ Applied Mathematics, 1964, 1(1): 11-25.
  * [15] R. Coifman, S. Steinerberger, Nonlinear phase unwinding of functions, J Fourier Anal Appl., 2017, 23(4): 778-809.
  * [16] R. Coifman, J. Peyriére, Phase unwinding, or invariant subspace decompositions of Hardy spaces, J Fourier Anal Appl., 2019, 25: 684-695.
  * [17] J.S. Chen, S. Yoon, H.P. Wang, W.K. Liu, An improved reproducing kernel particle method for nearly incompressible finite elasticity, Computer methods in applied mechanics and engineering, 2000, 181(1-3): 117-145.
  * [18] F. Deutsch, Best approximation in inner product spaces, Springer, 2001.
  * [19] J. Descloux, Approximations in $L^{p}$ and Chebyshev approximations, Journal of the Society for Industrial and Applied Mathematics, 1963, 11(4): 1017-1026.
  * [20] P. Duren, A. Schuster, S. Alexander, Bergman spaces, American Mathematical Society, 2004.
  * [21] L.C. Evans, Partial differential equations, American Mathematical Society, 1998, 19(4): 7.
  * [22] C. Gu,Penalized likelihood estimation: convergence under incorrect model, Statistics and probability letters, 1998, 36(4): 359-364.
  * [23] F. Gunther, W.K. Liu, Implementation of boundary conditions for meshless methods, Computer Methods in Applied Mechanics and Engineering, 1998, 163(1-4): 205-230.
  * [24] F. Gunther, W.K. Liu, D. Diachin, M.A. Christon, Multi-scale meshfree parallel computations for viscous, compressible flows, Computer methods in applied mechanics and engineering, 2000, 190(3-4): 279-303.
  * [25] H. Hult, Approximating some Volterra type stochastic integrals with applications to parameter estimation, Stochastic processes and their applications, 2003, 105(1): 1-32.
  * [26] S. Hao, W.K. Liu, C.T. Chang, Computer implementation of damage models by finite element and meshfree methods, Computer methods in applied mechanics and engineering, 2000, 187(3-4): 401-340.
  * [27] W. Han, X. Meng, Error analysis of the reproducing kernel particle method, Computer methods in applied mechanics and engineering, 2001, 190(46-47): 6157-6181.
  * [28] R. Khalil, W. Deeb, Best approximation in $L^{p}(\mu,X)$, II, Journal of approximation theory, 1989, 59(3): 296-299.
  * [29] B. Korenblum, H. Hedenmalm, K.H. Zhu, D. Békollé, Theory of Bergman spaces, The Mathematical Intelligencer, 2005, 27(1): 85-86.
  * [30] F.M. Larkin, Optimal approximation in Hilbert spaces with reproducing kernel functions, Mathematics of Computation, 1970, 24(112): 911-921.
  * [31] W.K. Liu, Y. Chen, R.A. Uras, C.T. Chang, Generalized multiple scale reproducing kernel particle methods, Computer Methods in Applied Mechanics and Engineering, 1996, 139(1-4): 91-157.
  * [32] S. Li, W.K. Liu, Moving least-square reproducing kernel method Part II: Fourier analysis, Computer Methods in Applied Mechanics and Engineering, 1996, 139(1-4): 159-193.
  * [33] S. Li, W.K. Liu, T. Belytschko, Moving least-square reproducing kernel methods (I) methodology and convergence, Computer methods in applied mechanics and engineering, 1997, 143(1-2): 113-154.
  * [34] B.D. MacCluer, Elementary functional analysis, Springer, New York, 2009.
  * [35] Y.T. Li, L.M. Zhang, T. Qian, 2D partial unwinding$-$a novel non-linear phase decomposition of images, IEEE Transactions on Image Processing, 2019, 28(10): 4762-4773.
  * [36] J. Mercer, Function of positive and negative and their connection with the theory of integral equation, Philos. Trans. Roy. Soc. London Ser. A., 1909, 209: 415-446.
  * [37] E. H. Moore, R. W. Barnard, General Analysis I, the American Philosophical Society, 1935.
  * [38] F.M. Larkin, Optimal approximation in Hilbert spaces with reproducing kernel functions, mathematics of computation, 1970, 24(112): 911-921.
  * [39] W. Mi, T. Qian, Frequency domain identification: an algorithm based on adaptive rational orthogonal system, Automatica, 2012, 48(6): 1154-1162.
  * [40] W. Mi, T. Qian, On backward shift algorithm for estimating poles of systems, Automatica, 2014, 50(4): 1603-1610.
  * [41] S. Mallat, Z. Zhang, Matching pursuit with time-frequency dictionaries, IEEE Trans. Signal Processing, 1993, 41: 3397-3315.
  * [42] B.U. Park, W.C. Kim, M.C. Jones, On identity reproducing nonparametric regression estimators, Statistics and probability letters, 1997, 32(3): 279-290.
  * [43] T. Qian, Two-dimensional adaptive Fourier decomposition, Mathematical Methods in the Applied Sciences, 2016, 39(10): 2431-2448.
  * [44] T. Qian, Sparse representation of stochastic signals, accepted by Mathematical Methods in the Applied Sciences, 2021.
  * [45] T. Qian, Y.B. Wang, Adaptive Fourier series$-$a variation of greedy algorithm, Advances in Computational Mathematics, 2011, 34(3): 279-293.
  * [46] T. Qian, E. Wegert, Optimal Approximation by Blaschke Forms, Complex Variables and Elliptic Equations, 2013, 58(1): 123-133.
  * [47] T. Qian, J.Z. Wang, W.X. Mai, An enhancement algorithm for cyclic adaptive Fourier decomposition, Applied and Computational Harmonic Analysis, 2019, 47(2): 516-525.
  * [48] T. Qian, X.Y. Wang, L.M. Zhang, MIMO frequency domain system identification using matrix-valued orthonormal functions, Automatica, 2021, 133: 109882.
  * [49] W. Qu, P. Dang, Rational approximation in a class of weighted Hardy spaces, Complex Analysis and Operator Theory, 2019, 13(4): 1827-1852.
  * [50] W. Qu, P. Dang, Reproducing kernel approximation in weighted Bergman spaces: algorithm and applications, Mathematical Methods in the Applied Sciences, 2019, 42(12): 4292-4304.
  * [51] W. Qu, T. Qian, G.T. Deng, A stochase sparse representation: $n$-best approximation to random signals and computation, Applied and Computational Harmonic Analysis, 2021, 55(1): 185-198.
  * [52] W. Qu, T. Qian, H.C. Li, K.H. Zhu, Best kernel approximation in Bergman spaces, Applied Mathematics and Computation, 2021.
  * [53] W. Qu, C.K. Chui, G.T. Deng, T. Qian, Sparse approximation to the Dirac-$\delta$ distribution, Analysis and Applications, 2021.
  * [54] I. Singer, The theory of best approximation and functional analysis, Society for Industrial and Applied Mathematics, 1974: 259-269.
  * [55] K. Seip, Interpolation and sampling in spaces of analytic functions, American Mathematical Soc., 2004.
  * [56] S.A. Sahab, M.S. Khan, S. Sessa, A result in best approximation theory, Journal of approximation theory, 1988, 55(3): 349-351.
  * [57] D.J. Strauss, G. Steidl, Hybrid wavelet-support vector classification of waveforms, Journal of computational and applied mathematics, 2002, 148(2): 375-400.
  * [58] L.N. Trefethen, Approximation Theory and Approximation Practice, Extended Edition, Society for Industrial and Applied Mathematics, 2019.
  * [59] V.N. Temlyakov, Greedy algorithm and $m$-term trigonometric approximation, Constructive Approximation, 2009, 107: 971-980.
  * [60] V.N. Temlyakov, Greedy approximation, Cambridge University Press, 2011.
  * [61] T.E. Voth, M.A. Christon, Discretization errors associated with reproducing kernel methods: one-dimensional domains, Computer methods in applied mechanics and engineering, 2001, 190(18-19): 2429-2446.
  * [62] Y.B. Wang, T. Qian, Pseudohyperbolic distance and $n$-best rational approximation, Mathematical Methods for Applied Sciences, 2021.
  * [63] J.L. Walsh, The existence of rational functions of best approximation, Transactions of the American Mathematical Society, 1931, 33(3): 668-689.
  * [64] J.L. Walsh, Interpolation and approximation by rational functions in the complex plane, AMS, 1965.
  * [65] Y.B. Wang, T. Qian, Pseudo-hyperbolic distance and $n$-best rational approximation in $H^{2}$ space, Mathematical Methods in the Applied Sciences, 2021, 44(11): 8497-8504.
  * [66] F. Yang, M. Chen, J.C. Chen, Y. Zhang, T. Qian, Sparse representations of solutions to a class of random boundary value problems, arXiv:submit/4064143 [math.GM].
  * [67] D. X. Zhou, Capacity of reproducing kernel spaces in learning theory, IEEE Transactions on Information Theory, 2003, 49(7): 1743-1752.
  * [68] K.H. Zhu, Interpolating and recapturing in reproducing Hilbert spaces, Bull. Hong Kong Mathematical Society, 1997, 1: 21-33.
  * [69] K.H. Zhu, Operator theory in function spaces, American Mathematical Society, 2007, 138.
  * [70] S. Zaremba, Sur le calcul numerique des fonctions demandees dans le probleme de Dirichlet et le probleme hydrodynamique, Bulletin International de I Acadamic des Sciences de Cracovie, 1908: 125-195.