# Estimating Rényi’s $\alpha$-Cross-Entropies in a Matrix-Based Way

Isaac J. Sledge, _Member, IEEE_ and José C. Príncipe, _Life Fellow, IEEE_
Isaac J. Sledge is the Senior Machine Learning Scientist and Dr. Delores M.
Etter Assistant Secretary of the Navy Top Engineer with the Advanced Signal
Processing and Automated Target Recognition Branch, Naval Surface Warfare
Center, Panama City, FL, USA (email: isaac.j.sledge@navy.mil). He the Chief
Machine Learning Scientist with the Machine Intelligence Defense (MIND) lab at
the Naval Sea Systems Command.José C. Príncipe is a Distinguished Professor
and the Don D. and Ruth S. Eckis Chair with both the Department of Electrical
and Computer Engineering and the Department of Biomedical Engineering,
University of Florida, Gainesville, FL, USA (email: principe@ufl.edu). He is
the director of the Computational NeuroEngineering Laboratory (CNEL) at the
University of Florida.The work of the authors was funded by grants
N00014-19-WX-00636 (Marc Steinberg), N00014-21-WX-00476 (J. Tory Cobb),
N00014-21-WX-00525 (Thomas McKenna), and N00014-21-WX-01348 (Marc Steinberg)
from the US Office of Naval Research. The first author was also supported by
in-house laboratory independent research (ILIR) grant N00014-19-WX-00687
(Frank Crosby) from the US Office of Naval Research and a Naval Innovation in
Science and Engineering (NISE) grant from NAVSEA.

###### Abstract

Abstract—Conventional information-theoretic quantities assume access to
probability distributions. Estimating such distributions is not trivial. Here,
we consider function-based formulations of cross entropy that sidesteps this a
priori estimation requirement. We propose three measures of Rényi’s
$\alpha$-cross-entropies in the setting of reproducing-kernel Hilbert spaces.
Each measure has its appeals. We prove that we can estimate these measures in
an unbiased, non-parametric, and minimax-optimal way. We do this via sample-
constructed Gram matrices. This yields matrix-based estimators of Rényi’s
$\alpha$-cross-entropies. These estimators satisfy all of the axioms that
Rényi established for divergences. Our cross-entropies can thus be used for
assessing distributional differences. They are also appropriate for handling
high-dimensional distributions, since the convergence rate of our estimator is
independent of the sample dimensionality.

Python code for implementing these measures can be found at
https://github.com/isledge/MBRCE.

###### Index Terms:

Index Terms—Rényi’s cross entropy, estimator, reproducing-kernel Hilbert space
(RKHS), information-theoretic learning (ITL)

## 1\. Introduction

Information-theoretic quantities, such as cross entropy, are defined over
probability distributions [1, 2]. These distributions are often not known in
advance. When the only information available is from a finite set of samples,
then an appropriate distribution-estimation process must usually be employed
[3, 4, 5, 6].

Accurately estimating probability distributions is known to be highly
challenging. Improper estimation can greatly impede the use of the ensuing
information-theoretic quantities, especially for learning [7].

Our lab has shown that entropy-like and mutual-information-like quantities can
be estimated in a principled way without knowledge of the underlying
distributions [8, 9, 10]. This is done using the theory of functional
operators in reproducing-kernel Hilbert spaces.

Here, we consider an operator-based framework for estimating cross-entropy-
like quantities. It characterizes distributional overlap. Our framework avoids
the intermediate step of explicit estimating distributions [11, 12, 13].
Implicit distributional estimates are formed due to the nature of our
framework, though.

More specifically, we assume access to a sets of available samples drawn from
some unknown, arbitrary distributions. We also suppose that these underlying
distributions can be losslessly embedded in separable reproducing-kernel
Hilbert spaces [14] by mean-element mappings [15] (see Section 2.1). We use
either universal kernels [16] or characteristic kernels [17, 18] to specify
the reproducing-kernel Hilbert spaces. The resulting mappings are provably
guaranteed to be injective [19, 20]. That is, different distributions can be
distinguished within this Hilbert space (see Section 2.1). This beneficial
property permits defining measures of distance [21], like cross entropies,
using Hilbert-space operators. Semi-metric [22, 15] and metric [23, 24, 25,
26, 27] distances can also be considered between operators.

Direct access to the distributions is not required to assess distributional
differences via operator-based cross-entropy measures. By solving a
corresponding pre-image problem [28], the distributions can be recovered, at
least in a point-estimate sense [29, 30, 31]. Associated moments and interval
measures can also be retrieved [32, 33]. It is not our aim to do this here,
though. We simply specify a functional-operator-based cross-entropy measure.

We illustrate that the Hilbert-space operators can be empirically approximated
using representations that rely on finite sums of Dirac measures centered at
the samples [34]. These operator estimates are unbiased. They also converge to
the true operators at the minimax-optimal rate as a function of the sample
size. This convergence rate is independent of the sample dimensionality (see
Sections 3.2 and 4.2). Numerically computing these operators exactly is
difficult, however. They rely on access to an orthonormal basis for functional
spaces.

The operators that we define have multi-linear forms that coincide with
correlations of functions in a reproducing-kernel Hilbert space (See Section
2.2). We refer to these covariance operators as Gram operators. We show that
the Gram operators can be replaced with easily-computable Gram matrices (see
Sections 3.2 and 4.2). The Gram matrices are obtainable from the application
of either bounded, translation-invariant kernels [35] or non-translation-
invariant, strictly-positive-definite kernels [36] to a set of samples. The
Gram matrices posses spectral properties that are provably consistent with
those of the empirical Gram operators. Due to the form of our operator-based
divergence functionals, we still obtain minimax optimality when using Gram
matrices (see Sections 3.2 and 4.2). This optimality allows us to create
matrix-based notions of information-theoretic measures that act as though they
had direct access to the true, but unknown, probability distributions from
which the samples are drawn. This is true even for finite-sample cases. We
thus do not need access to the functional-space basis.

We apply this formulation to generalize Rényi’s $\alpha$-cross-entropies [37].
Rényi’s cross-entropies are very general and contain many popular measures as
limit cases [38]. We develop three operator-based cross-entropy measures (see
Sections 3.1 and 4.1). They are parent quantities to notions of marginal,
joint, and conditional Rényi’s $\alpha$-entropies [39] that we previously
defined in an operator-theoretic sense [9, 10] (see Appendix A.1). The use of
kernels that are additionally infinitely-divisible [40] is needed to recover
these $\alpha$-entropies. This is not a restrictive condition. It requires
only a simple modification of the Gram-matrix entries.

We prove that the corresponding matrix-based Rényi’s $\alpha$-cross-entropies
satisfy the fundamental axioms that Rényi originally considered for
divergences (see Appendix A.1). They also have other beneficial properties
(see Appendix A.2). These properties are present regardless of the sample-set
size and dimensionality. Our matrix-based functionals thus can behave as
divergences even for small sample sets drawn from high-dimensional
distributions. However, a sufficient number of samples is still needed to
ensure that the matrix-based-cross-entropy magnitudes are close enough to
those from the operator-based cross-entropies.

Throughout our presentation of these measures, we assume that our sample sets
are vectorial in nature before being converted to Gram matrices. This is done
for the sake of presentation convenience. Certain versions of our
$\alpha$-cross-entropies can act as divergences for kernel-transformed
representations that are inherently matrix-based, though. Examples include
adjacency matrices of undirected graphs along with square similarity and
dissimilarity matrices. Such data can be easily converted to Gram matrices via
the application of kernels to the matrix entries. This assumes axisymmetry of
the underlying matrices, though. Likewise, it assumes that the matrices are
square.

After defining each measure, we assess their empirical properties (see
Sections 3.3 and 4.3). We show that these measures empirically converge at a
rate that is independent of the sample dimensionality, just as our theory
predicts. We also illustrate the importance of the kernel choice on the
resulting cross entropies.

Our work here represents the first instance of operator-theoretic cross-
entropy. We show that there are some connections with our measures and those
from quantum information theory (see Section 5). The interpretation and
utility of our measures are very different, though. These distinctions permit
investigators to use our estimators to assess population cross-entropies,
optimize information-theoretic cost functions, and more. The quantum variants
cannot be employed in the same manner. They are suited only for assessing
information-theoretic quantities of random matrices. Random matrices have
characteristics that often differ greatly from those of the Gram matrices.

## 2\. Preliminaries

The matrix-based cross-entropies that we define assess differences between
probability distributions without needing direct access to them. In this
section, we outline the theory that makes this possible, which are operators
posed in reproducing-kernel Hilbert spaces. Such a functional space is
appropriate for processing probability distributions. This is because
probability distributions are functions.

We start by recalling the notion of the mean element for univariate random
variables (see Definition 2.1). Mean-element mappings are a principled way of
characterizing probability measures. These maps losslessly convert probability
measures into points within a reproducing-kernel Hilbert space of functions
(see Proposition 2.1). Provided that the mapping is injective, then the
Hilbert space separates points. Unique distributions are assigned to unique
points (see Proposition 2.3). Distance measures can hence be defined within
this Hilbert space to assess how much one probability measure is related to
another (see Proposition 2.2). Direct access to the underlying distributions
is not required to do this for injective mappings (see Proposition 2.4). This
makes the mean-element map amenable for many applications where the true form
of the distributions is either unknown or difficult to estimate in a non-
parametric way. Efficiently assessing cross entropy is one such application.

Following the univariate case, we consider the bivarate case. We define the
bivariate mean-element map (see Definition 2.2) on tensor-product Hilbert
spaces (see Proposition 2.5). These spaces permit studying the interaction of
samples from different distributions in a reproducing-kernel Hilbert space
(see Proposition 2.6). For the univariate case, injectivity of the mean-
element map is guaranteed for characteristic kernels. In the bivariate case,
the matter is more complicated. Products of characteristic kernels are not
necessarily characteristic (see Definitions 2.3–2.4 and Proposition 2.7).
Injectivity can be lost. We thus require strict positive definiteness on the
product kernel, which occurs whenever the kernel pairs are universal (see
Proposition 2.8). Many popular translation-varying and translation-invariant
kernels are universal. Consequently, kernel universality is not a very
restrictive assumption in practice.

We illustrate that the mean-element maps are used in the construction of
covariance operators on Hilbert spaces and tensor-product Hilbert spaces [41]
(see Definitions 2.5–2.6). We refer to these covariance operators as Gram
operators. Bilinear forms associated with the Gram operators correlate
functions, and thus probability distributions, in reproducing-kernel Hilbert
spaces. In the next section, we show that the operators have empirical
versions that can be constructed via distribution samples. The empirical
versions, which are Gram matrices, quantify relationships between sample pairs
and hence the intra-distributional organization of samples. That is, they
provide insights into the distribution shape, which is needed for assessing
cross entropy. Covariance operators can also be defined to quantify inter-
distributional shifts and therefore how much the distributions overlap (see
Definition 2.7). This too helps with assessing cross entropy in certain cases.

An important result established in this section is that any process applied to
the trace of the Gram operators will extend to the mean-element maps (see
Propositions 2.11 and 2.14). That is, the embeddings of probability
distributions will be transformed.

This is a powerful property that we exploit in our cross-entropy-like
criteria. We define these criteria in terms of Gram-operator trace. These
criteria provably satisfies foundational divergence axioms. They hence
quantify distributional differences. Due to the mean-element-map relationship,
access to the underlying probability distributions is not required when acting
on the Gram-operator traces and their empirical estimates. This property
permits investigators to use our information-theoretic criteria as efficient
drop-in estimators for a variety of problems. No parametric assumptions are
made about the distributions for appropriate kernel choices. Arbitrary
distributions can hence be characterized well and their differences assessed
by our cross-entropy-like criteria.

Our formulation of the covariance and cross-covariance operators, and hence
their traces, relies on access to a basis for the reproducing-kernel Hilbert
space (See Definitions 2.5–2.6 and Proposition 2.13). Any orthonormal set is
provably a basis (see Proposition 2.10). The trace is independent of this
basis choice. However, it can be difficult to practically specify any such set
depending on the selected kernel. The basis may have countably-infinite
components, in some cases (see Propositions 2.9 and 2.12). In the next
section, we sidestep this issue. We rely on the connections between the
covariance operators and the mean-element maps to show that, like the latter
[42], the former can be estimated in an unbiased, minimax-optimal, non-
parametric manner from the distributional samples. We also have behaved
spectral-difference bounds for eigenvalues of the Gram operators and empirical
Gram matrices. Given enough samples, the Gram matrices resemble the Gram
operators. Our formulation is thus well motivated.

Throughout, we assume access to a vector-based realization of matrices. This
enables us to present the background theory in an easy-to-understand manner.
This assumption is not practically required, though. It is merely for the sake
of convenience. Only in a few instances are vector samples explicitly
required. We discuss these cases in the next section (see Section 3).

## 2.1. Distribution-Embedding Operators

Univariate Operator Embeddings. We first recall some important properties of
working with marginal probability measures in functional spaces that will be
helpful throughout.

  * Definition 2.1: Univariate Mean-Element Map. Let $\mathcal{H}_{\kappa^{\prime}}$ be a separable, reproducing-kernel Hilbert space [43] with a continuous reproducing kernel defined on the sample space $\kappa^{\prime}\\!:\\!\mathcal{S}\\!\times\\!\mathcal{S}\\!\to\\!\mathbb{R}_{0,+}$. Let $\varphi:\mathcal{S}\\!\to\\!\mathcal{H}_{\kappa^{\prime}}$
be a measurable function which obeys
$\kappa^{\prime}(s,s^{\prime})\\!=\\!\langle\varphi(s),\varphi(s^{\prime})\rangle_{\mathcal{H}_{\kappa^{\prime}}}$.
For a non-negative, non-zero-everywhere,
normalized kernel that is integrable, the mean element,
$\mu_{p_{\mathcal{S}}}$, is defined via the Bochner integral

$\mathbb{E}_{(s,s^{\prime})\sim
p_{\mathcal{S}}}[\kappa^{\prime}(s,s^{\prime})]=\\!\int\\!\\!\\!\\!\int_{\mathcal{S}}\langle\varphi(s),\varphi(s^{\prime})\rangle_{\mathcal{H}_{\kappa^{\prime}}}dp_{\mathcal{S}}(s,s^{\prime})$

where $p_{\mathcal{S}}\\!\in\\!\mathcal{M}^{1}_{+}(\mathcal{S})$ is a Borel
probability measure over the topological sample space $\mathcal{S}$.

The mean element $\mu_{p_{\mathcal{S}}}\\!=\\!\mathbb{E}_{(s,s^{\prime})\sim
p_{\mathcal{S}}}[\kappa^{\prime}(s,s^{\prime})]$ can be viewed as an embedding
of the measure in $\mathcal{H}_{\kappa^{\prime}}$. That is, for
the set of all Borel probability measures,
$\mu_{\kappa^{\prime}}\\!:\\!\mathcal{M}^{1}_{+}(\mathcal{S})\\!\to\\!\mathcal{H}_{\kappa^{\prime}}$,
with $\mu_{\kappa^{\prime}}(p_{\mathcal{S}})\\!=\\!\mu_{p_{\mathcal{S}}}$.
Unique distributions can
be mapped to unique points in the reproducing-kernel Hilbert space, for
appropriate kernel choices. This permits comparing distributions in this
functional space.

Next, we outline the condition under which the embedding
$\mu_{\kappa^{\prime}}(p_{\mathcal{S}})$ exists and belongs to
$\mathcal{H}_{\kappa^{\prime}}$.

  * Proposition 2.1: Univariate Riesz Representation Property. For a separable reproducing-kernel Hilbert space, with a continuous reproducing kernel defined on the samples, let $\kappa^{\prime}(s,s^{\prime})\\!=\\!\langle\varphi(s),\varphi(s^{\prime})\rangle$, where $\varphi$
is measurable. For a function $f\\!\in\\!\mathcal{H}_{\kappa^{\prime}}$, if
$\mathbb{E}_{(s,s^{\prime})\sim
p_{\mathcal{S}}}[\kappa^{\prime}(s,s^{\prime})^{1/2}]\\!\in\\!\mathbb{R}_{0,+}$,
then we have that
$\mu_{p_{\mathcal{S}}}\\!\in\\!\mathcal{H}_{\kappa^{\prime}}$ and
$\mathbb{E}_{p_{\mathcal{S}}}[f(s)]\\!=\\!\langle
f,\mu_{p_{\mathcal{S}}}\rangle_{\mathcal{H}_{\kappa^{\prime}}}$.

From the equality $\mathbb{E}_{p_{\mathcal{S}}}[f(s)]\\!=\\!\langle
f,\mu_{p_{\mathcal{S}}}\rangle_{\mathcal{H}_{\kappa^{\prime}}}$, we
simultaneously obtain the reproducing property of the expect-
ation operation in a reproducing-kernel Hilbert space. That is, the functional
expectation can be performed, with respect to the distribution, via an inner
product of the function $f\\!\in\\!\mathcal{H}_{\kappa^{\prime}}$ and the
embedding $\mu_{p_{\mathcal{S}}}\\!\in\\!\mathcal{H}_{\kappa^{\prime}}$.

There are two additional properties worth mentioning.

  * Proposition 2.2: Distribution Distinguishability Property. Let $\mathcal{H}_{\kappa^{\prime}}$ be a separable reproducing kernel Hilbert space with a continuous reproducing kernel defined on state-action pairs such that $\kappa^{\prime}(s,s^{\prime})\\!=\\!\langle\varphi(s),\varphi(s^{\prime})\rangle$, where $\varphi$ is measurable. Assume that this Hilbert space is dense on the space of all continuous, bounded functions. We have that

    * (i) The mean mapping $p_{\mathcal{S}}\\!\mapsto\\!\mu_{p_{\mathcal{S}}}$ is injective for all $p_{\mathcal{S}}\\!\in\\!\mathcal{M}^{1}_{+}(\mathcal{S})$, where $\mathcal{M}^{1}_{+}(\mathcal{S})$ is the set of all Borel
probability measures on the topological sample space.

    * (ii) $\langle\;\\!\mu_{p_{\mathcal{S}}}\\!-\\!n^{-1}\sum_{i=1}^{n}\kappa^{\prime}(s_{i},s_{i}^{\prime}),\cdot\rangle\\!\leq\\!2n^{-1}\mathbb{E}_{s_{i}}\mathbb{E}_{\sigma_{i}}[\,\textnormal{sup}_{\|f\|_{\mathcal{H}_{\kappa^{\prime}}}\leq 1}\,|\;\\!\\!\sum_{i=1}^{m}\sigma_{i}f(s_{i})|]\\!+\\!c_{1}$ is satisfied, with
probability at least $1\\!-\\!\delta$, whenever
$\|f\|_{\infty}\\!\leq\\!c_{2}$, $f\\!\in\\!\mathcal{H}_{\kappa^{\prime}}$,
where $c_{1}\\!=\\!c_{2}(-\textnormal{log}(\delta)/n)^{1/2}$.

The first property implies that we can define distance measures and distance
metrics between probability distributions in reproducing-kernel Hilbert
spaces. We will define three such measures in the next section. The second
property imparts that we do not need access to these distributions in order to
estimate the distance-measure magnitudes. This is possible provided that the
Rademacher average is well behaved. The second property is crucial for our
purposes. Without it, the theory we present in the next section would not be
possible.

Injectivity of the mean-element mappings underlies much of our ability to
operate on probability measures in reproducing-kernel Hilbert spaces. We
therefore characterize one of its properties.

  * Proposition 2.3: Distributional Equality. Let $p_{\mathcal{S}},q_{\mathcal{S}}\\!\in\\!\mathcal{M}^{1}_{+}(\mathcal{S})$. If the mapping $p_{\mathcal{S}}\\!\mapsto\\!\mu_{p_{\mathcal{S}}}$ is injective for a
separable, reproducing- kernel Hilbert space, then $\mathbb{E}_{s\sim
p_{\mathcal{S}}}[f(s)]\\!=\\!\mathbb{E}_{s\sim q_{\mathcal{S}}}[f(s)]$,
$\forall f\\!\in\\!\mathcal{H}_{\kappa^{\prime}}$, where

$\mathbb{E}_{s\sim
q_{\mathcal{S}}}[f(s)]=\\!\int_{\mathcal{S}}f(s)dq_{\mathcal{S}}(s).$

That is, the probability measures are equivalent,
$p_{\mathcal{S}}\\!=\\!q_{\mathcal{S}}$.

Thus, $\langle
p_{\mathcal{S}}\\!-\\!q_{\mathcal{S}},\cdot\rangle_{\mathcal{H}_{\kappa^{\prime}}}\\!=\\!0$
if and only if the measures are equal,
$p_{\mathcal{S}}\\!=\\!q_{\mathcal{S}}$. Kernels underlying injective mappings
ensure that no information is lost when mapping distributions into a
reproducing-kernel Hilbert space. Absent this guarantee, it would not be
appropriate to pose our function cross-entropies in such a functional space.

Several kernels, like the isotropic Gaussian function, yield injective mean
mappings. Kernels of this type are referred to as characteristic. The
following result characterizes when this occurs for arbitrary kernels.

  * Proposition 2.4: Characteristic Kernel Condition. Let $\phi(\omega^{\prime})$ be a continuous function on $\mathbb{R}^{d}$. Let a kernel obey $\kappa^{\prime}(s,s^{\prime})\\!=\\!\langle\phi(s),\phi(s^{\prime})\rangle_{\mathcal{H}_{\kappa^{\prime}}}$. If, for any $\xi\\!\in\\!\mathbb{R}^{d}$ and $\beta\\!\in\\!\mathbb{R}$, there exists $\tau,\tau_{0}\\!\in\\!\mathbb{R}$, such that

$\int_{\mathbb{R}^{d}}\phi^{\prime}(\tau(\omega^{\prime}\\!+\\!\xi))^{2}\phi^{\prime}(\omega^{\prime})^{-1}d\omega^{\prime}\\!<\\!\infty,\;\;\;\phi^{\prime}(\omega^{\prime})=\\!\int_{-\infty}^{\infty}\phi(\omega^{\prime})e^{-2\pi
i(s)\beta}ds,$

for all $\tau_{0}\\!<\\!\tau$, then $\mathcal{H}_{\kappa^{\prime}}$ is dense
in $\mathbb{L}^{2}(p^{\prime}_{\mathcal{S}})$ where
$p^{\prime}_{\mathcal{S}}\\!\in\\!\mathcal{M}^{1}_{+}(\mathbb{R}^{d})$. Thus,
$\kappa^{\prime}$ is a characteristic kernel with respect
to the Borel sigma-field and defines an injective mean-element mapping.

We typically take the sample space to be some multi-dimensional real space. It
can also be the space of real-valued matrices. In these settings, the
characteristic kernels underlying the injective mean mappings are
generalizations of characteristic functions. That is, they generalize the
property that characteristic functions uniquely determine a Borel probability
measure on the reals.

Bivariate Operator Embeddings. We now extend the notion of embedding
probability measures to the bivariate case so that they can be natively
handled in a quotient Hilbert space.

We first show that it is possible to construct a quotient Hilbert space with a
reproducing kernel.

  * Proposition 2.5: Quotient Hilbert Space Existence. Let $\mathcal{H}_{\kappa_{1}^{\prime}},\mathcal{H}_{\kappa_{2}^{\prime}}$ be Hilbert spaces with reproducing kernels $\kappa_{1}^{\prime},\kappa_{2}^{\prime}$. The tensor product of the vector spaces admits a functional completion $\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}$ which is a quotient reproducing-kernel Hilbert space with a product reproducing kernel $(\kappa^{\prime}_{1}\otimes\kappa_{2}^{\prime})((s,s^{\prime}),(a,a^{\prime}))=$
$\kappa_{1}^{\prime}(s,a)\otimes\kappa_{2}^{\prime}(s^{\prime},a^{\prime})$,
where $s,s^{\prime}\\!\in\\!\mathcal{S}$ and
$a,a^{\prime}\\!\in\\!\mathcal{A}$. Here, $\mathcal{S},\mathcal{A}$ are sample
spaces.

Moreover, we can define a product measure,
$\rho_{\mathcal{S}\times\mathcal{A}}$, on the sigma-algebras associated with
$\mathcal{H}_{\kappa_{1}^{\prime}},\mathcal{H}_{\kappa_{2}^{\prime}}$. The
existence of this product measure, for the sigma-finite case, can be shown by
Carathéodory’s extension theorem.

As before, we can losslessly project product measures into quotient
reproducing-kernel Hilbert spaces via an expectation operation.

  * Definition 2.2: Bivariate Mean-Element Map. Let $\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}$ be a separable, reproducing-kernel Hilbert
space. For a product measurable function
$\varphi^{\otimes}\\!:\\!\mathcal{S}\\!\times\\!\mathcal{A}\\!\to\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}$
which obeys the following condition
$(\kappa^{\prime}_{1}\otimes\kappa_{2}^{\prime})((s,a),\cdot)\\!=\\!\langle\varphi^{\otimes}(s,a),\cdot\rangle_{\mathcal{H}_{\kappa^{\prime}_{1}}\otimes\mathcal{H}_{\kappa^{\prime}_{2}}}$,
we have, for a non- negative, non-zero-everywhere kernel, that

$\mathbb{E}_{((s,a),(s^{\prime}a^{\prime}))\sim\rho_{\mathcal{S}\times\mathcal{A}}}[(\kappa^{\prime}_{1}\otimes\kappa_{2}^{\prime})((s,a),(s^{\prime}a^{\prime}))]=\\!\int\\!\\!\\!\\!\int_{\mathcal{S}\times\mathcal{A}}\langle\varphi^{\otimes}(s,a),\varphi^{\otimes}(s^{\prime},a^{\prime})\rangle_{\mathcal{H}_{\kappa^{\prime}_{1}}\otimes\mathcal{H}_{\kappa^{\prime}_{2}}}d\rho_{\mathcal{S}\times\mathcal{A}}((s,a),(s^{\prime},a^{\prime}))$

is the product mean element
$\mu_{\rho_{\mathcal{S}\times\mathcal{A}}}\\!=\\!\mathbb{E}_{((s,a),\cdot)\sim\rho_{\mathcal{S}\times\mathcal{A}}}[\kappa^{\prime}_{1,2}((s,a),\cdot)]$,
where
$\rho_{\mathcal{S}\times\mathcal{A}}\\!\in\\!\mathcal{M}^{1}_{+}(\mathcal{S}\\!\times\\!\mathcal{A})$
is a Borel
probability measure over the product topological space.

However, we must impose $\mathbb{E}_{(s,s^{\prime})\sim
p_{\mathcal{S}}}[\kappa_{1}^{\prime}(s,s^{\prime})]\\!\in\\!\mathbb{R}_{0,+}$
and $\mathbb{E}_{(a,a^{\prime})\sim
q_{\mathcal{A}}}[\kappa_{2}^{\prime}(a,a^{\prime})]\\!\in\\!\mathbb{R}_{0,+}$,
where the Borel mea-
sures $p_{\mathcal{S}}\\!\in\\!\mathcal{M}^{1}_{+}(\mathcal{S})$ and
$q_{\mathcal{A}}\\!\in\\!\mathcal{M}^{1}_{+}(\mathcal{A})$, so that the
product kernel is Bochner $\rho_{\mathcal{S}\times\mathcal{A}}$-integrable.

The corresponding embedding
$\mu_{\kappa^{\prime}_{1}\otimes\,\kappa^{\prime}_{2}}(\rho_{\mathcal{S}\times\mathcal{A}})\\!=\\!\mu_{\rho_{\mathcal{S}\times\mathcal{A}}}$
naturally exists in
$\mathcal{H}_{\kappa^{\prime}_{1}}\\!\otimes\mathcal{H}_{\kappa^{\prime}_{2}}$.

  * Proposition 2.6: Bivariate Riesz Representation Property. Let $\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}$ be a separable, quotient
reproducing-kernel Hilbert space, with a continuous reproducing kernel. For
$f\\!\in\\!\mathcal{H}_{\kappa_{1}^{\prime}}$ and
$g\\!\in\\!\mathcal{H}_{\kappa_{2}^{\prime}}$, if
$\mathbb{E}_{((s,a),\cdot)\sim\rho_{\mathcal{S}\times\mathcal{A}}}[(\kappa^{\prime}_{1}\otimes\kappa_{2}^{\prime})((s,a),\cdot)^{1/2}]\\!\in\\!\mathbb{R}_{0,+}$,
then we have that
$\mu_{\rho_{\mathcal{S}\times\mathcal{A}}}\\!\\!\in\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}$
and hence that
$\mathbb{E}_{\rho_{\mathcal{S}\times\mathcal{A}}}[f(s)g(a)]\\!=\\!\langle
f\otimes
g,\mu_{\rho_{\mathcal{S}\times\mathcal{A}}}\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\mathcal{H}_{\kappa_{2}^{\prime}}}$.

Product characteristic kernels also exist for injective mappings. The quotient
reproducing-kernel Hilbert space is hence expressive enough to distinguish
between distributions from their embeddings.

  * Definition 2.3: Product Kernel Characteristicness. A positive-definite product kernel $\kappa_{1}^{\prime}\otimes\kappa_{2}^{\prime}$ is characteristic to a set of probability measures if $\mathcal{M}_{+}^{1}(\mathcal{S}\\!\times\\!\mathcal{A})\\!\to\\!\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\mathcal{H}_{\kappa_{2}^{\prime}}\\!:\\!\rho_{\mathcal{S}\times\mathcal{A}}\\!\mapsto\\!\mu_{\rho_{\mathcal{S}\times\mathcal{A}}}$ is injective.

It has been shown that injective mean-element mappings are exactly the same as
moment-generating functions of samples on either the marginal space or the
quotient space. The mean-element map contains information of all moments. This
property lends credence to the richness of a reproducing-kernel-Hilbert-space
representation in losslessly describing the probability distributions and
distinguishing them via their embeddings. No parametric assumptions are made
for appropriate kernel choices. Arbitrary distributions can hence be modeled
and processed when relying on constructs that employ the mean-element
mappings.

It is important to observe that the product of two characteristic kernels,
$\kappa_{1}^{\prime},\kappa_{2}^{\prime}$, does not necessarily yield a
characteristic product kernel $\kappa_{1}^{\prime}\otimes\kappa_{2}^{\prime}$.
Rather, the product may only belong to a weaker class of kernels that do not
guarantee injectivity [44], which we outline below.

  * Definition 2.4: Product Kernel $i$-Characteristicness. Let $\kappa_{1}^{\prime}\\!:\\!\mathcal{S}\\!\times\\!\mathcal{S}\\!\to\\!\mathbb{R}_{0,+}$ and $\kappa_{2}^{\prime}\\!:\\!\mathcal{A}\\!\times\\!\mathcal{A}\\!\to\\!\mathbb{R}_{0,+}$ be
bounded kernels on topological spaces $\mathcal{S}$ and $\mathcal{A}$,
respectively. Let
$\mathcal{J}\\!\subseteq\\!\mathbb{L}_{2}(\mathcal{S}\\!\times\\!\mathcal{A})$
such that $0\\!\in\\!\mathcal{J}$. The product kernel
$\kappa_{1}^{\prime}\\!\otimes\kappa_{2}^{\prime}$ is
$\mathcal{J}$-integrally-strictly-positive-definite if and only if

$\int\\!\\!\\!\\!\int_{\mathcal{S}\times\mathcal{A}}(\kappa_{1}^{\prime}\\!\otimes\kappa_{2}^{\prime})((s,a),(s^{\prime},a^{\prime}))df(s,a)df(s^{\prime},a^{\prime})\\!>\\!0,\;\;\forall
f\\!\in\\!\mathcal{J}\backslash\\{0\\}.$

If
$\mathcal{J}\\!=\\!\\{\rho_{\mathcal{S}\times\mathcal{A}}-p_{\mathcal{S}}\otimes
p_{\mathcal{A}}:\rho_{\mathcal{S}\times\mathcal{A}}\\!\in\\!\mathcal{M}_{1}^{+}(\mathcal{S}\\!\times\\!\mathcal{A})\\}$,
then the tensor-product kernel is said to be $i$-characteristic.

If $\mathcal{S}$ and $\mathcal{A}$ are second-countable, then, for
characteristic kernels $\kappa_{1}^{\prime},\kappa_{2}^{\prime}$, their tensor
product $\kappa_{1}^{\prime}\otimes\kappa_{2}^{\prime}$ is $i$-characteristic.
The converse is true, provided that the topological spaces are additionally
Hausdorff.

  * Proposition 2.7: Product Kernel Non-Characteristicness. Let $\kappa_{1}^{\prime}\otimes\kappa_{2}^{\prime}$ be an arbitrary $i$-characteristic
tensor-product kernel. This product kernel may not be characteristic, even if
$\kappa_{1}^{\prime},\kappa_{2}^{\prime}$ are, individually, characteristic on
their respective topological spaces $\mathcal{S}$, $\mathcal{A}$.

For injectivity to be present, we it is helpful to have that the product
kernel be $c_{0}$-universal. This is possible if both
$\kappa_{1}^{\prime},\kappa_{2}^{\prime}$ are universal kernels. Translation
invariance of the kernels is not required [36] if we replace integrally-
strictly positive definiteness with strict positive definiteness. The
isotropic-Gaussian kernel is universal, as are others like the exponential-
inner-product kernel.

  * Proposition 2.8: Product Kernel Strict Positive Definiteness. Let $\kappa_{1}^{\prime}\\!:\\!\mathcal{S}\\!\times\\!\mathcal{S}\\!\to\\!\mathbb{R}_{0,+}$ and $\kappa_{2}^{\prime}\\!:\\!\mathcal{A}\\!\times\\!\mathcal{A}\\!\to\\!\mathbb{R}_{0,+}$
be bounded, $c_{0}$-kernels on locally compact Polish spaces $\mathcal{S}$ and
$\mathcal{A}$, respectively. Let
$\mathcal{J}\\!\subseteq\\!\mathcal{M}_{b}(\mathcal{S}\\!\times\\!\mathcal{A})$,
such that $0\\!\in\\!\mathcal{J}$, be a subset of the space of all finite
signed measures on the product topological space. The kernel
$\kappa_{1}^{\prime}\\!\otimes\kappa_{2}^{\prime}$ is $\mathcal{J}$-strictly-
positive-definite if and only if

$\int\\!\\!\\!\\!\int_{\mathcal{S}\times\mathcal{A}}(\kappa_{1}^{\prime}\\!\otimes\kappa_{2}^{\prime})((s,a),(s^{\prime},a^{\prime}))d\rho_{\mathcal{S}\times\mathcal{A}}((s,a),(s^{\prime},a^{\prime}))\\!>\\!0,\;\;\forall\rho_{\mathcal{S}\times\mathcal{A}}\\!\in\\!\mathcal{J}\backslash\\{0\\}.$

If $\mathcal{J}\\!=\\!\mathcal{M}_{b}(\mathcal{S}\\!\times\\!\mathcal{A})$,
the space of all finite signed measures, then the tensor-product kernel is
said to be $c_{0}$-uni-
versal and hence characteristic.

Without $c_{0}$-universality, the covariance operators that we define would
not necessary correspond to transformations of embedded probability measures.
Throughout, we refer to $c_{0}$-universal kernels as just universal kernels.

## 2.2. Gram Operators

Univariate Operators. We now implicitly use the mean-element map to define the
notion of a covariance operator for both univariate and bivariate-product
random variables. We refer to this covariance operator as a Gram operator
since it behaves akin to a conventional Gramian, but in a functional sense. We
will use this operator to mainly quantify marginal distribution shape.

We start with the Gram operator
$\kappa\\!:\\!\mathcal{H}_{\kappa^{\prime}}\\!\to\\!\mathcal{H}_{\kappa^{\prime}}$
for the univariate product case. We also define its trace, as our notions of
cross-entropy rely on traces of univariate Gram operators.

  * Definition 2.5: Univariate Gram Operator. For a separable reproducing-kernel Hilbert space, with a continuous, characteristic reproducing kernel, let $\kappa^{\prime}(s,s^{\prime})\\!=\\!\langle\varphi(s),\varphi(s^{\prime})\rangle_{\mathcal{H}_{\kappa^{\prime}}}$, where $\varphi$ is measurable. For $f,g\\!\in\\!\mathcal{H}_{\kappa^{\prime}}$, the univariate Gram operator $\kappa\\!:\\!\mathcal{H}_{\kappa^{\prime}}\\!\to\\!\mathcal{H}_{\kappa^{\prime}}$ is given by the symmetric bilinear form

$\mathcal{K}(f,g)=\langle f,\kappa
g\rangle=\\!\int\\!\\!\\!\\!\int_{\mathcal{S}}\\!\Bigg{(}\\!\langle
f,\varphi(s)\rangle_{\mathcal{H}_{\kappa^{\prime}}}\langle\varphi(s^{\prime}),g\rangle_{\mathcal{H}_{\kappa^{\prime}}}\\!\\!\Bigg{)}dp_{\mathcal{S}}(s,s^{\prime})$

where $f(s)\\!=\\!\langle f,\varphi(s)\rangle_{\mathcal{H}_{\kappa^{\prime}}}$
and thus $\mathcal{K}(f,g)\\!=\\!\mathbb{E}_{(s,s^{\prime})\sim
p_{\mathcal{S}}}[f(s)g(s^{\prime})]$, for $s,s^{\prime}\\!\in\\!\mathcal{S}$.

We have that the trace of this operator is

$\textnormal{tr}(\kappa)=\sum_{j=1}^{h}\int\\!\\!\\!\\!\int_{\mathcal{S}}\\!\Bigg{(}\\!\langle\eta_{j},\varphi(s)\rangle_{\mathcal{H}_{\kappa^{\prime}}}\langle\varphi(s^{\prime}),\eta_{j}\rangle_{\mathcal{H}_{\kappa^{\prime}}}\\!\\!\Bigg{)}dp_{\mathcal{S}}(s,s^{\prime}),$

which follows from
$\textnormal{tr}(\kappa)\\!=\\!\sum_{j=1}^{h}\mathcal{K}(\eta_{j},\cdot)$.
Here, $\\{\eta_{j}\\}_{j=1}^{h}\\!\subset\\!\mathcal{H}_{\kappa^{\prime}}$ is
a complete orthonormal basis for $\mathcal{H}_{\kappa^{\prime}}$.

From this definition, we can view the Gram operator as an image point of an
embedding of the probability measure from which samples are drawn. This
operator defines a bilinear form $\mathcal{K}(f,g)\\!=\\!\langle f,\kappa
g\rangle$, with $f,g\\!\in\\!\mathcal{H}_{\kappa^{\prime}}$, that corr-
esponds to the correlation of functions that belong to the reproducing-kernel
Hilbert space $\mathcal{H}_{\kappa^{\prime}}$ induced by $\kappa^{\prime}$.

We have that $\kappa$ is a compact, trace-class operator. The corresponding
trace is finite. The trace is also independent of the choice of basis, as any
basis of a separable reproducing-kernel Hilbert space contains an orthonormal
set.

  * Proposition 2.9: Hilbert Space Separability. Let $\kappa^{\prime}$ be a reproducing kernel defined for a univariate random variable. A reproducing kernel Hilbert space $\mathcal{H}_{\kappa^{\prime}}$ is separable if and only if its dimension is, at most, countable. That Hilbert space’s dimensionality is the cardinality of its basis.

  * Proposition 2.10: Hilbert Space Basis Orthonormality. Let $\mathcal{H}_{\kappa^{\prime}}$ be a separable reproducing-kernel Hilbert
space induced by a continuous reproducing kernel $\kappa^{\prime}$. If
$\\{\eta_{j}\\}_{j=1}^{h}\\!\subset\\!\mathcal{H}_{\kappa^{\prime}}$ is an
orthonormal set, then there is a
basis that contains it.

Although a basis provably exists, specifying it can be difficult. The trace
operation assumes access to it. We therefore consider an approximation
strategy in the next section.

It is important to note that if there is no reproducing kernel, then the space
is non-separable. Non-separability complicates the Bochner integrability
underlying the mean-element maps [45]. Without the ability to losslessly embed
probability distributions in a functional space, we will not have guarantees
that our cross-entropy measures will behave well.

An important result is that the trace of the univariate Gram operator can be
expressed as the norm of the univariate mean-element map. Other
$\alpha$-traces are also related, though their relationship with the mean-
element mapping is more complicated.

  * Proposition 2.11: Univariate Operator-Map Equivalence. Let $\mathcal{H}_{\kappa^{\prime}}$ be a separable, reproducing-kernel Hilbert space with a continuous, characteristic reproducing kernel $\kappa^{\prime}(s,s^{\prime})\\!=\\!\langle\varphi(s),\varphi(s^{\prime})\rangle_{\mathcal{H}_{\kappa^{\prime}}}$, for a measurable $\varphi$. We have that the relationship between the 2-trace of the univariate Gram operator, $\kappa$, and the univariate mean-element map $\mu_{\rho_{\mathcal{S}}}$ is

$\int\\!\\!\\!\\!\int_{\mathcal{S}}\\!\Bigg{(}\\!\langle\varphi(s),\kappa\varphi(s^{\prime})\rangle_{\mathcal{H}_{\kappa^{\prime}}}\\!\\!\Bigg{)}dp_{\mathcal{S}}(s,s^{\prime})=\langle\hskip
0.28436pt\mu_{\rho_{\mathcal{S}}},\mu_{\rho_{\mathcal{S}}}\rangle_{\mathcal{H}_{\kappa^{\prime}\kappa^{\prime}}}.$

Here, $\mathcal{H}_{\kappa^{\prime}\kappa^{\prime}}$ denotes the reproducing-
kernel Hilbert space induced by the kernel
$\kappa^{\prime}(s,s^{\prime})\kappa^{\prime}(s,s^{\prime})$.

This finding demonstrates that, in working with univariate Gram operators, we
are implicitly dealing with the univariate mean-element maps. Any measure
defined in terms of these Gram operators will thus transform embeddings of the
probability measures in a reproducing-kernel Hilbert space.

Bivariate Operators. We now define an operator
$\gamma\\!:\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}\\!\to\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}$
for bivariate product random variables, where
$\mathcal{H}_{\kappa_{1}^{\prime}}$ and $\mathcal{H}_{\kappa_{2}^{\prime}}$
are two reproducing-kernel Hilbert spaces. This extends the notion of the Gram
operator to a case that will prove useful when handling pairs of univariate
operators.

  * Definition 2.6: Bivariate Gram Operator. For separable reproducing-kernel Hilbert spaces $\mathcal{H}_{\kappa_{1}^{\prime}}$ and $\mathcal{H}_{\kappa_{2}^{\prime}}$, with continuous reproducing kernels defined on state-action pairs, let $\kappa^{\prime}_{1}(s,s^{\prime})\\!=\\!\langle\varphi(s),\varphi(s^{\prime})\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}}$ and
$\kappa^{\prime}_{2}(a,a^{\prime})\\!=\\!\langle\psi(a),\psi(a^{\prime})\rangle_{\mathcal{H}_{\kappa_{2}^{\prime}}}$,
where $\varphi\\!:\\!\mathcal{S}\\!\to\\!\mathcal{H}_{\kappa_{1}^{\prime}}$
and $\psi\\!:\\!\mathcal{A}\\!\to\\!\mathcal{H}_{\kappa_{2}^{\prime}}$ are
measurable. Assume that the product
kernel is characteristic. We define the bivariate Gram operator $\gamma$,
which is over the quotient, completed Hilbert space, via the symmetric
bilinear form

$\mathcal{G}(f,g)=\langle f,\gamma
g\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\,\mathcal{H}_{\kappa_{2}^{\prime}}}\\!\\!\,=\\!\int\\!\\!\\!\\!\int_{\mathcal{S}\times\mathcal{A}}\Bigg{(}\\!\langle
f,\varphi^{\otimes}((s,a),\cdot)\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\,\mathcal{H}_{\kappa_{2}^{\prime}}}\langle\varphi^{\otimes}((s,a),\cdot),g\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\,\mathcal{H}_{\kappa_{2}^{\prime}}}\\!\Bigg{)}d\rho_{\mathcal{S}\times\mathcal{A}}((s,a),\cdot).$

Here,
$f,g\\!\in\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}$.
The double integral is over the same joint topological space
$\mathcal{S}\\!\times\\!\mathcal{A}$.

Using the bivariate mean-element $\mu_{\rho_{\mathcal{S}\times\mathcal{A}}}$,
the corresponding trace of this operator is analogous to the univariate case,

$\textnormal{tr}(\gamma)=\sum_{q=1}^{r}\int\\!\\!\\!\\!\int_{\mathcal{S}\times\mathcal{A}}\\!\Bigg{(}\\!\langle\pi_{q},\varphi^{\otimes}((s,a),\cdot)\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}}\langle\varphi^{\otimes}((s,a),\cdot),\pi_{q}\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}}\\!\Bigg{)}d\rho_{\mathcal{S}\times\mathcal{A}}((s,a),\cdot).$

Here,
$\\{\pi_{q}\\}_{j=1}^{r}\\!\subset\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}$
is a complete orthonormal basis for the quotient, completed Hilbert space.

  * Proposition 2.12: Quotient Hilbert Space Separability. Let $\mathcal{H}_{\kappa_{1}^{\prime}}$ and $\mathcal{H}_{\kappa_{2}^{\prime}}$ be separable reproducing-kernel Hilbert spaces induced by a continuous reproducing kernels $\kappa_{1}^{\prime}$ and $\kappa_{1}^{\prime}$. If $\mathcal{H}_{\kappa_{1}^{\prime}}$ and $\mathcal{H}_{\kappa_{2}^{\prime}}$ have orthonormal bases $\\{\eta_{j}\\}_{j=1}^{h}$ and $\\{\vartheta_{k}\\}_{k=1}^{p}$, respectively, then $\\{\eta_{j}\otimes\vartheta_{k}\\}_{j,k}\\!\subset\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\mathcal{H}_{\kappa_{2}^{\prime}}$, is an orthonormal set. The Hilbert dimension of the tensor products is the product of the Hilbert dimensions.

For universal reproducing kernels, the Hilbert-space dimensionality is
infinite.

We can extend the trace of the bivariate Gram operators to arbitrary positive
powers, which we will need shortly. A similar result holds in the univariate
case.

  * Proposition 2.13: Bivariate Gram Operator $\alpha$-Trace. For a separable, quotient reproducing-kernel Hilbert space, with a continuous reproducing kernel, let $(\kappa_{1}^{\prime}\otimes\kappa_{2}^{\prime})((s,\cdot),(a,\cdot))\\!=\\!\langle\varphi^{\otimes}((s,a),\cdot),\cdot\rangle_{\mathcal{H}_{\kappa^{\prime}}}$, where $\varphi^{\otimes}$ is
measurable. Assume that the product kernel is characteristic. For
$\alpha\\!\in\\!\mathbb{R}_{0,+}\\!\backslash\\{1\\}$, we have that the
$\alpha$-trace of the bivariate Gram operator, $\gamma$, is

$\displaystyle\textnormal{tr}(\gamma^{\alpha})$
$\displaystyle=\sum_{q=1}^{r}\int\\!\\!\\!\\!\int_{\mathcal{S}\times\mathcal{A}}\\!\Bigg{(}\\!\langle\pi_{q},\varphi^{\otimes}((s,a),\cdot)\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\mathcal{H}_{\kappa_{2}^{\prime}}}\langle\varphi^{\otimes}((s,a),\cdot),\gamma^{\alpha}\pi_{q}\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\mathcal{H}_{\kappa_{2}^{\prime}}}\\!\\!\Bigg{)}d\rho_{\mathcal{S}\times\mathcal{A}}((s,a),\cdot)$
$\displaystyle=\int\\!\\!\\!\\!\int_{\mathcal{S}\times\mathcal{A}}\\!\Bigg{(}\\!\langle\varphi^{\otimes}((s,a),\cdot),\gamma^{\alpha-1}\varphi^{\otimes}((s,a),\cdot)\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\mathcal{H}_{\kappa_{2}^{\prime}}}\\!\Bigg{)}d\rho_{\mathcal{S}\times\mathcal{A}}((s,a),\cdot).$

The inner product is a positive-definite function that depends on the product
probability measure.

As with the univariate case, the 2-trace of the bivariate Gram operator is
connected to the bivariate mean-element mapping. It is also equivalent to the
square of the Hilbert-Schmidt norm [24].

  * Proposition 2.14: Bivariate Operator-Map Equivalence. Let $\mathcal{H}_{\kappa_{1}^{\prime}}$ and $\mathcal{H}_{\kappa_{2}^{\prime}}$ be separable reproducing-kernel Hilbert spaces induced by a continuous reproducing kernels $\kappa_{1}^{\prime}$ and $\kappa_{1}^{\prime}$. Assume that the product kernel $(\kappa_{1}^{\prime}\otimes\kappa_{2}^{\prime})((s,\cdot),(a,\cdot))\\!=\\!\langle\varphi^{\otimes}((s,a),\cdot),\cdot\rangle_{\mathcal{H}_{\kappa^{\prime}_{1}}\otimes\mathcal{H}_{\kappa^{\prime}_{2}}}$ is characteristic. We have that the relationship between the 2-trace of the bivariate Gram operator, $\gamma$, and the bivariate mean-element map $\mu_{\rho_{\mathcal{S}\times\mathcal{A}}}$ is

$\int\\!\\!\\!\\!\int_{\mathcal{S}\times\mathcal{A}}\\!\Bigg{(}\\!\langle\varphi^{\otimes}((s,a),\cdot),\gamma\varphi^{\otimes}((s,a),\cdot)\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\mathcal{H}_{\kappa_{2}^{\prime}}}\\!\Bigg{)}d\rho_{\mathcal{S}\times\mathcal{A}}((s,a),\cdot)=\langle\hskip
0.28436pt\mu_{\rho_{\mathcal{S}\times\mathcal{A}}},\mu_{\rho_{\mathcal{S}\times\mathcal{A}}}\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}\kappa_{1}^{\prime}}\otimes\mathcal{H}_{\kappa_{2}^{\prime}\kappa_{2}^{\prime}}}.$

Here,
$\mathcal{H}_{\kappa_{1}^{\prime}\kappa_{1}^{\prime}\otimes\kappa_{2}^{\prime}\kappa_{2}^{\prime}}$
denotes the reproducing-kernel Hilbert space induced by the tensor-product
kernel
$(\kappa_{1}^{\prime}\kappa_{1}^{\prime}\\!\otimes\\!\kappa_{2}^{\prime}\kappa_{2}^{\prime})((s,s^{\prime}),(a,a^{\prime}))$.

As a byproduct of the mean-element map relationship, we also obtain that the
Gram operators are non-parametric, lossless characterizations of the
interactions between probability measures. The operators also contain
information about all statistical moments. Without these guarantees, then we
could not be assured that working in such a functional space would preserve
the necessary traits of the probability measures to facilitate proper
analysis.

Lastly, we consider a bivariate operator that corresponds to cross-
correlations of functions. We will use this operator to quantify the overlap
of marginal distributions.

  * Definition 2.7: Joint, Bivariate Gram Operator. For separable reproducing-kernel Hilbert spaces $\mathcal{H}_{\kappa_{1}^{\prime}}$ and $\mathcal{H}_{\kappa_{2}^{\prime}}$, with continuous, universal reproducing kernels defined on state-action pairs, let $\kappa^{\prime}_{1}(s,s^{\prime})\\!=\\!\langle\varphi(s),\varphi(s^{\prime})\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}}$
and
$\kappa^{\prime}_{2}(a,a^{\prime})\\!=\\!\langle\psi(a),\psi(a^{\prime})\rangle_{\mathcal{H}_{\kappa_{2}^{\prime}}}$,
where
$\varphi\\!:\\!\mathcal{S}\\!\times\\!\mathcal{S}\\!\to\\!\mathcal{H}_{\kappa_{1}^{\prime}}$
and
$\psi\\!:\\!\mathcal{A}\\!\times\\!\mathcal{A}\\!\to\\!\mathcal{H}_{\kappa_{2}^{\prime}}$
are measurable. We define the
joint, bivariate Gram operator $\lambda$ via the bilinear form,

$\mathcal{L}(f,g)=\langle f,\lambda
g\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes^{*}\,\mathcal{H}_{\kappa_{2}^{\prime}}}=\\!\int_{\mathcal{S}\times\mathcal{A}}\Bigg{(}\\!\langle
f,\varphi(s)\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}}\langle\varphi(s),\psi(a)\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\,\mathcal{H}_{\kappa_{2}^{\prime}}}\langle\psi(a),g\rangle_{\mathcal{H}_{\kappa_{2}^{\prime}}}\\!\Bigg{)}d\rho_{\mathcal{S}\times\mathcal{A}}(s,a).$

Here, $f\\!\in\\!\mathcal{H}_{\kappa_{1}^{\prime}}$,
$g\\!\in\\!\mathcal{H}_{\kappa_{2}^{\prime}}$.

## 3\. Bipartite Cross-Entropy

Rényi, in [37], pursued an axiomatic investigation of Shannon’s entropy. He
showed that a parameterized family of entropies could be constructed. Rényi
also created bipartite relative entropies and bipartite cross-entropies, both
of which are parameterized. The latter of these is specified below.

  * Definition 3.1: Rényi’s $\alpha$-Cross-Entropy Let $q_{1},q_{2}\\!:\\!\mathcal{S}\\!\to\\!\mathbb{R}_{0,+}$ be probability densities over a common topological sample space $\mathcal{S}$. The $\alpha$-order Rényi’s cross-entropy, for $\alpha\\!\in\\!\mathbb{R}_{+}\\!\backslash\\{1\\}$, is

$H_{\alpha}(q_{1}\|q_{2})=\frac{1}{\alpha\\!-\\!1}\textnormal{log}\Bigg{(}\\!\int_{\mathcal{S}}q_{1}^{\alpha}(s)q_{2}^{1-\alpha}(s)dp_{\mathcal{S}}(s)\\!\Bigg{)}.$

Here, $p_{\mathcal{S}}\\!\in\\!\mathcal{M}^{1}_{+}(\mathcal{S})$ is a Borel
probability measure over $\mathcal{S}$.

Intuitively, the $\alpha$-cross-entropy is a measure of distributional
overlap.

In this section, we extend Rényi’s $\alpha$-cross-entropies to take as input
non-commutative Gram operators in reproducing-kernel Hilbert spaces. We start
with the most straightforward generalization, which we refer to as the
bipartite, non-mirrored Rényi’s $\alpha$-cross-entropy (see Definition 3.2).
This version provably satisfies the same axioms that Rényi considered for
divergences. It therefore acts like a divergence.

Unfortunately, this measure only satisfies the data-processing inequality on a
small set of parameter values. Outside of this set, any transformation from a
given class of functions will artificially increase the distinguishability of
the operators, leading to erroneous magnitudes. We hence construct another
relative entropy for non-commutative operators. We refer to this as the
bipartite, mirrored Rényi’s $\alpha$-cross-entropy (see Definition 3.3). It
obeys this inequality on the full parameter-value range. It too satisfies the
fundamental axioms.

Both the mirrored and non-mirrored cross-entropies rely on traces of either
the univariate or bivariate Gram operators. They hence rely on access to a
basis for the reproducing-kernel Hilbert space. As we noted, it can be
difficult to explicitly specify a basis, depending on the kernel choice. Here,
we sidestep this issue by considering principled approximations of the Gram
operators, which are empirical Gram matrices. We do this for the univariate
case (see Definition 3.4 and Proposition 3.1) and the bivariate case (see
Definition 3.5 and Proposition 3.2). This approximation leads us to define
matrix-based estimators of cross-entropy (see Algorithms 3.1 and 3.2). The
Gram matrices are unbiased, non-parametric estimates of the Gram operators.
These matrices can be constructed through the application of kernels to
pairwise distances between vector samples. They can also be formed from the
application of kernels to representations encoded by symmetric, non-negative
matrices that are square.

We show that the spectral characteristics of the Gram matrices and Gram
operators are consistent (see Propositions 3.1 and 3.2). This holds for the
univariate and bivariate cases. We additionally bound the difference between
the traces of the Gram operators and the traces of the Gram matrices (see
Propositions 3.1 and 3.2). These bounds are minimax optimal (see Propositions
3.3 and 3.4). They hence are the best obtainable bounds for the worst possible
condition. Our matrix-based cross-entropies do not require direct access to
the underlying probability distributions, just samples from them, to work
well. Such results indicate that the Gram matrices are suitable substitutes
for the operators. As well, the bounds are completely independent of the
sample-set dimensionality (see Propositions 3.3 and 3.4). This implies that
our cross-entropies are appropriate for handling high-dimensional samples
well. Alternate estimators, like those that rely on approximating and
tightening evidence lower bounds [46], can be rather sensitive to the sample
dimensionality [47].

It is important to note that proper matrix conditioning must be imposed for
the Gram matrices, though, for them to be useful for assessing cross entropy.
This is because we will be, potentially, raising the Gram matrices to
fractional powers, which will entail matrix inversion. Poorly conditioned
matrices will yield poor empirical results.

## 3.1. Operator-Based Cross-Entropies

Bipartite, Non-Mirrored Rényi’s $\alpha$-Cross-Entropy. For non-commutative
Gram operators, we specify an operator-based divergence measure that strongly
resembles Rényi’s classical $\alpha$-divergences. We refer to this function as
the bipartite, non-mirrored case, as it contains only a single instance of two
Gram operators in its primary term.

  * Definition 3.2: Bipartite, Non-Mirrored Rényi’s $\alpha$-Cross-Entropy. Let $\mathcal{H}_{\kappa_{1}^{\prime}}$, $\mathcal{H}_{\kappa_{2}^{\prime}}$ be separable reproducing-kernel Hilbert spaces, with continuous, universal reproducing kernels $\kappa_{1}^{\prime},\kappa_{2}^{\prime}$. Let $\kappa_{1}\\!:\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\to\\!\mathcal{H}_{\kappa_{1}^{\prime}}$, $\kappa_{2}\\!:\\!\mathcal{H}_{\kappa_{2}^{\prime}}\\!\to\\!\mathcal{H}_{\kappa_{2}^{\prime}}$ be the corresponding univariate Gram operators. The bipartite, non-mirrored, operator-based Rényi’s $\alpha$-cross-entropy of $\kappa_{1}$ and $\kappa_{2}$, with respect to $\alpha\\!\in\\!\mathbb{R}_{0,+}\\!\backslash\\{1\\}$, is

$C_{\alpha}(\kappa_{1}\|\kappa_{2})=\frac{1}{\alpha\\!-\\!1}\textnormal{log}\Bigg{(}\\!\textnormal{tr}(\kappa_{1}^{\alpha}\kappa_{2}^{1-\alpha})\\!\Bigg{)}\\!-\\!\frac{1}{\alpha\\!-\\!1}\textnormal{log}\Bigg{(}\\!\textnormal{tr}(\kappa_{1})\\!\Bigg{)},\;\textnormal{for
}\textnormal{supp}(\kappa_{2})\\!\subseteq\\!\textnormal{supp}(\kappa_{1}),$

with $C_{\alpha}(\kappa_{1}\|\kappa_{2})\\!=\\!\infty$ otherwise.
$\kappa_{1}^{\alpha}\kappa_{2}^{1-\alpha}\\!=\\!\gamma$ is, naturally, a
bivariate Gram operator. Here, $\textnormal{supp}(\kappa_{*})$ is the
support of $\kappa_{*}$, i.e., the span of the eigenvectors of $\kappa_{*}$
corresponding to non-zero eigenvalues. The variable $\kappa_{*}$ represents
either $\kappa_{1}$ or $\kappa_{2}$.

In this cross-entropy measure, the arguments of the log terms can be viewed as
transformations that transfer statistical properties of the measures to
reproducing-kernel Hilbert spaces. This follows from the theory established in
the previous section. The second log term acts as a normalization term. It is
consistent with what Rényi proposed for incomplete distributions [37].

As we illustrate in the appendix (see Appendix A.1), this operator-based
divergence measure satisfies the five major axioms that Rényi specified for
divergence measures. The sixth axiom, that of normalization, is trivial to
demonstrate for Gram operators associated with single-Dirac-measure
distributions. This operator-based cross-entropy can thus behave like a
divergence. Several other properties are realized too, which are presented in
the appendix.

Bipartite, Mirrored Rényi’s $\alpha$-Cross-Entropy. As we note in the
appendix, an issue with the preceding measure is that it does not satisfy the
data-processing inequality. Certain transformations of the operators can
artificially change the cross-entropy scores. We hence consider an amended
operator-based Rényi’s $\alpha$-cross-entropy that will not improperly alter
the distributional difference. We refer to this function as the mirrored case,
due to the appearance of dual non-commutative operators in the primary term.

  * Definition 3.3: Bipartite, Mirrored Rényi’s $\alpha$-Cross-Entropy. Let $\mathcal{H}_{\kappa_{1}^{\prime}}$, $\mathcal{H}_{\kappa_{2}^{\prime}}$ be separable reproducing-kernel Hilbert spaces, with continuous, universal reproducing kernels $\kappa_{1}^{\prime},\kappa_{2}^{\prime}$. Let $\kappa_{1}\\!:\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\to\\!\mathcal{H}_{\kappa_{1}^{\prime}}$, $\kappa_{2}\\!:\\!\mathcal{H}_{\kappa_{2}^{\prime}}\\!\to\\!\mathcal{H}_{\kappa_{2}^{\prime}}$ be the corresponding Gram operators. The mirrored, operator-based Rényi’s $\alpha$-divergence of $\kappa_{1}$ and $\kappa_{2}$, with respect to $\alpha\\!\in\\!\mathbb{R}_{+}\\!\backslash\\{1\\}$, is

$C^{\prime}_{\alpha}(\kappa_{1}\|\kappa_{2})=\frac{1}{\alpha\\!-\\!1}\textnormal{log}\Bigg{(}\\!\textnormal{tr}\Bigg{(}\\!\kappa_{2}^{\frac{1-\alpha}{2\alpha}}\kappa_{1}\kappa_{2}^{\frac{1-\alpha}{2\alpha}}\\!\Bigg{)}^{\\!\\!\alpha}\Bigg{)}\\!-\\!\frac{1}{\alpha\\!-\\!1}\textnormal{log}\Bigg{(}\\!\textnormal{tr}(\kappa_{1})\\!\Bigg{)},\;\textnormal{for
}\textnormal{supp}(\kappa_{2})\\!\subseteq\\!\textnormal{supp}(\kappa_{1}),$

and $C^{\prime}_{\alpha}(\kappa_{1}\|\kappa_{2})\\!=\\!\infty$ otherwise.
$\kappa_{2}^{(1-\alpha)/2\alpha}\kappa_{1}\kappa_{2}^{(1-\alpha)/2\alpha}\\!=\\!\gamma$
is a bivariate Gram operator.

This operator-based cross-entropy also satisfies the six fundamental axioms
that Rényi outlined for divergences, amongst other properties. We provide more
details in the appendix (see Appendix A.1).

## 3.2. Empirical Estimation of Operator-Based Cross-Entropies

These cross-entropy measures assume access to the univariate Gram operators
and hence the underlying probability distributions. In practice, the
distributions underlying the operators will only be known through random
samples. Therefore, the operators $\kappa$ and $\gamma$ can be estimated by
$\hat{\kappa}$ and $\hat{\gamma}$ under the condition that
$\mathbb{E}_{(s_{i},s^{\prime})\sim\hat{p}_{\mathcal{S}}}[\hat{\kappa}(s_{i},s^{\prime})]\\!=\\!\kappa$
and
$\mathbb{E}_{(s_{i},a_{i})\sim\hat{\rho}_{\mathcal{S}\times\mathcal{A}}}[\hat{\gamma}((s_{i},a_{i}),\cdot)]\\!=\\!\gamma$.

We first show that the univariate Gram operators can be approximated.

  * Definition 3.4: Univariate Gram Operator Approximation. Let $\mathcal{H}_{\kappa^{\prime}}$ be a separable reproducing-kernel Hilbert space, with a continuous, universal reproducing kernel. An unbiased empirical estimate $\hat{\kappa}(f,g)$,
$f,g\\!\in\\!\mathcal{H}_{\kappa^{\prime}}$, of the symmetric, self-adjoint
operator $\kappa$ is

$\hat{\kappa}(f,g)=\langle
f,\hat{\kappa}g\rangle_{\mathcal{H}_{\kappa^{\prime}}}\\!=\\!\int\\!\\!\\!\\!\int_{\mathcal{S}}\langle
f,\varphi(s_{i})\rangle_{\mathcal{H}_{\kappa^{\prime}}}\langle\varphi(s^{\prime}),g\rangle_{\mathcal{H}_{\kappa^{\prime}}}d\hat{p}_{\mathcal{S}}(s_{i},s^{\prime})$

which evaluates to $n^{-1}\sum_{i=1}^{n}\langle
f,\varphi(s_{i})\rangle_{\mathcal{H}_{\kappa^{\prime}}}\langle\varphi(s^{\prime}),g\rangle_{\mathcal{H}_{\kappa^{\prime}}}$,
where $s_{i}\\!\in\\!\mathcal{S}$, $i\\!\in\\!\mathbb{N}_{1,n}$, are samples.
This corre-
sponds to replacing $p_{\mathcal{S}}$ with an empirical version
$\hat{p}_{\mathcal{S}}\\!\in\\!\mathcal{M}^{1}_{+}(\mathcal{S})$ formed by a
series of $n$ Dirac measures $\delta_{s_{i}}$,
$\hat{p}_{\mathcal{S}}\\!=\\!\sum_{i=1}^{n}\delta_{s_{i}}/n$.

As well, the bivariate Gram operators can be approximated.

  * Definition 3.5: Bivariate Gram Operator Approximation. Let $\mathcal{H}_{\kappa_{1}^{\prime}}$, $\mathcal{H}_{\kappa_{2}^{\prime}}$ be separable reproducing-kernel Hilbert spaces, with continuous, universal reproducing kernels $\kappa_{1}^{\prime},\kappa_{2}^{\prime}$. An unbiased empirical estimate $\hat{\gamma}(f,g)$, $f,g\\!\in\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes\\!\mathcal{H}_{\kappa_{2}^{\prime}}$, of the symmetric, self-adjoint operator $\gamma$ is

$\hat{\gamma}(f,g)=\\!\int\\!\\!\\!\\!\int_{\mathcal{S}\times\mathcal{A}}\langle
f,\varphi^{\otimes}((s_{i},a_{i}),\cdot)\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\,\mathcal{H}_{\kappa_{2}^{\prime}}}\langle\varphi^{\otimes}((s_{i},a_{i}),\cdot),g\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\,\mathcal{H}_{\kappa_{2}^{\prime}}}d\hat{\rho}_{\mathcal{S}\times\mathcal{A}}((s_{i},a_{i}),\cdot),$

which evaluates to $n^{-1}\sum_{i=1}^{n}\langle
f,\varphi^{\otimes}((s_{i},a_{i}),\cdot)\rangle_{\mathcal{H}_{\kappa^{\prime}_{1}}\otimes\mathcal{H}_{\kappa^{\prime}_{2}}}\langle\varphi^{\otimes}((s_{i},a_{i}),\cdot),g\rangle_{\mathcal{H}_{\kappa^{\prime}}}$,
where $s_{i}\\!\in\\!\mathcal{S}$, $a_{i}\\!\in\\!\mathcal{A}$,
$i\\!\in\\!\mathbb{N}_{1,n}$ are samples. This corresponds to replacing
$\rho_{\mathcal{S}\times\mathcal{A}}$ with an empirical version
$\hat{\rho}_{\mathcal{S}\times\mathcal{A}}\\!\in\\!\mathcal{M}^{1}_{+}(\mathcal{S}\\!\times\\!\mathcal{A})$
formed
by a series of $n$ Dirac measures.

Due to the connection between the reproducing kernels and the measurable
feature maps, we can substitute, for the Gram operator $\kappa$, a non-
negative, scaled Gram matrix $[\hat{\kappa}]_{i,j}\\!\in\\!\mathbb{R}_{0,+}$,
$i,j\\!\in\\!\mathbb{N}_{1,n}$, constructed by all pairwise evalua-
tions of a normalized kernel $\kappa^{\prime}$ to random samples,
$s_{i},s_{j}\\!\in\\!\mathcal{S}$, $i,j\\!\in\\!\mathbb{N}_{1,n}$.
Alternatively, for undirected graphs, a
kernel can be applied to the entries of square, symmetric adjacency matrices.
Not all kernels can be applied directly to all matrices, though.

  * Proposition 3.1: Univariate Gram Operator and Gram Matrix Relationship. Let $[\hat{\kappa}]_{i,j}\\!=\\!\langle\varphi(s_{i}),\varphi(s_{j})\rangle_{\mathcal{H}_{\kappa^{\prime}}}$ be an empirical Gram matrix, $\hat{\kappa}\\!\in\\!\mathbb{R}_{0,+}^{n\times n}$, for a measurable feature map $\varphi$. Assume that the corresponding kernel $\kappa^{\prime}$ is continuous and universal.

Let $\hat{\kappa}$ be an unbiased empirical estimate of the univariate Gram
operator $\kappa$. $\hat{\kappa}$ has at most $n$ positive eigenvalues that
satisfy $[\hat{\kappa}]\beta_{i}/n\\!=\\!\lambda^{1}_{i}\beta_{i}$, where
$\lambda_{i}\\!\in\\!\mathbb{R}_{+}$ are eigenvalues and
$\beta_{i}\\!\in\\!\mathbb{R}^{n}$ are eigenvectors. As well, $n\lambda_{i}$
are all positive eigenvalues of $[\hat{\kappa}]$ under the assumption that the
Gram matrices are normalized as
$n^{-1}\sum_{i=1}^{n}[\hat{\kappa}]_{i,i}\\!=\\!1$.

We impose a trace normalization constraint on the Gram matrices ensures that
they will be infinitely divisible. This permits us to construct notions of
operator- and matrix-based entropy and mutual information via our cross-
entropies. We discuss this topic more in the appendix (see Appendix A.1).

We can, similarly, substitute a non-negative Gram matrix
$[\hat{\gamma}]_{i,j}\\!\in\\!\mathbb{R}_{0,+}$,
$i,j\\!\in\\!\mathbb{N}_{1,n}$, for the bivariate Gram operator $\gamma$.
$[\hat{\gamma}]$ corresponds to the non-linear combination of univariate Gram
matrices $[\hat{\kappa}]$.

  * Proposition 3.2: Bivariate Gram Operator and Gram Matrix Relationship. Let $\hat{\gamma}\\!\in\\!\mathbb{R}_{0,+}^{n\times n}$ be an empirical Gram matrix, $[\hat{\gamma}]_{i,j}\\!=\\!\langle\varphi^{\otimes}(s_{i},a_{i}),\varphi^{\otimes}(s_{j},a_{j})\rangle_{\mathcal{H}_{\kappa^{\prime}_{1}}\otimes\mathcal{H}_{\kappa^{\prime}_{2}}}$, for a measurable feature map $\varphi^{\otimes}$. Assume that the
corresponding kernels $\kappa_{1}^{\prime},\kappa_{2}^{\prime}$ are continuous
and universal.

Let $\hat{\gamma}$ be an unbiased empirical estimate of the univariate Gram
operator $\gamma$. $\hat{\gamma}$ has at most $n$ positive eigen-
values that satisfy $[\hat{\gamma}]\beta_{i}\\!=\\!\tau_{i}\beta_{i}$, where
$\lambda_{i}\\!\in\\!\mathbb{R}_{+}$ are eigenvalues and
$\beta_{i}\\!\in\\!\mathbb{R}^{n}$ are eigenvectors. As well, $\tau_{i}$ are
all positive eigenvalues of $[\hat{\gamma}]$.

Performing this substitution yields matrix-based $\alpha$-cross-entropies,
which are stated in Algorithms 3.1 and 3.2. Note that, due to the Gram-matrix
scaling constraint, $n^{-1}\sum_{i=1}^{n}[\kappa_{1}]_{i,i}\\!=\\!1$, the
second log term evaluates to zero for both cross-entropies.

It is important to note that both the kernel choice and kernel parameters
greatly influence the entries of the Gram matrices. It hence influences the
cross-entropy magnitudes. We refer to [36, 48] for discussions about kernel
choices. We also refer to [22, 49, 50] for discussions about kernel
parameters, at least in the context of using the $\alpha$-cross-entropies as a
two-sample test [51]. For the general problem of learning kernels, see [52,
53, 54].

Data: $\alpha\\!\in\\!\mathbb{R}_{0,+}\\!\backslash\\{1\\}$: Order of the
cross entropy. Input:
$[\hat{\kappa}_{1}],[\hat{\kappa}_{2}]\\!\in\\!\mathbb{R}_{0,+}^{n\times n}$:
unit-trace-normalized, properly conditioned Gram matrices with the
constraint
$\textnormal{supp}([\hat{\kappa}_{2}])\\!\subseteq\\!\textnormal{supp}([\hat{\kappa}_{1}])$.
1 Return $\displaystyle
C_{\alpha}([\hat{\kappa}_{1}]\|[\hat{\kappa}_{2}])\\!=\\!\frac{1}{\alpha\\!-\\!1}\textnormal{log}\Bigg{(}\\!\textnormal{tr}([\hat{\kappa}_{1}]^{\alpha}[\hat{\kappa}_{2}]^{1-\alpha})\\!\Bigg{)}$.
Algorithm 3.1 Non-Mirrored Matrix-Based Rényi’s $\alpha$-Cross-Entropy
$C_{\alpha}([\hat{\kappa}_{1}]\|[\hat{\kappa}_{2}])$ Data:
$\alpha\\!\in\\!\mathbb{R}_{0,+}\\!\backslash\\{1\\}$: Order of the cross
entropy. Input:
$[\hat{\kappa}_{1}],[\hat{\kappa}_{2}]\\!\in\\!\mathbb{R}_{0,+}^{n\times n}$:
unit-trace-normalized, properly conditioned Gram matrices with the
constraint
$\textnormal{supp}([\hat{\kappa}_{2})]\\!\subseteq\\!\textnormal{supp}([\hat{\kappa}_{1}])$.
1 Return $\displaystyle
C^{\prime}_{\alpha}([\hat{\kappa}_{1}]\|[\hat{\kappa}_{2}])\\!=\\!\frac{1}{\alpha\\!-\\!1}\textnormal{log}\Bigg{(}\\!\textnormal{tr}\Bigg{(}\\![\hat{\kappa}_{2}]^{\frac{1-\alpha}{2\alpha}}[\hat{\kappa}_{1}][\hat{\kappa}_{2}]^{\frac{1-\alpha}{2\alpha}}\\!\Bigg{)}^{\\!\\!\alpha}\,\Bigg{)}$.
Algorithm 3.2 Mirrored Matrix-Based Rényi’s $\alpha$-Cross-Entropy
$C^{\prime}_{\alpha}([\hat{\kappa}_{1}]\|[\hat{\kappa}_{2}])$
---

Replacing the empirical Gram operators with matrices is not arbitrary. As we
noted above, the spectral characteristics of the empirical Gram operators and
Gram matrices are consistent. The trace function therefore returns a
proportionally equivalent result in either case, which lends credence to using
easily computable Gram matrices.

  * Proposition 3.3: Univariate Gram Matrix Convergence Properties. Let $\mathcal{H}_{\kappa^{\prime}}$ be a separable reproducing-kernel Hilbert space. Let $\hat{\kappa}$ be an approximate Gram operator and $[\hat{\kappa}]\\!\in\\!\mathbb{R}_{0,+}^{n\times n}$ be the corresponding normal-
ized Gram matrix. Assume that the corresponding kernel $\kappa^{\prime}$ is
continuous and universal. We have that, for
$\alpha\\!\in\\!\mathbb{R}_{+}\\!\backslash\\{1\\}$,

    * (i) $\textnormal{tr}(\hat{\kappa})\\!=\\!\textnormal{tr}([\hat{\kappa}])$.

    * (ii) For the univariate operator $\kappa$, the inequality $|\textnormal{tr}(\kappa)\\!-\\!\textnormal{tr}(\hat{\kappa})|\\!\leq\\!c_{1}(2\textnormal{log}(2/\delta)/n)^{1/2}$ is satisfied with
probability $1\\!-\\!\delta$, where the kernel magnitudes obey
$\kappa^{\prime}\\!\leq\\!c_{1}$, with $c_{1}\\!\in\\!\mathbb{R}_{+}$. Here,
$\textnormal{tr}(\kappa)\\!=\\!\sum_{j=1}^{h}\langle\eta_{j},\kappa\eta_{j}\rangle_{\mathcal{H}_{\kappa^{\prime}_{1}}}$,
where $\eta_{j}$ are elements of the orthonormal basis for
$\mathcal{H}_{\kappa^{\prime}}$.

    * (iii) The inequality $\textnormal{inf}_{\hat{\kappa}}\,\textnormal{sup}_{\kappa}\,\|\textnormal{tr}([\hat{\kappa}])\\!-\\!\textnormal{tr}(\kappa)\|_{\mathcal{H}_{\kappa^{\prime}}}\\!\geq\\!c_{2}(2c_{3}/n)^{1/2}$ is satisfied with non-zero probability,
where $c_{2},c_{3}\\!\in\\!\mathbb{R}_{+}$. Hence, the trace of the univariate
Gram matrix is minimax optimal.

  * Proposition 3.4: Bivariate Gram Matrix Convergence Properties. Let $\mathcal{H}_{\kappa^{\prime}_{1}}$, $\mathcal{H}_{\kappa^{\prime}_{2}}$ be separable reproducing-kernel Hilbert spaces. Let $\hat{\kappa_{1}},\hat{\kappa_{2}}$ be approximate univariate Gram operators and $[\hat{\kappa}_{1}],[\hat{\kappa}_{2}]\\!\in\\!\mathbb{R}_{0,+}^{n\times n}$ be the corresponding normalized Gram matrices. Assume that the corresponding kernel $\kappa^{\prime}_{1},\kappa_{2}^{\prime}$ are continuous and universal. Moreover, assume that $\hat{\gamma}$ and $[\hat{\gamma}]\\!\in\\!\mathbb{R}_{0,+}^{n\times n}$ are, respectively, continuous combinations of $\hat{\kappa}_{1},\hat{\kappa}_{2}$ and $[\hat{\kappa}_{1}],[\hat{\kappa}_{2}]$. We have that

    * (i) $\textnormal{tr}(\hat{\gamma})\\!=\\!\textnormal{tr}([\hat{\gamma}])$, where $[\hat{\gamma}]\\!=\\![\hat{\kappa}_{1}]^{\alpha}[\hat{\kappa}_{2}]^{1-\alpha}$ or $[\hat{\gamma}]\\!=\\![\kappa_{2}]^{(1-\alpha)/2\alpha}[\kappa_{1}][\kappa_{2}]^{(1-\alpha)/2\alpha}$, with $\hat{\gamma}$ having ana-
logous forms.

    * (ii) For the bivariate operator $\gamma$, the inequality $|\textnormal{tr}(\gamma)\\!-\\!\textnormal{tr}(\hat{\gamma})|\\!\leq\\!\alpha c_{1}c_{2}c_{3}(2\textnormal{log}(2/\delta)/n)^{1/2}$ is satisfied with probability $1\\!-\\!\delta$. We assume that the kernel magnitudes obey $\kappa^{\prime}_{1}\\!\leq\\!c_{1}$ and $\kappa^{\prime}_{2}\\!\leq\\!c_{2}$, with $c_{1},c_{2}\\!\in\\!\mathbb{R}_{+}$. $c_{3}\\!\in\\!\mathbb{R}_{+}$ takes the value $c_{3}\\!=\\!1$ for the non-mirrored case and $c_{3}\\!=\\!\alpha$ for the mirrored case. For both cross-entropies,
$\textnormal{tr}(\gamma)\\!=\\!\sum_{q=1}^{r}\langle\pi_{q},\gamma\pi_{q}\rangle_{\mathcal{H}_{\kappa^{\prime}_{1}}\otimes\mathcal{H}_{\kappa^{\prime}_{2}}}$,
where $\pi_{q}$ are elements of the orthonormal basis for
$\mathcal{H}_{\kappa^{\prime}_{1}}\\!\otimes\\!\mathcal{H}_{\kappa^{\prime}_{2}}$.

    * (iii) The inequality $\textnormal{inf}_{\hat{\gamma}}\,\textnormal{sup}_{\gamma}\,\|\textnormal{tr}(\hat{\gamma})\\!-\\!\textnormal{tr}(\gamma)\|_{\mathcal{H}_{\kappa^{\prime}_{1}}\otimes\mathcal{H}_{\kappa^{\prime}_{2}}}\\!\geq\\!c_{4}(2c_{5}/n)^{1/2}$ is satisfied with non-zero
probability, for constants $c_{4},c_{5}\\!\in\\!\mathbb{R}_{+}$. Hence, the
trace of the bivariate Gram matrix is minimax optimal.

We can thus deduce that Gram matrices, when constructed from universal
kernels, provide a sufficiently good characterization of the empirical
distributions. This matrix-based cross-entropies hence can assess differences
between probability distributions without needing direct access to those
distributions.

It is important to notice that the convergence rate of the Gram matrices to
the Gram operators is independent of the dimensionality of the samples. It is,
instead, primarily a function of the number of samples used in the estimation
process. This property makes our matrix-based notions of cross-entropy
appealing for many applications. Moreover, our measures avoid the concerns for
plug-in density estimators [55]. Namely, our measures do not have slow-rate-
of-convergence issues for arbitrary distributions [56, 57].

## 3.3. Numerical Examples

Figure 3.1: An overview of the sample-convergence properties for matrix-based
cross-entropies. (a)–(c) Two-dimensional scatter plots for samples drawn from
two Gaussian distributions with equivalent means and variances. Only the
number of samples differs in the scatter plots, with the left-most plot having
the fewest samples and the right-most plot having the most samples. (d)–(h)
plots of $C_{\alpha}([\hat{\kappa}_{1}]\|[\hat{\kappa}_{2}])$ as a function of
the cross-entropy order, the number of samples, and the dimensionality of the
distributions. We consider (d) 2-dimensional, (e) 10-dimensional, (f)
25-dimensional, (g) 50-dimensional, and (h) 100-dimensional distributions.
Here we used a translation-invariant, Gaussian kernel,
$\kappa^{\prime}(s,s^{\prime})\\!=\\!\textnormal{exp}(-\sigma\langle
s\\!-\\!s^{\prime},\cdot\rangle)$, $\sigma\\!=\\!1$, for both
$\kappa_{1}^{\prime}$ and $\kappa_{2}^{\prime}$. (i)–(m) plots of
$C_{\alpha}^{\prime}([\hat{\kappa}_{1}]\|[\hat{\kappa}_{2}])$ as a function of
the cross-entropy order, the number of samples, and the dimensionality of the
distributions. We consider the same distributional dimensionality as in
(d)–(h). We again used a translation-invariant, Gaussian kernel. These results
show that the convergence rate of the matrix-based cross-entropies is
independent of the sample dimensionality.

Dimension Agnosticity Analysis. The theory that we outlined above posits that
the convergence rate of the Gram-matrix-based cross-entropy to the Gram-
operator-based cross-entropy is independent of the vector-sample
dimensionality. It is informative to show that this behavior occurs. Examples
of it are displayed in Figure 3.1.

In Figure 3.1, we plot cross-entropy magnitudes for our bipartite, mirrored
and non-mirrored measures. We draw vector samples from Gaussian distributions
(red and blue) with near-equivalent variances and means. The same number of
samples are used for each distribution. We vary the distribution
dimensionality between two and a hundred. Snapshots of the two-dimensional
samples, for increasing sample amounts, are shown in Figure 3.1(a)–(c). For
each set of samples, we compute the pairwise sample distances and use the
Gaussian kernel to form the Gram matrices. We fix the kernel bandwidth to a
constant, positive value.

Due to the high amount of distributional overlap, we expect that the cross-
entropy magnitudes will be low. They will not, however, necessarily be zero,
since the sample sets are not equivalent. The results in Figure 3.1(d)–(h) and
(i)–(m) are aligned with these expectations, regardless of the sample
dimensionality. They also highlight the minimax convergence rate guarantees.
In each case, the cross-entropy magnitude grows proportionally to the square
root of the number of samples. The growth rate is additionally a function of
the cross-entropy order. This is anticipated, since the constant terms in the
minimax bounds depend on this parameter. Moreover, the cross-entropy curves
are remarkably consistent, regardless of the dimensionality.

Distribution Assessment Analysis. It is also informative to demonstrate how
well the cross-entropy measures can assess distributional overlap when
employing different kernels. We consider two examples for vector samples drawn
from Gaussian distributions. These examples are presented in Figures 3.2–3.3.

In Figures 3.2, we consider the case of distributional mean shift. Samples for
one distribution (red) are held fixed while the mean for the samples for the
other distribution (blue) are shifted. The same number of samples are used for
each distribution. Snapshots are shown in Figures 3.2(a), (f), and (g). Both
distributions have the same variance. We hence expect that the matrix-based,
bipartite, mirrored $\alpha$-cross-entropy should reach a minimum value
whenever the distributional means are equivalent. The cross-entropy should
gradually rise as the means diverge and hence the distributions have
increasingly minimal overlap. This behavior is observed in Figure 3.2(k) when
using translation-varying kernels, like the exponential inner-product kernel.
It does not, however, emerge for translation-invariant kernels, like the
popular Gaussian kernel, which can be seen in Figure 3.2(j). In both cases, we
plot the cross-entropy as a function of the mean shift amount and cross-
entropy order.

Figure 3.2: An overview of the importance of kernel choice when considering
distributional shifts. (a) A two-dimensional scatter plot for samples drawn
from two Gaussian distributions with equivalent variances but different means.
(b)–(c) Empirical Gram matrices $\hat{\kappa}_{1}$ and $\hat{\kappa}_{2}$,
respectively, for the blue and red distributions shown in (a). Here a
translation-invariant, Gaussian kernel was used,
$\kappa^{\prime}(s,s^{\prime})\\!=\\!\textnormal{exp}(-\sigma\langle
s\\!-\\!s^{\prime},\cdot\rangle)$, $\sigma\\!=\\!1$. (d)–(e) Empirical Gram
matrices $\hat{\kappa}_{1}$ and $\hat{\kappa}_{2}$, respectively, for the blue
and red distributions shown in (a). Here a translation-varying, exponential-
inner- product kernel was used,
$\kappa^{\prime}(s,s^{\prime})\\!=\\!\textnormal{exp}(\sigma
s^{\top}\\!s^{\prime})$, $\sigma\\!=\\!1$. In (f)–(g), we shift the mean of
the samples from the blue distribution in (a). This shift is not captured by
the translation-invariant kernel. Only the Gram matrices for the translation-
varying kernel change. In (h) and (i), we plot the empirical Gram matrices
$\hat{\kappa}_{1}$ associated with the blue distribution in (f)–(g). The
empirical Gram matrices $\hat{\kappa}_{2}$ associated with the red
distribution in (f)–(g) stay the same, since the distribution does not change.
(j)–(k) Plots of the matrix-based Rényi’s $\alpha$-cross-entropy
$C_{\alpha}(\hat{\kappa}_{1}\|\hat{\kappa}_{2})$ as a function of the mean
shift amount of the blue distribution. The $\alpha$-cross-entropy is constant
for the translation-invariant kernel. It changes for the translation- varying
kernel. The lowest $\alpha$-cross-entropy magnitude is achieved when the means
of the two distributions are equivalent, as shown in (g). This result aligns
with our expectation. It hence demonstrates that translation-varying kernels
should be employed when using either
$C_{\alpha}(\hat{\kappa}_{1}\|\hat{\kappa}_{2})$ or
$C_{\alpha}^{\prime}(\hat{\kappa}_{1}\|\hat{\kappa}_{2})$ if distributional
offsets are to be captured in addition to distributional shape changes.
Figure 3.3: An overview of cross-entropy estimation when considering
distributional shape changes. (a) A two-dimensional scatter plot for samples
drawn from two Gaussian distributions with equivalent variances and means.
(b)–(c) Empirical Gram matrices $\hat{\kappa}_{1}$ and $\hat{\kappa}_{2}$,
respectively, for the blue and red distributions shown in (a). Here a
translation-invariant, Gaussian kernel was used,
$\kappa^{\prime}(s,s^{\prime})\\!=\\!\textnormal{exp}(-\sigma\langle
s\\!-\\!s^{\prime},\cdot\rangle)$, $\sigma\\!=\\!1$. (d)–(e) Empirical Gram
matrices $\hat{\kappa}_{1}$ and $\hat{\kappa}_{2}$, respectively, for the blue
and red distributions shown in (a). Here a translation-varying, exponential-
inner-product kernel was used,
$\kappa^{\prime}(s,s^{\prime})\\!=\\!\textnormal{exp}(\sigma
s^{\top}\\!s^{\prime})$, $\sigma\\!=\\!1$. In (f), we increase the variance of
the samples from the blue distribution in (a). This scale change is captured
by the empirical Gram matrices in (g) and (i). The Gram matrices in (h) and
(j) naturally do not change from (c) and (e), since the red distribution is
not modified. (k)–(l) Plots of the matrix-based Rényi’s $\alpha$-cross-entropy
$C_{\alpha}(\hat{\kappa}_{1}\|\hat{\kappa}_{2})$ as a function of the variance
scale amount of the blue distribution. The results for
$C^{\prime}_{\alpha}(\hat{\kappa}_{1}\|\hat{\kappa}_{2})$ are similar to those
for $C_{\alpha}(\hat{\kappa}_{1}\|\hat{\kappa}_{2})$ for this example and
hence not plotted. Regardless of which kernel is employed, the cross-entropies
reach a minimal value near the unit-scale case. This result aligns well with
our expectations. It hence demonstrates that both translation-invariant and
translation-varying kernels can be used to quantify changes in the
distribution shape.

There is a simple explanation for this discrepancy between expected and
observed behavior. The bipartite, non-mirrored cross-entropy does not account
for the cross-correlation of samples from both distributions. Rather, it
relies on Gram matrices that correspond to autocorrelations of samples from
only a single distribution. If the kernel is insensitive to uniform
translations of the samples, then the Gram matrices, like those in Figures
3.2(b) and (c), will remain constant. The cross-entropy measure will stay the
same too. This is due to the properties imparted by the kernel on the inner
product of the reproducing-kernel Hilbert space. Only for a translation-
varying kernel can the autocorrelation Gram matrices potentially change for
this type of distributional adjustment. This is illustrated in Figures 3.2(e),
(h), and (i).

In Figure 3.3, we consider distributional variance changes. Samples for one
distribution (red) are held fixed while the variance for the samples from the
other distribution (blue) are adjusted uniformly for each dimension. Snapshots
of this process are shown in Figures 3.3(a) and (f). Both distributions have
the same mean. We therefore anticipate that the matrix-based bipartite
$\alpha$-cross-entropy should achieve a maximal value when the variance is
zero and the samples from one distribution collapse to a delta function. It
should monotonically decrease as the variance is raised until the
distributional spreads are equal. Further increases in variance should yield
monotonic increases in the cross-entropy. This occurs in Figure 3.3(k) for the
translation-invariant Gaussian kernel. A similar trend is observed in Figure
3.3(k) for the translation-varying exponential-inner-product kernel. However,
the mirrored cross-entropy does does not achieve the highest value for the
zero-variance case. In both cases, we plot the cross-entropy as a function of
the variance scaling amount and the cross-entropy order.

This example again illustrates the influence of the kernel choice on the
estimated cross-entropy. For the translation-varying kernel that we employed,
properties of the inner product are such that samples which are far from the
population mean are given a greater emphasis than those closer to it. This
creates a few high-magnitude entries in the autocorrelation-based Gram
matrices as the variance scale is increased. Figures 3.3(d) and (i) show this.
These high-magnitude entries dominate the cross-entropy calculation, leading
to increasing cross-entropy values as the scale of one distribution is
continuously raised past the equivalence point. The translation-invariant
kernel that we use does not share this property.

Regardless of the kernel choice, it is important to notice that several
theoretical properties of the matrix-based cross-entropies are upheld in these
examples. We clearly have non-negativity and order monotonicity.

Another observation from these experiments is that the cross-entropy measure
cannot intrinsically detect when there is no distributional overlap. The
measure will simply continue to increase as the samples are shifted farther
apart. This is not a defect of our measure. It is simply a byproduct of
handling continuous distributions using finite samples. Moreover, it may be
possible that, after drawing additional samples, distributional overlap is
achieved, even if the sample means are far apart. This can occur, for
instance, with distributions that have infinite support. It is thus not always
appropriate to conclude that there is no distributional overlap.

## 4\. Tripartite Cross-Entropy

The bipartite cross-entropies that we define have issues that prevent their
use in certain circumstances. For instance, they assume that the sizes of the
empirical Gram matrices are equivalent. They hence codify the relationships
between distributions described by the same number of samples.

In some instances, investigators may wish to assess cross-entropies for
differing-cardinality sets. We therefore introduce a third cross-entropy
measure that overcomes this issue (see Definition 4.1). It is based on our
lab’s notion of the cross-information potential [58]. We refer to this measure
as the tripartite $\alpha$-cross-entropy. This measure utilizes a joint,
bivariate operator that performs cross-correlations of functions. It also
relies on univariate operators.

This measure offers other advantages. Due to the use of a joint, bivariate
operator, translation-invariant kernels can be employed within the tripartite
$\alpha$-cross-entropy to assess distributional shape and shift changes. In
contrast, both bipartite $\alpha$-cross-entropies can only quantify
distributional shape differences when using translation-invariant kernels.
This stems from properties of the non-joint, bivariate operators. Translation-
varying kernels are needed, for these bipartite measures, to quantify
distributional shifts and hence how much they overlap. Some translation-
varying kernels may alter the properties of the inner-product in a way that
yields unexpected cross-entropy behaviors, however. Translation-invariant
kernels, like the Gaussian kernel, tend yield cross-entropy magnitudes that
align better with intuition. We hence would prefer to use them whenever
possible.

We must note, though, that due to its construction, the measure cannot be
readily applied to purely matrix- or graph-based modalities. Rather, it relies
on access to vector-based samples, which are required to empirically
approximate the cross-correlation operator using Gram matrices. Such vector-
based samples can, however, usually be uncovered via multi-dimensional
scaling. Moreover, the pairwise distances between the vector-based samples
often resemble well the original matrix entries. This is thus not an overly
restrictive condition.

For the bipartite cross-entropies, we rely on the spectral consistency of the
approximate Gram operators and Gram matrices to initially motivate our
empirical formulation. For the tripartite cross-entropies, we only have
spectral consistency in some cases (see Definition 4.2 and Proposition 4.1).
This is because we may have differing-cardinality sets, which preclude
eigenanalyses. We therefore primarily rely on minimax optimality of the
measure (see Proposition 4.2) to motivate its use. Our minimax optimality
guarantee is limited currently to just radial universal kernels, though. While
the set of such kernels does encompass many popular choices, such as the
Gaussian kernel, it does not include all universal kernels. It is an open
problem as to if this constraint can be relaxed while still maintaining a
convergence rate that is independent of the sample dimensionality.

## 4.1. Operator-Based Cross-Entropies

For non-commutative Gram operators, we specify the tripartite Rényi’s
$\alpha$-cross-entropy below. We refer to it as such since it has three
operator-based arguments.

  * Definition 4.1: Tripartite Rényi’s $\alpha$-Cross-Entropy. Let $\mathcal{H}_{\kappa_{1}^{\prime}}$, $\mathcal{H}_{\kappa_{2}^{\prime}}$ be separable reproducing-kernel Hilbert spaces, with continuous, universal reproducing kernels $\kappa_{1}^{\prime},\kappa_{2}^{\prime}$. Let $\kappa_{1}\\!:\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\to\\!\mathcal{H}_{\kappa_{1}^{\prime}}$, $\kappa_{2}\\!:\\!\mathcal{H}_{\kappa_{2}^{\prime}}\\!\to\\!\mathcal{H}_{\kappa_{2}^{\prime}}$. Let
$\kappa_{1,2}\\!:\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes^{*}\\!\mathcal{H}_{\kappa_{2}^{\prime}}\\!\to\\!\mathcal{H}_{\kappa_{1}^{\prime}}\\!\otimes^{*}\\!\mathcal{H}_{\kappa_{2}^{\prime}}$.
The tripartite Rényi’s $\alpha$-cross-entropy of $\kappa_{1}$, $\kappa_{1,2}$,
and $\kappa_{2}$, for $\alpha\\!\in\\!\mathbb{R}_{+}\\!\backslash\\{1\\}$, is

$C^{\prime\prime}_{\alpha}(\kappa_{1}\|\kappa_{1,2}\|\kappa_{2})=\frac{1}{\alpha\\!-\\!1}\textnormal{log}\Bigg{(}\\!\mathbb{E}_{(s,s^{\prime})\sim
p_{\mathcal{S}}}[\kappa_{1}]\\!+\\!\mathbb{E}_{(a,a^{\prime})\sim
p_{\mathcal{A}}}[\kappa_{2}]\\!-\\!2\mathbb{E}_{(s,a)\sim\rho_{\mathcal{S}\times\mathcal{A}}}[\kappa_{1,2}]\\!\Bigg{)}\\!+\\!\frac{1}{\alpha\\!-\\!1}\textnormal{log}\Bigg{(}\\!\textnormal{tr}(\kappa_{1}^{\alpha})\\!\Bigg{)},$

for
$\textnormal{supp}(\kappa_{2})\\!\subseteq\\!\textnormal{supp}(\kappa_{1})$,
and
$C^{\prime\prime}_{\alpha}(\kappa_{1}\|\kappa_{1,2}\|\kappa_{2})\\!=\\!\infty$
otherwise. $\kappa_{1,2}\\!=\\!\lambda$ is a joint, bivariate Gram operator.

We take inspiration from Shannon’s cross-entropy for this cross-entropy
measure. The first term, an operator-based cross-information potential, acts
like a divergence. The second term is an operator-based entropy.

Much like the bipartite measures, the tripartite version satisfies the six
axioms that Rényi proposed for divergences. It thus, theoretically, behaves as
a divergence. Practically, it does too. We elaborate on these properties and
others in the appendix (see Appendix A.1).

## 4.2. Empirical Estimation of Operator-Based Cross-Entropies

As with the previous two measures, we approximate the operators. We have
already considered the univariate case, so all that remains is the joint,
bivariate case. The associated operator $\lambda$ can be estimated by
$\hat{\lambda}$ subject to the
constraint
$\mathbb{E}_{s_{i}\sim\hat{p}_{\mathcal{S}},a_{j}\sim\hat{p}_{\mathcal{A}}}[\hat{\lambda}(s_{i},a_{j})]\\!=\\!\lambda$.

  * Definition 4.2: Joint, Bivariate Gram Operator Approximation. An unbiased empirical estimate $\hat{\lambda}(f,g)$, $f\\!\in\\!\mathcal{H}_{\kappa_{1}^{\prime}}$, $g\\!\in\\!\mathcal{H}_{\kappa_{2}^{\prime}}$, of the self-adjoint operator $\lambda$ is

$\hat{\gamma}(f,g)=\\!\int_{\mathcal{S}}\\!\int_{\mathcal{A}}\Bigg{(}\langle
f,\varphi(s_{i})\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}}\langle\varphi(s_{i}),\psi(a_{j})\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\,\mathcal{H}_{\kappa_{2}^{\prime}}}\langle\psi(a_{j}),g\rangle_{\mathcal{H}_{\kappa_{2}^{\prime}}}\\!\Bigg{)}dp_{\mathcal{S}}(s_{i})dp_{\mathcal{A}}(a_{j}),$

which evaluates to $n^{-1}m^{-1}\sum_{i=1}^{n}\sum_{j=1}^{m}\langle
f,\varphi(s_{i})\rangle_{\mathcal{H}_{\kappa^{\prime}_{1}}}\langle\varphi(s_{i}),\psi(a_{j})\rangle_{\mathcal{H}_{\kappa_{1}^{\prime}}\otimes\,\mathcal{H}_{\kappa_{2}^{\prime}}}\langle\psi(a_{j}),g\rangle_{\mathcal{H}_{\kappa^{\prime}_{1}}}$.
Here, $s_{i}\\!\in\\!\mathcal{S}$
and $a_{j}\\!\in\\!\mathcal{A}$, $i\\!\in\\!\mathbb{N}_{1,n}$ and
$j\\!\in\\!\mathbb{N}_{1,n}$ are samples.

We can again substitute, for the Gram operator $\lambda$, a non-negative Gram
matrix $[\hat{\lambda}]_{i,j}\\!\in\\!\mathbb{R}_{0,+}$,
$i\\!\in\\!\mathbb{N}_{1,n}$, $j\\!\in\\!\mathbb{N}_{1,m}$. This provides a
matrix-based estimate of cross-entropy, which is outlined in Algorithms 4.1.

Access to vector-based samples is required to form the joint, bivariate Gram
matrix. For data that naturally exist as either graphs or matrices, an
embedding to a metric space will need to found first. If no metric-space
embedding of the matrix- or graph-based samples is possible, then the joint
operator cannot be formed. Only our bipartite cross-entropy measures can be
applied.

We previously motivated the use of Gram matrices by claiming that they share
spectral properties with the corresponding Gram operators. Below, we do the
same for the joint, bivariate Gram operator in the case where $m\\!=\\!n$.

  * Proposition 4.1: Joint, Bivariate Gram Operator and Gram Matrix Relationship. Let $\hat{\lambda}\\!\in\\!\mathbb{R}_{0,+}^{n\times n}$ be an
empirical Gram matrix,
$[\hat{\lambda}]_{i,j}\\!=\\!\langle\varphi(s_{i}),\psi(a_{j})\rangle_{\mathcal{H}_{\kappa^{\prime}_{1}}\otimes\mathcal{H}_{\kappa^{\prime}_{2}}}$,
for measurable feature maps $\varphi,\psi$. Assume that the
corresponding kernels $\kappa_{1}^{\prime},\kappa_{2}^{\prime}$ are continuous
and universal.

Let $\hat{\lambda}$ be an unbiased empirical estimate of the univariate Gram
operator $\lambda$. $\hat{\lambda}$ has at most $n$ positive eigen-
values that satisfy $[\hat{\lambda}]\beta_{i}\\!=\\!\tau_{i}\beta_{i}$, where
$\tau_{i}\\!\in\\!\mathbb{R}_{+}$ are eigenvalues and
$\beta_{i}\\!\in\\!\mathbb{R}^{n}$ are eigenvectors. As well, $\tau_{i}$ are
all
positive eigenvalues of $[\hat{\lambda}]$.

Data: $\alpha\\!\in\\!\mathbb{R}_{0,+}\\!\backslash\\{1\\}$: Order of the
divergence. Input: $[\hat{\kappa}_{1}]\\!\in\\!\mathbb{R}_{0,+}^{n\times n}$,
$[\hat{\kappa}_{1,2}]\\!\in\\!\mathbb{R}_{0,+}^{n\times m}$,
$[\hat{\kappa}_{2}]\\!\in\\!\mathbb{R}_{0,+}^{m\times m}$: properly
conditioned Gram matrices with the constraint
$\textnormal{supp}([\hat{\kappa}_{2}])\\!\subseteq\\!\textnormal{supp}([\hat{\kappa}_{1}])$.
1 Return $\displaystyle
C^{\prime\prime}_{\alpha}([\hat{\kappa}_{1}]\|[\hat{\kappa}_{1,2}]\|[\hat{\kappa}_{2}])\\!=\\!\frac{1}{\alpha\\!-\\!1}\textnormal{log}\Bigg{(}\\!\mathbb{E}_{(s,s^{\prime})\sim
p_{\mathcal{S}}}[[\hat{\kappa}_{1}]]\\!+\\!\mathbb{E}_{(a,a^{\prime})\sim
p_{\mathcal{A}}}[[\hat{\kappa}_{2}]]\\!-\\!2\mathbb{E}_{(s,a)\sim\rho_{\mathcal{S}\times\mathcal{A}}}[[\hat{\kappa}_{1,2}]]\\!\Bigg{)}\\!+\\!\frac{1}{\alpha\\!-\\!1}\textnormal{log}\Bigg{(}\\!\textnormal{tr}([\hat{\kappa}_{1}]^{\alpha})\\!\Bigg{)}$.
Algorithm 4.3 Tripartite Matrix-Based Rényi’s $\alpha$-Cross-Entropy
$C_{\alpha}^{\prime\prime}([\hat{\kappa}_{1}]\|[\hat{\kappa}_{1,2}]\|[\hat{\kappa}_{2}])$
---

For the bipartite cross-entropy measures, we have spectral consistency results
that justify our use of Gram matrices. For the tripartite measure, we do not
necessarily have them, though. This is because an eigenproblem cannot be
solved for the joint, bivariate Gram matrix, given that the matrix may be
square. We therefore just rely on minimax optimality of the divergence term to
motivate this approximation.

  * Proposition 4.2: Tripartite Cross-Entropy Convergence Properties. Let $\mathcal{H}_{\kappa_{1}^{\prime}},\mathcal{H}_{\kappa_{2}^{\prime}}$ be separable reproducing-kernel Hilbert spaces with continuous, radial, universal reproducing kernels. Let ${\kappa}_{1}$, ${\kappa}_{1,2}$, and ${\kappa}_{2}$ be Gram oper-
ators and $[\hat{\kappa}_{1}]\\!\in\\!\mathbb{R}_{0,+}^{n\times n}$,
$[\hat{\kappa}_{1,2}]\\!\in\\!\mathbb{R}_{0,+}^{n\times m}$, and
$[\hat{\kappa}_{2}]\\!\in\\!\mathbb{R}_{0,+}^{m\times m}$ be the corresponding
Gram matrices, with $n,m\\!\in\\!\mathbb{N}$. We have that

    * (i) $\textnormal{inf}_{[\hat{\kappa}_{1}]}\,\textnormal{sup}_{{\kappa}_{1}}\,|\mathbb{E}_{s_{i},s_{j}\sim p_{\mathcal{S}}}[[\hat{\kappa}_{1}]]\\!-\\!\mathbb{E}_{s,s^{\prime}\sim p_{\mathcal{S}}}[\kappa_{1}]|\\!\geq\\!c_{1}n^{-1/2}$ with non-zero probability, where $c_{1}\\!\in\\!\mathbb{R}_{+}$.

    * (ii) $\textnormal{inf}_{[\hat{\kappa}_{1,2}]}\,\textnormal{sup}_{{\kappa}_{1,2}}\,|\mathbb{E}_{s_{i},a_{j}\sim p_{\mathcal{S}\times\mathcal{A}}}[[\hat{\kappa}_{1,2}]]\\!-\\!\mathbb{E}_{s,a\sim p_{\mathcal{S}\times\mathcal{A}}}[\kappa_{1,2}]|\\!\geq\\!c_{2}(n^{-1/2}\\!+\\!m^{-1/2})$ with non-zero prob-
ability, where $c_{2}\\!\in\\!\mathbb{R}_{+}$.

    * (iii) $\textnormal{inf}_{[\hat{\kappa}_{2}]}\,\textnormal{sup}_{{\kappa}_{2}}\,|\mathbb{E}_{a_{i},a_{j}\sim p_{\mathcal{A}}}[[\hat{\kappa}_{1}]]\\!-\\!\mathbb{E}_{a,a^{\prime}\sim p_{\mathcal{A}}}[\kappa_{2}]|\\!\geq\\!c_{3}m^{-1/2}$ with non-zero probability, where $c_{3}\\!\in\\!\mathbb{R}_{+}$.

This holds regardless of if $m\\!<\\!n$ or $n\\!\leq\\!m$. Therefore, the
Gram-matrix-based version of the tripartite Rényi’s $\alpha$-cross entropy
converges at a minimax-optimal rate for
$\alpha\\!\in\\!\mathbb{R}_{+}\\!\backslash\\{1\\}$.

Again, this claim shows that we can estimate cross-entropy-like quantities
without direct access to the underlying probability distributions.

As with our bipartite cross-entropy measures, the tripartite version converges
at a rate that is independent of the dimensionality of the sample
dimensionality when using radial kernels. For non-radial kernels, we obtain a
sub-optimal convergence rate that depends on the sample dimensionality.

## 4.3. Numerical Examples

Figure 4.1: An overview of the cross-entropy estimation when considering
distributional shape changes and shifts. In (a)–(c), we plot the Gram matrices
$\hat{\kappa}_{1}$, $\hat{\kappa}_{1,2}$, and $\hat{\kappa}_{2}$,
respectively, obtained for samples from two Gaussian distributions with
equivalent means and variances. Here a translation-invariant, Gaussian kernel
was used, $\kappa^{\prime}(s,s^{\prime})\\!=\\!\textnormal{exp}(-\sigma\langle
s\\!-\\!s^{\prime},\cdot\rangle)$, $\sigma\\!=\\!1$. We then shifted the means
of both distributions so that they were further apart. As shown in (d) and
(f), the intra-distributional Gram matrices, $\hat{\kappa}_{1}$ and
$\hat{\kappa}_{2}$, remain the same. The inter-distributional Gram matrix,
$\hat{\kappa}_{1,2}$, changes to reflect this mean shift. In (g), we find that
the $\alpha$-cross-entropy
$C_{\alpha}(\hat{\kappa}_{1}\|\hat{\kappa}_{1,2}\|\hat{\kappa}_{2})$ changes
according to the shift amount. This result aligns well with our expectations.
In (h)–(n), we consider the case where the variance for one of the
distributions changes. Increasing the variance leads to changes in both
$\hat{\kappa}_{1}$ and $\hat{\kappa}_{1,2}$, which can be seen by comparing
(h) and (i) to (k) and (l). The Gram matrix $\hat{\kappa}_{2}$ remains the
same in (j) and (m), since it is not modified. In (n), we find that the
$\alpha$-cross-entropy
$C_{\alpha}(\hat{\kappa}_{1}\|\hat{\kappa}_{1,2}\|\hat{\kappa}_{2})$ changes
according to the variance scale amount. This result aligns well with our
expectations. The plots in (g) and (n) also show that the $\alpha$-cross-
entropy is monotonically increasing with respect to $\alpha$. Taken together,
these results indicate that
$C_{\alpha}(\hat{\kappa}_{1}\|\hat{\kappa}_{1,2}\|\hat{\kappa}_{2})$ overcomes
the issues associated with $C_{\alpha}(\hat{\kappa}_{1}\|\hat{\kappa}_{2})$
and $C_{\alpha}^{\prime}(\hat{\kappa}_{1}\|\hat{\kappa}_{2})$ when
translation-invariant kernels are employed.

Distribution Assessment Analysis. While the bipartite cross-entropy measures
can assess both distributional shift and scale changes, their magnitudes do
not always align well with intuition depending on the kernel type. We show
that the tripartite measure does not share this shortcoming. Examples are
presented in Figure 4.1.

In Figure 4.1, we consider both distributional mean and variance shifts. We
draw samples from two-dimensional Gaussian distributions. For the mean-shift
experiments, samples for one distribution are held fixed while the mean for
the samples for the other distribution are shifted. The distribution variances
are the same. For the variance-shift experiments, we artificially scale the
distributional spread for one distribution. The means are kept the same. The
same number of samples are drawn from each distribution. As with the
experiments for the bipartite measures, for the mean-shift experiments, we
expect the tripartite measure to achieve a minimum whenever the distributional
means are equal. The tripartite cross-entropy should monotonically increase as
the means diverge and hence the distributions have increasingly minimal
overlap. As well, for the variance-scale experiments, we expect the tripartite
measure to achieve a minimum when the variances are equivalent and then
monotonically increase according to the scale amount. These behaviors are
present in Figure 4.1(g) and (n) when using the translation-invariant Gaussian
kernel.

For both the mean-shift and variance-scale experiments, the observed behavior
is a byproduct of using a joint, bivariate Gram matrix within the measure.
This matrix corresponds to cross-correlations of functions. It hence captures
inter-distributional sample-distance changes, even for translation-invariant
kernels, as highlighted in Figures 4.1(b) and (e). Moreover, such a matrix
provides ancillary details about distributional shape differences, again due
to capturing inter-distributional distance changes. This can be seen in
Figures 4.1(i) and (l). The univariate Gram matrices, in contrast, correspond
to autocorrelations of functions and hence intra-distributional sample
distances. Such matrices do not change with global translations, as shown in
Figures 4.1(a) and (c) along with Figures 4.1(d) and (e), when using
translation-invariant kernels, which is due to properties of the induced inner
product. These matrices are only sensitive to local distance modifications and
hence distributional shape changes. This is apparent from Figures 4.1(h) and
(j) along with Figures 4.1(k) and (m).

## 5 Comparisons

The matrix-based $\alpha$-cross-entropies that we have defined resemble
quantum-information-theoretic counterparts. Our non-mirrored $\alpha$-cross-
entropy is analogous to Petz’s original formulation of a quantum Rényi
relational entropy for stochastic matrices [59, 60]. Our mirrored variant is
also related to the minimal divergence of Müller-Lennert et al. [61] for
matrices. A variety of properties have been demonstrated for these quantum
measures [62, 63, 64, 65] and many have shown how to use these measures to
generalize classical notions from information theory to the quantum case [66,
67]. We refer readers to [68, 69, 70, 71] for additional work on quantum
information theory.

There is, however, a fundamental distinction between our work and theirs. In
the quantum setting, the operators correspond to positive-semi-definite
density matrices. Entries of these matrices are mixtures of state vectors that
a system may assume. These divergences hence assess differences between
quantum states in a Hilbert space. In our case, the operators quantify
correlations between functions. We construct empirical versions of these
operators, in the form of Gram matrices, which correspond to mappings of the
finite sample set to a separable, reproducing-kernel Hilbert space of
functions. Due to the connections of Gram matrices with mean-element maps, we
are assured that our cross-entropies are assessing differences between
probability measures without needing direct knowledge of them. This behavior
is not present in the quantum-information-theoretic case. This is because the
random matrices are not formed from the application of kernels to the samples
and hence do not characterize embeddings of distributions. As a consequence of
this distinction, the matrix-based quantum divergences cannot be applied for
statistical testing [72] in the non-quantum case [73]. Our matrix-based cross-
entropies can. Our measures can also be readily extended to multiple
variables, which offers similar functionality to [74].

Our matrix-based cross-entropies thus are an entirely different formulation
that have only superficial connections to the quantum relative entropies. As
well, our measures do not appear to stem from any other operator-theoretic
concepts, such as either delta divergences, log-determinant divergences [75],
or log-Hilbert-Schmidt metrics [76], all of which are defined for Gram
matrices. Our contributions thus appear to provide the first matrix-based,
unbiased, non-parametric estimators for Rényi’s cross-entropy. Additionally,
our measures can be transformed to specify Rényi-divergence-like quantities,
along with quantities associated with special cases, like the Kullback-Leibler
divergence. Our measure does not require distributional assumptions to assess
distributional overlap. It thus is suitable for general statistical testing of
arbitrary distributions. Previous definitions of non-quantum, matrix-based
divergences have typically been limited to embeddings of Gaussian
distributions [77]. No convergence and convergence-rate guarantees have been
provided. Moreover, the measures defined in [77] can only handle sample sets
with equivalent cardinalities. While our bipartite measures share the same
trait, our tripartite measure does not.

## 6\. Conclusions

In this paper, we propose a functional-operator formulation of Rényi’s
$\alpha$-cross-entropy. We approximate the underlying operators using Gram-
matrix-based representations. This yields Rényi’s $\alpha$-cross-entropies
suitable for kernelized matrix- and graph-based data. The $\alpha$-cross-
entropies can also be applied to vector-based data that are converted into
Gram matrices through the application of a kernel to vector pairs.

We prove that the matrix-based $\alpha$-cross-entropies satisfy the axiomatic
properties that Rényi established for divergences. Our criteria thus, in
theory, can act as divergences. It is appropriate to consider them as
divergences for matrix and graph samples that do not have metric-space vector
realizations. For vector samples converted into Gram matrices, though, our
bipartite $\alpha$-cross-entropies should not be viewed as divergences, even
though their expressions resemble the classical case. This is because
characteristics of our $\alpha$-cross-entropies do not permit effective
statistical testing in this case. They, for instance, can only compare
matrices of the same size. As well, certain kernel choices may not
simultaneously quantify shape and positional differences in the functional-
space-embedded distributions.

Due to these shortcomings, we also offer a tripartite, matrix-based Rényi’s
$\alpha$-cross-entropy. This criterion innately overcomes all of the
aforementioned issues of our bipartite $\alpha$-cross-entropies, regardless of
if the kernels are either translation varying or translation invariant. It
hence is more appealing for statistical testing. This cross-entropy, however,
can only be applied for graphs and matrices that have a vector realization. If
no realization exists, then our $\alpha$-cross-entropies must be used. This is
because our $\alpha$-divergence requires the approximation of a joint cross-
covariance operator between sample sets, which can only be constructed if
vector samples are available. Our bipartite $\alpha$-cross entropies, in
contrast, require only the approximation of marginal covariance operators that
are obtained from each set independently. The approximations for the marginal
operators can be formed for either vector samples or matrix-based
representations.

A major advantage of our cross-entropy measures is that the convergence rate
for the operator approximation depends only on the number of samples, not the
sample dimensionality. This property allows practitioners to consider
assessing distributional overlap for high-dimensional samples where existing
plug-in estimators, like Parzen windows, would be largely ineffective. For the
bipartite measures, this convergence-rate guarantee is available for all
universal kernels and arbitrary distributions. For the tripartite measure, we
only have a dimensionally-agnostic error rate for radial universal kernels
applied to samples from arbitrary distributions. In either case, we have
minimax optimality. We thus avoid slow-convergence-rate issues that are
prevalent for plug-in density estimators.

## References

  * [1] R. L. Stratonovich, _Information Theory_. Moscow, Soviet Union: Sovetskoe Radio, 1975.
  * [2] T. M. Cover and J. A. Thomas, _Elements of Information Theory_. New York, NY, USA: John Wiley and Sons, 2006.
  * [3] J. Ziv and N. Merhav, “A measure of relative entropy between individual sequences with application to universal classification,” _IEEE Transactions on Information Theory_ , vol. 39, no. 4, pp. 1270–1279, 1933. Available: http://dx.doi.org/10.1109/18.243444
  * [4] G. A. Darbellay and I. Vajda, “Estimation of the information by an adaptive partitioning of the observation space,” _IEEE Transactions on Information Theory_ , vol. 45, no. 4, pp. 1315–1321, 1999. Available: http://dx.doi.org/10.1109/18.761290
  * [5] Q. Wang, S. R. Kulkarni, and S. Verdu, “Divergence estimation of continuous distributions based on data-dependent partitions,” _IEEE Transactions on Information Theory_ , vol. 51, no. 9, pp. 3064–3074, 2005. Available: http://dx.doi.org/10.1109/TIT.2005.853314
  * [6] H. Cai, S. R. Kulkarni, and S. Verdu, “Universal divergence estimation for finite-alphabet sources,” _IEEE Transactions on Information Theory_ , vol. 52, no. 8, pp. 3456–3475, 2006. Available: http://dx.doi.org/10.1109/TIT.2006.878182
  * [7] J. C. Príncipe, _Information Theoretic Learning_. New York City, NY, USA: Springer-Verlag, 2010.
  * [8] L. G. Sanchez Giraldo and J. C. Príncipe, “Information theoretic learning with infinitely divisible kernels,” in _Proceedings of the International Conference on Learning Representations (ICLR)_ , Scottsdale, AZ, USA, May 2-4 2013, pp. 1–11. Available: https://arxiv.org/abs/1301.3551
  * [9] L. G. Sanchez Giraldo, M. Rao, and J. C. Príncipe, “Measures of entropy from data using infinitely divisible kernels,” _IEEE Transactions on Information Theory_ , vol. 61, no. 1, pp. 535–548, 2014. Available: http://dx.doi.org/10.1109/TIT.2014.2370058
  * [10] S. Yu, L. G. Sánchez Giraldo, R. Jenssen, and J. C. Príncipe, “Multivariate extensions of matrix-based Rényi’s $\alpha$-order entropy functional,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 42, no. 11, pp. 2960–2966, 2020. Available: http://dx.doi.org/10.1109/TPAMI.2019.2932976
  * [11] D. Pál, B. Póczos, and C. Szepesvári, “Estimation of Rényi entropy and mutual information based on generalized nearest-neighbor graphs,” in _Advances in Neural Information Processing Systems (NIPS)_ , J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, Eds. Red Hook, NY, USA: Curran Associates, 2010, pp. 1849–1857.
  * [12] A. O. Hero and O. J. J. Michel, “Asymptotic theory of greedy approximations to minimal k-point random graphs,” _IEEE Transactions on Information Theory_ , vol. 46, no. 6, pp. 1921–1938, 1999. Available: http://dx.doi.org/10.1109/18.782114
  * [13] Q. Wang, S. R. Kulkarni, and S. Verdu, “Divergence estimation for multidimensional densities via $k$-nearest-neighbor distances,” _IEEE Transactions on Information Theory_ , vol. 55, no. 5, pp. 2392–2405, 2009. Available: http://dx.doi.org/10.1109/TIT.2009.2016060
  * [14] K. Fukumizu, G. R. G. Lanckriet, and B. K. Sriperumbudur, “Learning in Hilbert vs. Banach spaces: A measure embedding viewpoint,” in _Advances in Neural Information Processing Systems (NIPS)_ , J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Q. Weinberger, Eds. Red Hook, NY, USA: Curran Associates, 2011, pp. 1773–1781.
  * [15] B. K. Sriperumbudur, A. Gretton, K. Fukumizu, G. R. G. Lanckriet, and B. Schölkopf, “Injective Hilbert space embeddings of probability measures,” in _Proceedings of the Conference on Learning Theory (COLT)_ , Helsinki, Finland, July 9-12 2008, pp. 111–122.
  * [16] A. Christmann and I. Steinwart, “Universal kernels on non-standard input spaces,” in _Advances in Neural Information Processing Systems (NIPS)_ , J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, Eds. Red Hook, NY, USA: Curran Associates, 2011, pp. 406–414.
  * [17] K. Fukumizu, A. Gretton, G. R. G. Lanckriet, B. Schölkopf, and B. K. Sriperumbudur, “Characteristic kernels on groups and semigroups,” in _Advances in Neural Information Processing Systems (NIPS)_ , D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, Eds. Red Hook, NY, USA: Curran Associates, 2009, pp. 473–480.
  * [18] Y. Nishiyama and K. Fukumizu, “Characteristic kernels and infinitely divisible distributions,” _Journal of Machine Learning Research_ , vol. 17, no. 1, pp. 1–28, 2016.
  * [19] B. K. Sriperumbudur, K. Fukumizu, and G. R. G. Lanckriet, “On the relation between universality, characteristic kernels and RKHS embedding of measures,” in _Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS)_ , Sardinia, Italy, May 13-15 2010, pp. 773–780.
  * [20] ——, “Universality, characteristic kernels and RKHS embedding of measures,” _Journal of Machine Learning Research_ , vol. 12, no. 1, pp. 2389–5410, 2011.
  * [21] K. P. Chwialkowski, D. Sejdinovic, and A. Gretton, “A wild bootstrap for degenerate kernel tests,” in _Advances in Neural Information Processing Systems (NIPS)_ , Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger, Eds. Red Hook, NY, USA: Curran Associates, 2014, pp. 3608–3616.
  * [22] A. Gretton, K. Borgwardt, M. Rasch, B. Schölkopf, and A. J. Smola, “A kernel method for the two-sample problem,” in _Advances in Neural Information Processing Systems (NIPS)_ , B. Schölkopf, J. Platt, and T. Hoffman, Eds. Red Hook, NY, USA: Curran Associates, 2007, pp. 513–520.
  * [23] M. Hein and O. Bousquet, “Hilbertian metrics and positive definite kernels on probability measures,” in _Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS)_ , Bridgetown, Barbados, January 6-8 2005, pp. 136–143.
  * [24] K. Fukumizu, A. Gretton, X. Sun, and B. Schölkopf, “Kernel measures of conditional dependence,” in _Advances in Neural Information Processing Systems (NIPS)_ , J. Platt, D. Koller, Y. Singer, and S. T. Roweis, Eds. Red Hook, NY, USA: Curran Associates, 2008, pp. 489–496.
  * [25] M. H. Quang, M. S. Biagio, and V. Murino, “Log-Hilbert-Schmidt metric between positive definite operators on Hilbert spaces,” in _Advances in Neural Information Processing Systems (NIPS)_ , Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger, Eds. Red Hook, NY, USA: Curran Associates, 2014, pp. 388–396.
  * [26] K. Chwialkowski and A. Gretton, “A kernel independence test for random processes,” in _Proceedings of the International Conference on Machine Learning (ICML)_ , Beijing, China, June 21-26 2014, pp. 1422–1430. Available: https://arxiv.org/abs/1402.4501
  * [27] D. Sejdinovic, B. K. Sriperumbudur, A. Gretton, and K. Fukumizu, “Equivalence of distance-based and RKHS-based statistics in hypothesis testing,” _Annals of Statistics_ , vol. 41, no. 5, pp. 2263–2291, 2013. Available: http://dx.doi.org/10.1214/13-AOS1140
  * [28] J. T.-Y. Kwok and I. W.-H. Tsang, “The pre-image problem in kernel methods,” _IEEE Transactions on Neural Networks_ , vol. 15, no. 6, pp. 1517–1525, 2004\. Available: http://dx.doi.org/10.1109/TNN.2004.837781
  * [29] L. Song, X. Zhang, A. J. Smola, A. Gretton, and B. Schölkopf, “Tailoring density estimation via reproducing kernel moment matching,” in _Proceedings of the International Conference on Machine Learning (ICML)_ , Helsinki, Finland, July 5-9 2008, pp. 992–999. Available: http://dx.doi.org/10.1145/1390156.1390281
  * [30] Y. Chen, M. Welling, and A. J. Smola, “Super-samples from kernel herding,” in _Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI)_ , Catalina Island, CA, USA, July 8-11 2010, pp. 109–116. Available: https://arxiv.org/abs/1203.3472
  * [31] F. Huszár and D. Duvenaud, “Optimally-weighted herding is Bayesian quadrature,” in _Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI)_ , Catalina Island, CA, USA, August 15-17 2012, pp. 377–386. Available: https://arxiv.org/abs/1204.1664
  * [32] M. Kanagawa and K. Fukumizu, “Recovering distributions from Gaussian RKHS embeddings,” in _Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS)_ , Reykjavik, Iceland, April 22-25 2014, pp. 457–465.
  * [33] M. Kanagawa, B. K. Sriperumbudur, and K. Fukumizu, “Convergence guarantees for kernel-based quadrature rules in misspecified settings,” in _Advances in Neural Information Processing Systems (NIPS)_ , D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, Eds. Red Hook, NY, USA: Curran Associates, 2016, pp. 3296–3304.
  * [34] B. K. Sriperumbudur, K. Fukumizu, A. Gretton, B. Schölkopf, and G. R. G. Lanckriet, “On the empirical estimation of integral probability metrics,” _Electronic Journal of Statistics_ , vol. 6, no. 1, pp. 1550–1599, 2012. Available: http://dx.doi.org/10.1214/12-EJS722
  * [35] B. K. Sriperumbudur, A. Gretton, K. Fukumizu, B. Schölkopf, and G. R. G. Lanckriet, “Hilbert space embeddings and metrics on probability measures,” _Journal of Machine Learning Research_ , vol. 11, no. 1, pp. 1517–1561, 2012\.
  * [36] K. Fukumizu, A. Gretton, G. R. G. Lanckriet, B. Schölkopf, and B. K. Sriperumbudur, “Kernel choice and classifiability for RKHS embeddings of probability distributions,” in _Advances in Neural Information Processing Systems (NIPS)_ , Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, and A. Culotta, Eds. Red Hook, NY, USA: Curran Associates, 2010, pp. 1750–1758.
  * [37] A. Rényi, “On measures of entropy and information,” in _Selected Papers of Alfréd Rényi_ , P. Turán, Ed. Budapest, Turkey: Akadémiai Kaidó, 1976, vol. 2, pp. 565–580.
  * [38] T. van Erven and P. Harremoës, “Rényi divergence and Kullback-Leibler divergence,” _IEEE Transactions on Information Theory_ , vol. 60, no. 7, pp. 3797–3820, 2014. Available: http://dx.doi.org/10.1109/TIT.2014.2320500
  * [39] A. Teixeira, A. Matos, and L. Antunes, “Conditional Rényi entropies,” _IEEE Transactions on Information Theory_ , vol. 58, no. 7, pp. 4273–4277, 2012. Available: http://dx.doi.org/10.1109/TIT.2012.2192713
  * [40] R. A. Horn, “The theory of infinitely divisible matrices and kernels,” _Transactions of the American Mathematical Society_ , vol. 136, pp. 269–286, 1969. Available: http://dx.doi.org/10.2307/1994714
  * [41] C. Baker, “Joint measures and cross-covariance operators,” _Transactions of the American Mathematical Society_ , vol. 186, pp. 273–289, 1973. Available: http://dx.doi.org/10.2307/1996566
  * [42] I. Tolstikhin, B. K. Sriperumbudur, and K. Muandet, “Minimax estimation of kernel mean embeddings,” _Journal of Machine Learning Research_ , vol. 18, no. 1, pp. 1–47, 2017.
  * [43] N. Aronszajn, “Theory of reproducing kernels,” _Transactions of the American Mathematical Society_ , vol. 68, no. 3, pp. 337–404, 1950. Available: http://dx.doi.org/10.2307/1990404
  * [44] Z. Szabó and B. K. Sriperumbudur, “Characteristic and universal tensor product kernels,” _Journal of Machine Learning Research_ , vol. 18, no. 1, pp. 1–29, 2018. Available: https://arxiv.org/abs/1708.08157
  * [45] H. Owhadi and C. Scovel, “Separability of reproducing kernel spaces,” _Proceedings of the American Mathematical Society_ , vol. 145, no. 5, pp. 2131–2138, 2017. Available: http://doi.org/10.1090/proc/13354
  * [46] Y. Li and R. E. Turner, “Rényi divergence variational inference,” in _Advances in Neural Information Processing Systems (NIPS)_ , D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, Eds. Red Hook, NY, USA: Curran Associates, 2016, pp. 1073–1081.
  * [47] C. J. Stone, “Optimal rates of convergence for nonparametric estimators,” _Annals of Statistics_ , vol. 8, no. 6, pp. 1348–1360, 1980. Available: http://dx.doi.org/10.1214/aos/1176345206
  * [48] A. Gretton, D. Sejdinovic, H. Strathmann, S. Balakrishnan, M. Pontil, K. Fukumizu, and B. K. Sriperumbudur, “Optimal kernel choice for large-scale two-sample tests,” in _Advances in Neural Information Processing Systems (NIPS)_ , F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Red Hook, NY, USA: Curran Associates, 2012, pp. 1205–1213.
  * [49] A. Gretton, K. Fukumizu, Z. Harchaoui, and B. K. Sriperumbudur, “A fast, consistent kernel two-sample test,” in _Advances in Neural Information Processing Systems (NIPS)_ , Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, and A. Culotta, Eds. Red Hook, NY, USA: Curran Associates, 2009, pp. 673–681.
  * [50] A. Gretton, K. Borgwardt, M. Rasch, B. Schölkopf, and A. J. Smola, “A kernel two-sample test,” _Journal of Machine Learning Research_ , vol. 13, no. 1, pp. 723–773, 2012.
  * [51] M. Eric, F. Bach, and Z. Harchaoui, “Testing for homogeneity with kernel Fisher discriminant analysis,” in _Advances in Neural Information Processing Systems (NIPS)_ , J. Platt, D. Koller, Y. Singer, and S. T. Roweis, Eds. Red Hook, NY, USA: Curran Associates, 2008, pp. 609–616.
  * [52] A. Argyriou, R. Hauser, C. A. Micchelli, and M. Pontil, “A DC-programming algorithm for kernel selection,” in _Proceedings of the International Conference on Machine Learning (ICML)_ , Pittsburgh, PA, USA, June 25-29 2006, pp. 41–48. Available: http://dx.doi.org/10.1145/1143844.1143850
  * [53] M. Kloft, U. Brefeld, P. Laskov, K.-R. Müller, A. Zien, and S. Sonnenburg, “An extended level method for efficient multiple kernel learning,” in _Advances in Neural Information Processing Systems (NIPS)_ , D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, Eds. Red Hook, NY, USA: Curran Associates, 2009, pp. 1825–1832.
  * [54] ——, “Efficient and accurate $\ell_{p}$-norm multiple kernel learning,” in _Advances in Neural Information Processing Systems (NIPS)_ , Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, and A. Culotta, Eds. Red Hook, NY, USA: Curran Associates, 2010, pp. 997–1005.
  * [55] A. Antos and I. Kontoyiannis, “Convergence properties of functional estimates for discrete distributions,” _Random Structures and Algorithms_ , vol. 19, no. 3-4, pp. 163–193, 2001. Available: http://dx.doi.org/10.1002/rsa.10019
  * [56] L. Birgé, “On estimating a density using Hellinger distance and some other strange facts,” _Probability Theory and Related Fields_ , vol. 710, no. 1, pp. 271–291, 1986. Available: http://dx.doi.org/10.1007/BF00332312
  * [57] L. Devroye, “Another proof of a slow convergence result of Birgé,” _Statistics and Probability Letters_ , vol. 23, no. 1, pp. 63–67, 1995. Available: http://dx.doi.org/10.1016/0167-7152(94)00095-P
  * [58] J.-W. Xu, A. R. C. Paiva, I. Park, and J. C. Príncipe, “A reproducing kernel Hilbert space framework for information-theoretic learning,” _IEEE Transactions on Signal Processing_ , vol. 56, no. 12, pp. 5891–5902, 2008. Available: http://dx.doi.org/10.1109/TSP.2008.2005085
  * [59] D. Petz, “Quasi-entropies for finite quantum systems,” _Reports on Mathematical Physics_ , vol. 23, no. 1, pp. 57–65, 1986. Available: http://dx.doi.org/10.1016/0034-4877(86)90067-4
  * [60] F. Hiai, M. Mosonyi, D. Petz, and C. Bény, “Quantum $f$-divergences and error correction,” _Reviews in Mathematical Physics_ , vol. 23, no. 7, pp. 691–747, 2011. Available: http://dx.doi.org/10.1142/S0129055X11004412
  * [61] L. Müller-Lennert, D. Dupuis, O. Szehr, S. Fehr, and M. Tomamichel, “On quantum Rényi entropies: A new generalization and some properties,” _Journal of Mathematical Physics_ , vol. 54, no. 1, pp. 122 203(1–20), 2013\. Available: http://dx.doi.org/10.1063/1.4838856
  * [62] D. Petz, “Monotonicity of quantum relative entropy revisited,” _Reviews of Mathematical Physics_ , vol. 15, no. 1, pp. 79–91, 2003. Available: http://dx.doi.org/10.1142/S0129055X03001576
  * [63] S. Beigi, “Sandwiched Rényi divergence satisfies data processing inequality,” _Journal of Mathematical Physics_ , vol. 54, no. 1, pp. 122 202(1–12), 2013. Available: http://dx.doi.org/10.1063/1.4838855
  * [64] R. L. Frank and E. H. Lieb, “Montonicity of a relative Rényi entropy,” _Journal of Mathematical Physics_ , vol. 54, no. 1, pp. 122 201(1–5), 2013\. Available: http://dx.doi.org/10.1063/1.4838835
  * [65] D. Sutter, M. Tomamichel, and A. W. Harrow, “Strengthened monotonicity of relative entropy via pinched Petz recovery map,” _IEEE Transactions on Information Theory_ , vol. 62, no. 5, pp. 2907–2913, 2016. Available: http://dx.doi.org/10.1109/TIT.2016.2545680
  * [66] M. Tomamichel, R. Colbeck, and R. Renner, “A fully quantum asymptotic equipartition property,” _IEEE Transactions on Information Theory_ , vol. 55, no. 12, pp. 5840–5847, 2009. Available: http://dx.doi.org/10.1109/TIT.2009.2032797
  * [67] M. Mosonyi and F. Hiai, “On the quantum Rényi relative entropies and related capacity formulas,” _IEEE Transactions on Information Theory_ , vol. 57, no. 4, pp. 2474–2487, 2011. Available: http://dx.doi.org/10.1109/TIT.2011.2110050
  * [68] M. B. Ruskai, “Inequalities for quantum entropy: A review with conditions for equality,” _Journal of Mathematical Physics_ , vol. 43, no. 1, pp. 4358(1–18), 2002. Available: http://dx.doi.org/10.1063/1.1497701
  * [69] L. Zhang and J. Wu, “A lower bound of quantum conditional mutual information,” _Journal of Physics A: Mathematical and Theoretical_ , vol. 47, no. 1, pp. 415 303(1–11), 2014. Available: http://dx.doi.org/10.1088/1751-8113/47/41/415303
  * [70] O. Fawzi and R. Renner, “Quantum conditional mutual information and approximate Markov chains,” _Communications in Mathematical Physics_ , vol. 340, no. 1, pp. 575–611, 2015. Available: http://dx.doi.org/10.1007/s00220-015-2466-x
  * [71] P. Erker, “How not to Rényi-generalize the quantum conditional mutual information,” _Journal of Physics A: Mathematical and Theoretical_ , vol. 48, no. 1, pp. 275 303(1–9), 2015. Available: http://dx.doi.org/10.1088/1751-8113/48/27/275303
  * [72] A. Gretton and L. Györfi, “Consistent nonparametric tests of independence,” _Journal of Machine Learning Research_ , vol. 11, no. 1, pp. 1391–1423, 2011.
  * [73] M. Mosonyi and T. Ogawa, “Quantum hypothesis testing and the operational interpretation of the quantum Rényi relative entropies,” _Communications in Mathematical Physics_ , vol. 334, no. 1, pp. 1617–1648, 2015. Available: http://dx.doi.org/10.1007/s00220-014-2248-x
  * [74] D. Sejdinovic, A. Gretton, and W. Bergsma, “A kernel test for three-variable interactions,” in _Advances in Neural Information Processing Systems (NIPS)_ , C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, Eds. Red Hook, NY, USA: Curran Associates, 2013, pp. 637–646.
  * [75] S. Sra, “A new metric on the manifold of kernel matrices with application to matrix geometric means,” in _Advances in Neural Information Processing Systems (NIPS)_ , F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Red Hook, NY, USA: Curran Associates, 2012, pp. 144–152.
  * [76] H. Q. Minh, M. S. Siagio, and V. Murino, “Log-Hilbert-Schmidt metric between positive definite operators on Hilbert spaces,” in _Advances in Neural Information Processing Systems (NIPS)_ , Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger, Eds. Red Hook, NY, USA: Curran Associates, 2014, pp. 388–396.
  * [77] H. Q. Minh, “Infinite-dimensional log-determinant divergences between positive definite trace class operators,” _Linear Algebra and its Applications_ , vol. 528, no. 1, pp. 331–383, 2017. Available: http://dx.doi.org/10.1016/j.laa.2016.09.018

## Appendix A

## A.1. Matrix-based Rényi’s $\alpha$-Cross-Entropy Properties

In what follows, we demonstrate that our matrix-based Rényi’s $\alpha$-cross-
entropies possess the same characteristics as the classical Rényi’s
$\alpha$-divergences despite not directly having access to probability
distributions. Their quantities thus have similar interpretations.

Throughout, we assume that $\kappa_{1},\kappa_{2}$ are non-zero, positive-
semi-definite, univariate Gram operators for reproducing-kernel Hilbert spaces
$\mathcal{H}_{\kappa_{1}^{\prime}}$ and $\mathcal{H}_{\kappa_{2}^{\prime}}$.
We also assume that
$[\hat{\kappa}_{1}],[\hat{\kappa}_{2}]\\!\in\\!\mathbb{R}^{n\times n}_{0,+}$
are corresponding normalized Gram
matrices. Likewise, $\kappa_{1,2}$ is a joint, bivariate Gram operator that is
non-zero and $[\hat{\kappa}_{1,2}]\\!\in\\!\mathbb{R}^{n\times n}_{0,+}$ is
the associated Gram matrix. We sometimes relax the assumption that the Gram
matrices are square.

Non-negativity. First, we show that the matrix-based $\alpha$-cross-entropies
are non-negative in practical settings. The cross-entropies are zero when the
two arguments are equivalent.

  * Proposition A.1. If $\textnormal{tr}([\hat{\kappa}_{1}])\\!\geq\\!\textnormal{tr}([\hat{\kappa}_{2}])$, then $C_{\alpha}([\hat{\kappa}_{1}]\|[\hat{\kappa}_{2}])\\!\geq\\!0$. If $[\hat{\kappa}_{1}]\\!=\\![\hat{\kappa}_{2}]$, and hence $\textnormal{tr}([\hat{\kappa}_{1}])\\!=\\!\textnormal{tr}([\hat{\kappa}_{2}])$,
then $C_{\alpha}([\hat{\kappa}_{1}]\|[\hat{\kappa}_{2}])\\!=\\!0$. The same
results hold in the mirrored case. Non-negativity similarly holds in the
tripartite case for square and non-square Gram matrices, whereas nullity holds
only for square matrices.

A stronger statement, that the $\alpha$-cross-entropy is zero if and only if
$[\hat{\kappa}_{1}]\\!=\\![\hat{\kappa}_{2}]$, with
$[\hat{\kappa}_{1}],[\hat{\kappa}_{2}]\\!\succeq\\!0$ and
$\textnormal{tr}([\hat{\kappa}_{1}])\\!\geq$
$\textnormal{tr}([\hat{\kappa}_{2}])$, is also possible. Demonstrating this
claim relies on an application of the inequalities in Proposition A.9.

For matrices that are not positive (semi-)definite, there is no guarantee that
the matrix-based $\alpha$-cross-entropies are non-negative. Fortunately, this
cannot occur in practice, as the cross-entropy arguments are Gram matrices.