# Information Theory and Point Processes

Ronald Mahler, Random Sets LLC, Eagan, MN, U.S.A

###### Abstract

This paper addresses theoretically correct vs. incorrect ways to apply
information theory to point processes.

##  I Introduction

Point proceess (p.p.) theory addresses the statistical behavior of randomly
varying finite ensembles of points in some state space [1]. There are multiple
formulations of the theory; here the focus will be on the vector-based
version. The purpose of the paper is to address theoretically correct vs.
incorrect ways of devising information-theoretic functionals for vector-based
p.p.’s.

Several such functionals were recently introduced in [2]. In what follows it
will be demonstrated that the theoretical approach in that paper is incorrect
and that, as a consequence, Eqs. (42-54) of [2] are mathematically undefined
when employed in typical engineering applications. In particular, in these
applications the key Eqs. (42,47,51) involve summations of terms with
incommensurable units of measurement; and this remains the case even if the
theoretically correct approach is applied instead.

The paper is organized as follows: summary of the approach in [2] (Section
II); description of the correct approach (Section III); and critique of the
mathematically undefined formulas (Section IV).

##  II Background

###  II-A Vector-Based Point Processes

Let $\mathfrak{X}$ be a topological space (hereafter referred to as the “base
space”), endowed with a measure $\lambda_{\mathfrak{X}}(B)$ (hereafter
referred to as the “base measure”) defined on the Borel-measurable subsets $B$
of $\mathfrak{X}$. In typical engineering applications $\mathfrak{X}$ is a
region of a Euclidean space with unit of measurement $\iota$, which is what
will be assumed hereafter. In this case $\lambda_{\mathfrak{X}}$ is Lebesgue
measure and the unit of measurement of $\lambda_{\mathfrak{X}}(B)$ is $\iota$.

According to [2], p. 1157, a p.p. is a random variable (r.v.) $\Phi$ on the
disjoint-union space $\mathfrak{X}^{\infty}=\biguplus_{n\geq
0}\mathfrak{X}^{n}$ of finite ordered sequences $\varphi=(x_{1},...,x_{n})$
with $x_{1},...,x_{n}\in\mathfrak{X}$; and where for $n=0$, $\varphi=\phi$
denotes the empty sequence; and where we define $|\varphi|=n$. Thus
$\mathfrak{X}^{\infty}$ is the topological sum of
$\mathfrak{X}^{0}=\\{\phi\\},\mathfrak{X},\mathfrak{X}^{2},...$; the open
subsets $\mathfrak{O}$ of which are $\mathfrak{O}=\biguplus_{n\geq
0}(\mathfrak{O}\cap\mathfrak{X}^{n}\mathcal{)}$ such that
$\mathfrak{O}\cap\mathfrak{X}^{n}$ are open in $\mathfrak{X}^{n}$; and thus
where the $\mathfrak{O}\cap\mathfrak{X}^{n}$ are generated by the Cartesian
products $O_{1}\times...\times O_{n}$ for open
$O_{1},...,O_{n}\subseteq\mathfrak{X}$. The $\mathfrak{X}^{n}$ for $n\geq 0$
are both open and closed in $\mathfrak{X}^{\infty}$, so that
$\mathfrak{X}^{\infty}$ is topologically disconnected.

The statistics of $\Phi$ are described by its probability measure
$P_{\Phi}(\mathfrak{B})$ where $\mathfrak{B}$ is a Borel-measurable subset of
$\mathfrak{X}^{\infty}$. That is, $\mathfrak{B}=\biguplus_{n\geq
0}(\mathfrak{B}\cap\mathfrak{X}^{n}\mathcal{)}$ such that
$\mathfrak{B}\cap\mathfrak{X}^{n}$ are measurable in $\mathfrak{X}^{n}$; and
thus where the $\mathfrak{B}\cap\mathfrak{X}^{n}$ are generated by the
$B_{1}\times...\times B_{n}$ for measurable
$B_{1},...,B_{n}\subseteq\mathfrak{X}$. The measure $P_{\Phi}$ is assumed to
be symmetric—i.e., if $\mathfrak{B}^{\prime}$ is constructed from
$\mathfrak{B}$ by permuting some or all of the entries in some or all of the
$\varphi\in\mathfrak{B}$, then
$P_{\Phi}(\mathfrak{B}^{\prime})=P_{\Phi}(\mathfrak{B})$. The projection
measure

$P_{\Phi}^{(n)}(\mathfrak{B})=P_{\Phi}(\mathfrak{B}\cap\mathfrak{X}^{n})$ (1)

describes the realizations of $\Phi$ with $n$ elements; and
$P_{\Phi}(\mathfrak{B})=\sum_{n\geq 0}P_{\Phi}^{(n)}(\mathfrak{B})$.

###  II-B Probability Generating Functional of a p.p.

According to [3], p. 2, the probability generating functional (p.g.fl.)
$\mathcal{G}_{\Phi}(h)$ of a p.p. $\Phi$ was introduced by Bartlett and
Kendall in the late 1940s and is ([3]; [1]; [2], Eq. (29)):

$\mathcal{G}_{\Phi}(h)=\sum_{n\geq
0}\int\left(\prod_{i=1}^{n}h(x_{i})\right)P_{\Phi}^{(n)}(dx_{1},...,dx_{n})$
(2)

where the “test function” $h:\mathfrak{X}\rightarrow[0,1]$ is unitless. Note
that
$0\leq\mathcal{G}_{\Phi}(h)\leq\mathcal{G}_{\Phi}(1)=P_{\Phi}(\mathfrak{X}^{\infty})=1$.

###  II-C Chain Differential of a p.g.fl.

If $F(h)\geq 0$ is a functional on $h:\mathfrak{X}\rightarrow[0,1]$, then its
chain differential is ([4]; [2], Eq. (35)):

$\delta
F(h;\eta)=\lim_{i\rightarrow\infty}\frac{F(h+\varepsilon_{i}\eta_{i})-F\left(h\right)}{\varepsilon_{i}}$
(3)

if the limits exist and are identical for any sequences
$\varepsilon_{i}\rightarrow 0$ and $\eta_{i}\rightarrow\eta$
(pointwise).111The chain differential of $F$ is a modification of the Gâteaux
differential of $F$, which is defined as $\lim_{\varepsilon\rightarrow
0^{+}}\varepsilon^{-1}(F(h+\varepsilon\eta)-F\left(h\right))$. Also,
$\delta^{n}F(h;\eta_{1},...,\eta_{n})$ is defined recursively by
$(\delta^{n}F)(h;\eta_{1},...,\eta_{n})=\delta F_{n-1}(h;\eta_{n})$ where
$F_{n-1}(h)=\delta^{n-1}F(h;\eta_{1},...,\eta_{n-1})$. By convention,
$\delta^{0}F(h)=F(h)$.

Assume that $F(h)$ is well-behaved enough that:

(a) $\eta\mapsto\delta F(h;\eta)$ is linear and continuous,222Because of these
assumptions $\delta F(h;\eta)$ is a chain derivative [4], not just a chain
differential. in which case $\mu_{F,h}:B\mapsto\delta F(h;\mathbf{1}_{B})$ is
a measure on $\mathfrak{X}$; and

(b) $\mu_{F,h}$ is absolutely continuous w/r/t (with respect to)
$\lambda_{\mathfrak{X}}$.

Then $\delta F(h;\delta_{x})$ is shorthand for the Radon-Nikodým derivative
$(d\mu_{F,h}/d\lambda_{\mathfrak{X}})(x)$ ([2], Eq. (39)), where
$\delta_{x}(y)=(d\Delta_{x}/d\lambda_{\mathfrak{X}})(y)$ is the Dirac delta
function on $\mathfrak{X}$, $\ \Delta_{x}(B)=\mathbf{1}_{B}(x)$ is the Dirac
measure, and $\mathbf{1}_{B}$ is the set indicator function of $B$.

$P_{\Phi}$ and $\mathcal{G}_{\Phi}$ are related by ([3], Eq. (4.14); [2], Eq.
(36)):

$P_{\Phi}^{(n)}(B_{1}\times...\times
B_{n})=\frac{1}{n!}\delta^{n}\mathcal{G}_{\Phi}(0;\mathbf{1}_{B_{1}},...,\mathbf{1}_{B_{n}})$
(4)

for measurable $B_{1},...,B_{n}\subseteq\mathfrak{X}$, with
$P_{\Phi}^{(0)}(\\{\phi\\})=\mathcal{G}_{\Phi}(0)$. For $n\geq 0$ define ([2],
Eq. (39)):

$p_{\Phi}^{(n)}(x_{1},...,x_{n})=\frac{1}{n!}\delta^{n}\mathcal{G}_{\Phi}(0;\delta_{x_{1}},...,\delta_{x_{n}}),$
(5)

so that $p_{\Phi}^{(0)}=\mathcal{G}_{\Phi}(0)$. The left side of (5) is the
family of Janossy densities of $\Phi$ [1] indexed by $n\geq 1$. Because
$P_{\Phi}$ is symmetric, $p_{\Phi}^{(n)}(x_{1},...,x_{n})$ is symmetric w/r/t
$x_{1},...,x_{n}$ for each $n\geq 2$.

The $p_{\Phi}^{(n)}(x_{1},...,x_{n})$ exist only if $\Phi$ is “simple”—i.e.,
if $x_{1},...,x_{n}$ are distinct for any realizations
$\Phi=(x_{1},...,x_{n})$ with $n\geq 2$ [1]. Thus all p.p.’s in [2] have
implicitly been assumed to be simple.

The chain differential, Gâteaux differential, and Frechét derivative of a
p.g.fl. $\mathcal{G}_{\Phi}$ exist and are equal; and are equivalent to the
Volterra functional derivative333Functional derivative: see [5], p. 375. $\
(\delta\mathcal{G}_{\Phi}/\delta x)(h)$ of $\mathcal{G}_{\Phi}$ in the sense
that

$\delta\mathcal{G}_{\Phi}(h;\eta)=\int\eta(x)\cdot\frac{\delta\mathcal{G}_{\Phi}}{\delta
x}(h)dx$ (6)

and thus that
$\delta\mathcal{G}_{\Phi}(h;\delta_{x})=(\delta\mathcal{G}_{\Phi}/\delta
x)(h)$.444This all follows from the fact that a p.g.fl. is a functional power
series and thus is, in this sense, analytic [6]. It is well known that the
Frechét derivative admits a chain rule. The chain differential was introduced
in 2005 in [4] to permit a chain rule for functions that are not Frechét
differentiable. The need for it in a p.g.fl. context is therefore unclear.

For a more detailed critique of the vector-based p.p. framework, see [6].

##  III Information Theory and p.p.’s

The purpose of this section is to describe the theoretically correct way to
apply information theory to p.p.’s. Given the mathematical formulation in [2],
“theoretically correct” means “correctly measure-theoretic.”

###### Remark 1

This section is not needed to understand why Eqs. (42-54) of [2] are
erroneous—see (IV,1,29). Rather, it is required for the demonstration in
Remark 9 that these errors cannot be corrected by employing the theoretically
correct approach described in this section.

It is organized as follows: Lesbegue integration (Section III-A); differential
entropy as a simple example (Section III-B); measure and integration for
p.p.’s (Section III-C); the probability density function of a p.p. (Section
III-D); and optimal state estimation for p.p.’s (Section III-E).

###  III-A Lebesgue Integration

Let $\mu(B)$ be a unitless measure of the Lebesgue-measurable subsets
$B\subseteq\mathfrak{X}$ and assume that it is absolutely continuous w/r/t
$\lambda_{\mathfrak{X}}$—i.e., $\lambda_{\mathfrak{X}}(B)=0$ implies
$\mu(B)=0$. Then by the Radon-Nikodým theorem, there is an almost everywhere
unique Lebesgue-integrable function
$d\mu/d\lambda_{\mathfrak{X}}:\mathfrak{X}\rightarrow\mathbb{R}^{+}$—the
Radon-Nikodým derivative of $\mu$ w/r/t the base measure
$\lambda_{\mathfrak{X}}$—such that

$\mu(B)=\int_{B}\frac{d\mu}{d\lambda_{\mathfrak{X}}}(x)dx$ (7)

for all $B$ and where $\lambda_{\mathfrak{X}}(dx)$ has been abbreviated as
$dx$. Because $\mu(B)$ is unitless and $dx$ has unit $\iota$,
$(d\mu/d\lambda_{\mathfrak{X}})(x)$ must have unit $\iota^{-1}$.

###### Remark 2

Since $\mathcal{G}_{\Phi}$ and thus $\mu_{\mathcal{G}_{\Phi},h}$ are unitless,
it follows that
$\delta\mathcal{G}_{\Phi}(h;\delta_{x})=(d\mu_{\mathcal{G}_{\Phi},h}/d\lambda_{\mathfrak{X}})(x)$
has unit $\iota^{-1}$ and
$\delta^{n}\mathcal{G}_{\Phi}(h;\delta_{x_{1}},...,\delta_{x_{n}})$ has unit
$\iota^{-n}$.

###### Remark 3

From this it follows that $\ p_{\Phi}^{(n)}(x_{1},...,x_{n})$ has unit
$\iota^{-n}$ for $n\geq 0$.

###  III-B Simple Example: Differential Entropy

Let $\ \mathbf{X}\in\mathfrak{X}$ be an r.v. with probability measure
$P_{\mathbf{X}}(B)$ and probability density function (p.d.f.)
$f_{\mathbf{X}}(x)=(dP_{\mathbf{X}}/d\lambda_{\mathfrak{X}})(x)$, where
$dP_{\mathbf{X}}/d\lambda_{\mathfrak{X}}$ is the Radon-Nikodým derivative of
$P_{\mathbf{X}}$ w/r/t $\ \lambda_{\mathfrak{X}}$. Then the differential
entropy (DE) of $\mathbf{X}$ is:

$DE(f_{\mathbf{X}})=-\int f_{\mathbf{X}}(x)\log f_{\mathbf{X}}(x)dx=-\int\log
f_{\mathbf{X}}(x)P_{\mathbf{X}}(dx)\mathbf{.}$ (8)

The DE has two well-known limitations as a quantifier of information. First,
it is not invariant w/r/t change of coordinates. Second and in particular, it
is undefined if $\ \mathfrak{X}$ and thus $f_{\mathbf{X}}$ have units of
measurement and thus $\log f_{\mathbf{X}}(\mathbf{x})$ is undefined. A minimum
requirement for any quantifier of information (or entropy) should be that its
numerical value does not change if (for example) one converts from metric to
English units.

Csiszár information functionals, such as the Kullback-Leibler divergence

$KL(f_{\mathbf{X}};f_{0})=\int\log\left(\frac{f_{\mathbf{X}}(x)}{f_{0}(x)}\right)P_{\mathbf{X}}(dx),$
(9)

do not have these limitations—see, e.g., Eq. (3) of [7].

Nevertheless, the DE provides a simple illustration of how to correctly apply
information theory to p.p.’s. Here we employ it to summarize this approach,
assuming the measure-theoretic results to be established in Sections III-C and
III-D.

The DE of a p.p. $\Phi$ must be

$DE(f_{\Phi})=-\int f_{\Phi}(\varphi)\log f_{\Phi}(\varphi)d\varphi=-\int\log
f_{\Phi}(\varphi)P_{\Phi}(d\varphi)$ (10)

where $\
f_{\Phi}(\varphi)=(dP_{\Phi}/d\lambda_{\mathfrak{X}^{\infty}})(\varphi)$ and
where $\lambda_{\mathfrak{X}^{\infty}}(\mathfrak{B})$ is a measure on the p.p.
state space  $\mathfrak{X}^{\infty}$. But (10) is meaningless unless we answer
the following question: What is $\ \lambda_{\mathfrak{X}^{\infty}}$?

To be conceptually consistent, $\lambda_{\mathfrak{X}^{\infty}}$ must be an
extension of $\lambda_{\mathfrak{X}}$ to $\mathfrak{X}^{\infty}$. If
$\mathfrak{X}$ has unit of measurement $\iota$ then the simplest extension is
$\lambda_{\mathfrak{X}^{\infty}}=\lambda_{c}$ with $c>0$ as given below in
(11). This leads, in (18), to the formula
$f_{\Phi}(\varphi)=c^{|\varphi|}p_{\Phi}^{(|\varphi|)}(\varphi)$—which in
turn, means that if $\mathfrak{X}$ has a unit then
$p_{\Phi}^{(|\varphi|)}(\varphi)$ is not the p.d.f. of $\Phi$. That is:

###### Remark 4

Any p.p. quantifier of information (or entropy) that is measure-theoretic and
generally applicable must be defined using $f_{\Phi}(\varphi)$ rather than
$p_{\Phi}^{(|\varphi|)}(\varphi)$.

###  III-C Measure and Integration for p.p.’s

Just as the $\sigma$-algebra of $\mathfrak{X}^{\infty}$ is an extension of the
$\sigma$-algebra of $\mathfrak{X}$, so $\lambda_{\mathfrak{X}^{\infty}}$ is
the following well-known extension of $\lambda_{\mathfrak{X}}$ to
$\mathfrak{X}^{\infty}$ ([5], p. 715):

$\lambda_{c}(\mathfrak{B})=\mathbf{1}_{\mathfrak{B}}(\phi)+\sum_{n\geq
1}\frac{\lambda_{\mathfrak{X}}^{n}(\mathfrak{B}\cap\mathfrak{X}^{n})}{c^{n}}.$
(11)

Here, $c>0$ has unit $\iota$; $\
\lambda_{\mathfrak{X}}^{1}=\lambda_{\mathfrak{X}}$; and for $n\geq 2$,
$\lambda_{\mathfrak{X}}^{n}$ is the extension of $\lambda_{\mathfrak{X}}$ to
$\mathfrak{X}^{n}$, in which case
$\lambda_{\mathfrak{X}}^{n}(B_{1}\times...\times B_{n})$ has unit $\iota^{n}$.
Because of $c$, the summation in (11) is mathematically well-defined since its
terms are unitless. Like $\lambda_{\mathfrak{X}}$, $\lambda_{c}$ is a possibly
infinite-valued measure. Unlike $\lambda_{\mathfrak{X}}$, it is unitless.

The projection measures of $\lambda_{c}$ are
$\lambda_{c}^{(0)}(\mathfrak{B})=\mathbf{1}_{\mathfrak{B}}(\phi)$ and
$\lambda_{c}^{(n)}(\mathfrak{B})=\lambda_{\mathfrak{X}}^{n}(\mathfrak{B}\cap\mathfrak{X}^{n})/c^{n}$
for $n\geq 1$. In particular,
$\lambda_{c}^{(1)}(\mathfrak{B})=\lambda_{\mathfrak{X}}(\mathfrak{B}\cap\mathfrak{X})/c$.

Let $f(\varphi)$ be a nonnegative unitless function of $\varphi$. Then its
integral w/r/t $\lambda_{c}$ within
$\mathfrak{B}\subseteq\mathfrak{X}^{\infty}$ is

$\displaystyle\int_{\mathfrak{B}}f(\varphi)\lambda_{c}(d\varphi)$
$\displaystyle=$ $\displaystyle
f(\phi)\cdot\mathbf{1}_{\mathfrak{B}}(\phi)+\sum_{n\geq
1}\frac{1}{c^{n}}\int_{\mathfrak{B}\cap\mathfrak{X}^{n}}f(x_{1},...,x_{n})dx_{1}\cdots
dx_{n}$

where $\lambda_{\mathfrak{X}}^{n}(dx_{1},...,dx_{n})$ has been abbreviated as
$dx_{1}\cdots dx_{n}$. Thus
$\lambda_{c}(\mathfrak{B})=\int_{\mathfrak{B}}\lambda_{c}(d\varphi)$.

###  III-D Probability Density Function (p.d.f.) of a p.p.

If $P_{\Phi}$ is absolutely continuous w/r/t $\lambda_{c}$ then its p.d.f. is
$f_{\Phi}(\varphi)=dP_{\Phi}/d\lambda_{c})(\varphi)$, which is characterized
by the Radon-Nikodým theorem

$\displaystyle P_{\Phi}(\mathfrak{B})$ $\displaystyle=$
$\displaystyle\int_{\mathfrak{B}}f_{\Phi}(\varphi)\lambda_{c}(d\varphi)=\int_{\mathfrak{B}}\frac{dP_{\Phi}}{d\lambda_{c}}(\varphi)\lambda_{c}(d\varphi)$
(13) $\displaystyle=$
$\displaystyle\frac{dP_{\Phi}}{d\lambda_{c}}(\phi)\cdot\mathbf{1}_{\mathfrak{B}}(\phi)$
$\displaystyle+\sum_{n\geq
1}\frac{1}{c^{n}}\int_{\mathfrak{B}\cap\mathfrak{X}^{n}}\frac{dP_{\Phi}}{d\lambda_{c}}(x_{1},...,x_{n})dx_{1}\cdots
dx_{n}$

where the restiction of $dP_{\Phi}/d\lambda_{c}$ to $\mathfrak{X}^{n}$ is
equal to $dP_{\Phi}^{(n)}/d\lambda_{c}$. By (III-D), (1), and (4), for $n\geq
1$ the projection measures $P_{\Phi}^{(n)}$ of $P_{\Phi}$ are given by

$\displaystyle\frac{1}{n!}\delta^{n}\mathcal{G}_{\Phi}(0;\mathbf{1}_{B_{1}},...,\mathbf{1}_{B_{n}})$
$\displaystyle=$ $\displaystyle\frac{1}{c^{n}}\int_{B_{1}\times...\times
B_{n}}\frac{dP_{\Phi}^{(n)}}{d\lambda_{c}}(x_{1},...,x_{n})dx_{1}\cdots
dx_{n}$ $\displaystyle=$
$\displaystyle\frac{1}{c^{n}}\int_{B_{1}\times...\times
B_{n}}f_{\Phi}(x_{1},...,x_{n})dx_{1}\cdots dx_{n}.$ (16)

From this and (5) it follows that

$p_{\Phi}^{(n)}(x_{1},...,x_{n})=\frac{1}{c^{n}}\cdot
f_{\Phi}(x_{1},...,x_{n})$ (17)

and thus that the p.d.f. of $P_{\Phi}$ is the unitless function

$f_{\Phi}(\varphi)=c^{|\varphi|}p_{\Phi}^{(|\varphi|)}(\varphi).$ (18)

###### Remark 5

Since $c$ has unit $\iota$, $f_{\Phi}(\varphi)\neq
p_{\Phi}^{(|\varphi|)}(\varphi)$ even if $c=1\cdot\iota$ since
$f_{\Phi}(\varphi)$ is unitless and $p_{\Phi}^{(|\varphi|)}(\varphi)$ is not.
Thus the family $p_{\Phi}^{(n)}(x_{1},...,x_{n})$ of Janossy densities on
$\mathfrak{X}^{n}$ for $n\geq 0$ is not the same thing as the p.d.f.
$f_{\Phi}(\varphi)$ of $\Phi$, which is a single density on
$\mathfrak{X}^{\infty}$.

The application of advanced Gibbs statistical sampling techniques to exact
closed-form approximations of $f_{\Phi}(\varphi)$—or more precisely, of
$f_{\Phi}(X)$ as defined in Section III-E—has led to implementations of the
generalized labeled multi-Bernoulli (GLMB) filter that are capable of
simultaneous real-time tracking of over a million targets in significant
clutter using off-the-shelf computing equipment [8].

###### Remark 6

It follows that (III-D) can be rewritten as

$P_{\Phi}(\mathfrak{B})=\sum_{n\geq
0}\int_{\mathfrak{B}\cap\mathfrak{X}^{n}}p_{\Phi}^{(n)}(x_{1},...,x_{n})dx_{1}\cdots
dx_{n}.$ (19)

###### Remark 7

It also follows that if $n\geq 1$ then

$P_{\Phi}^{(n)}(dx_{1},...,dx_{n})=p_{\Phi}^{(n)}(x_{1},...,x_{n})dx_{1}\cdots
dx_{n}.$ (20)

For, from (III-C) and (18) we have

$\displaystyle\int f(x_{1},...,x_{n})P_{\Phi}^{(n)}(dx_{1},...,dx_{n})$
$\displaystyle=$ $\displaystyle\int
f(x_{1},...,x_{n})\cdot\frac{dP_{\Phi}^{(n)}}{d\lambda_{c}}(x_{1},...,x_{n})\lambda_{c}(dx_{1},...,dx_{n})$
$\displaystyle=$ $\displaystyle\frac{1}{c^{n}}\int f(x_{1},...,x_{n})\cdot
c^{n}p_{\Phi}^{(n)}(x_{1},...,x_{n})dx_{1}\cdots dx_{n}$ (22) $\displaystyle=$
$\displaystyle\int f(x_{1},...,x_{n})\cdot
p_{\Phi}^{(n)}(x_{1},...,x_{n})dx_{1}\cdots dx_{n}.$ (23)

###  III-E Optimal State Estimation for p.p.’s

The most probable realization of $\Phi$ is the maximum a posteriori (MAP)
estimate extracted from (18):

$(\hat{x}_{1},...,\hat{x}_{\hat{n}})_{c}=\arg\sup_{n,x_{1},...,x_{n}}c^{n}p_{\Phi}^{(n)}(x_{1},...,x_{n}).$
(24)

Since each $c>0$ determines a different most-probable estimate, (24) is
essentially useless unless we answer the following question: What is the best
choice for $c$?

To answer it, make the following changes from vector to finite-set notation.
Assume that $x_{1},...,x_{n}$ are distinct and let $X=\\{x_{1},...,x_{n}\\}$.
Write $f_{\Phi}(X)=n!\cdot p_{\Phi}^{(n)}(x_{1},...,x_{n})$ and $|X|=n$, in
which case (24) becomes

$\hat{X}_{c}=\arg\sup_{X}\frac{c^{|X|}}{|X|!}f_{\Phi}(X).$ (25)

According to the analysis of [5], pp. 499-500, for this estimate to be
accurate the magnitude of $c$ should be approximately equal to the accuracy
with which individual states $x\in\mathfrak{X}$ are to be estimated.

###### Remark 8

Thus $c\neq 1\cdot\iota$ in general.

##  IV Mathematically Undefined Formulas

We are now in a position to demonstrate that Eqs. (42-54) of [2] are
mathematically undefined.

Begin by inspecting the key formula Eq. (42), the “Information generating
functional for entropy”:

$\displaystyle\mathcal{G}_{\Phi}^{\alpha}(h)$ $\displaystyle=$
$\displaystyle\sum_{n\geq 0}\int\left(\prod_{i=1}^{n}h(x_{i})\right)$
$\displaystyle\cdot
p_{\Phi}^{(n)}(x_{1},...,x_{n})^{-\alpha}P_{\Phi}^{(n)}(dx_{1},...,dx_{n}).$

Since $P_{\Phi}^{(n)}$ is a unitless measure and since by Remark 3
$p_{\Phi}^{(n)}(x_{1},...,x_{n})^{-\alpha}$ has unit $\iota^{n\alpha}$, then
as long as $\alpha\neq 0$ the summation is mathematically undefined since its
terms have different units of measurement for $n\geq 0$—and thus are
incommensurable.

To see this more explicitly note that, by (20), we can rewrite (IV) as:

$\displaystyle\mathcal{G}_{\Phi}^{\alpha}(h)$ $\displaystyle=$
$\displaystyle\sum_{n\geq 0}\int\left(\prod_{i=1}^{n}h(x_{i})\right)$
$\displaystyle\cdot p_{\Phi}^{(n)}(x_{1},...,x_{n})^{1-\alpha}dx_{1}\cdots
dx_{n}$

where $p_{\Phi}^{(n)}(x_{1},...,x_{n})^{1-\alpha}dx_{1}\cdots dx_{n}$ has unit
$\iota^{-n(1-\alpha)}\cdot\iota^{n}=\iota^{n\alpha}$.

###### Remark 9

This error cannot be corrected by substituting $f_{\Phi}(x_{1},...,x_{n})$ in
place of $p_{\Phi}^{(n)}(x_{1},...,x_{n})$—indeed, it becomes worse. For then
$p_{\Phi}^{(n)}(x_{1},...,x_{n})^{1-\alpha}dx_{1}\cdots dx_{n}$ becomes
$c^{n(1-\alpha)}p_{\Phi}^{(n)}(x_{1},...,x_{n})^{1-\alpha}dx_{1}\cdots
dx_{n}$, which has unit
$\iota^{n(1-\alpha)}\cdot\iota^{-n(1-\alpha)}\cdot\iota^{n}=\iota^{n}$
regardless of the values of both $\alpha$ and $c$. In particular, the error
remains even when $\alpha=0$.

###### Remark 10

It might be argued that $c=1\cdot\iota$ suffices as an engineering
simplification or approximation, but this is not the case. By Remark 8, $c\neq
1\cdot\iota$ in general if we are to find the most probable estimate of
$\Phi$—i.e., find the best estimate for engineering purposes.

These remarks apply with full force to:

  1. 1.

Key formula Eq. (47), the “Laplace information functional for entropy”:

$\displaystyle\mathcal{L}_{\Phi}^{\alpha}(f)$ $\displaystyle=$
$\displaystyle\sum_{n\geq 0}\int\exp\left(-\sum_{i=1}^{n}f(x_{i})\right)$
$\displaystyle\cdot
p_{\Phi}^{(n)}(x_{1},...,x_{n})^{-\alpha}P_{\Phi}^{(n)}(dx_{1},...,dx_{n})$

(for functions $f:\mathfrak{X}\rightarrow\mathbb{R}^{+}$), which has the
general form
$\mathcal{L}_{\Phi}^{\alpha}(f)=\mathcal{G}_{\Phi}^{\alpha}(e^{-f})$ where
$e^{-f}(x)=e^{-f(x)}$.

  2. 2.

Key formula Eq. (51), the “cumulant information functional”:

$\mathcal{W}_{\Phi}^{\alpha}(f)=\log\mathcal{L}_{\Phi}^{\alpha}(f).$ (29)

  3. 3.

Any formula defined in terms of $\mathcal{G}_{\Phi}^{\alpha}(h)$,
$\mathcal{L}_{\Phi}^{\alpha}(h)$, or $\mathcal{W}_{\Phi}^{\alpha}(h)$—i.e.,
Eqs. (42-54).

Additional errors should be noted. Consider the “Shannon entropy,” Eq. (43),
which from Eq. (42) can be written as

$\displaystyle\left[\frac{\partial}{\partial\alpha}\mathcal{G}_{\Phi}^{\alpha}(1)\right]_{\alpha=0}$
$\displaystyle=$ $\displaystyle-\sum_{n\geq 0}\int\log
p_{\Phi}^{(n)}(x_{1},...,x_{n})P_{\Phi}^{(n)}(dx_{1},...,dx_{n})$
$\displaystyle=$ $\displaystyle-\int\log
p_{\Phi}^{(|\varphi|)}(\varphi)P_{\Phi}^{(|\varphi|)}(d\varphi).$ (31)

By Remark 5, this is a theorectically erroneous version of the differential
entropy formula (10) since $p_{\Phi}^{(|\varphi|)}(\varphi)\neq
f_{\Phi}(\varphi)$. It is also mathematically undefined since, by Remark 3,
$p_{\Phi}^{(n)}(x_{1},...,x_{n})$ has unit $\iota^{-n}$ and thus $\log
p_{\Phi}^{(|\varphi|)}(\varphi)$ is mathematically undefined.

The same is true of the “Shannon entropy moments,” Eq. (48):555As originally
written, Eq. (48) had a typo: $\mathcal{L}_{\Phi}^{\alpha}(f)$ should have
been $\mathcal{L}_{\Phi}^{\alpha}(0)$.

$\displaystyle\mathbb{E}_{\Phi}\left[(\log p_{\Phi})^{m}\right]$
$\displaystyle=$
$\displaystyle(-1)^{m}\left[\frac{\partial^{m}}{\partial\alpha^{m}}\mathcal{L}_{\Phi}^{\alpha}(0)\right]_{\alpha=0}$
(32) $\displaystyle=$ $\displaystyle\int(\log
p_{\Phi}^{(|\varphi|)}(\varphi))^{m}P_{\Phi}^{(|\varphi|)}(d\varphi).$ (33)

###### Remark 11

One could patch up these particular errors by substituting $f_{\Phi}(\varphi)$
in place of $p_{\Phi}^{(|\varphi|)}(\varphi)$. But this would not change the
fact that they are secondary errors inherited from the inherently erroneous
$\mathcal{G}_{\Phi}^{\alpha}(h)$ and $\mathcal{L}_{\Phi}^{\alpha}(h)$.

## References

  * [1] D. Daley and D. Vere-Jones, An Introduction to the Theory of Point Processes, Springer-Verlag, New York, 1988.
  * [2] D. Clark, “Local entropy statistics for point processes,” IEEE Trans. Info. Theory, Vol. 66, No. 2, pp. 1155-1163, 2020.
  * [3] J. Moyal, “The general theory of stochastic population processes,” Acta Mathematica, Vol. 108, pp. 1-31, 1962.
  * [4] P. Bernard, “Chain differentials with an application to the mathematical fear operator,” Nonlinear Analysis: Theory, Methods and Applications, Vol. 62, No. 7, pp. 1225-1233, 2005.
  * [5] R. Mahler, Statistical Multisource-Multitarget Information Fusion, Artech House, Norwood, MA, 2007.
  * [6] R. Mahler, “‘Statistics 103’ for multitarget tracking,” Sensors, 19(1): 202, 2019, open source: https://doi.org/10.3390/s19010202.
  * [7] Yu Qiao and Nobuaki Minematsu, “A study on invariance of f-divergence and its application to speech recognition,” IEEE Trans. Sign. Proc., Vol. 58, No. 7, pp. 3884-3890, 2010.
  * [8] M. Beard, B.-T. Vo, and B.-N. Vo, “A solution for large-scale multi-object tracking,” IEEE Trans. Sign. Proc., Vol. 68, pp. 2754-2769, 2020.