# Syntactic Requirements for Well-defined Hybrid Probabilistic Logic Programs

Damiano Azzolini University of Ferrara
Dipartimento di Ingegneria<EMAIL_ADDRESS>University of Ferrara
Dipartimento di Matematica e Informatica Fabrizio Riguzzi University of
Ferrara
Dipartimento di Matematica e Informatica<EMAIL_ADDRESS>
###### Abstract

Hybrid probabilistic logic programs can represent several scenarios thanks to
the expressivity of Logic Programming extended with facts representing
discrete and continuous distributions. The semantics for this type of programs
is crucial since it ensures that a probability can be assigned to every query.
Here, following one recent semantics proposal, we illustrate a concrete
syntax, and we analyse the syntactic requirements needed to preserve the well-
definedness.

##  1 Introduction

The power and expressivity of Probabilistic Logic Programming (PLP) [9, 19]
have been utilized to represent many real world situations [3, 10, 15].
Usually, probabilistic logic programs involve only discrete random variables
with Bernoulli or Categorical distributions. Numerous solutions emerged to
also handle continuous distributions [11, 13, 26], increasing the
expressiveness of PLP and giving birth to hybrid probabilistic logic programs,
that is, programs that include discrete and continuous random variables.
Inference in this type of programs is hard since it combines the complexity of
the grounding computation with the intractability of a distribution defined by
a mixture of random variables.

Usually, inference in general hybrid probabilistic logic programs (i.e.,
without imposing restrictions on the type of distributions allowed) is done by
leveraging knowledge compilation and using external solvers [26] or by
sampling [5, 17].

The semantics of hybrid programs has been proved well-defined in [4], but the
authors neither provided an explicit syntax, nor introduced explicit syntactic
requirements. In this paper, we close this gap by introducing an explicit
syntax and the syntactic requirements needed to preserve the well-definedness
of the semantics of hybrid probabilistic logic programs.

The paper is structured as follows: in Section 2 we introduce the semantics
and the syntax for programs with only discrete random variables with both
finite and infinite number of explanations. The semantics and syntax for
hybrid programs is introduced in Section 3, and the syntactic requirements are
analysed in Section 4. Section 5 concludes the paper. All the Sections are
accompanied by several examples, to make the concepts clearer.

##  2 Probabilistic Logic Programming

Probabilistic Logic Programming (PLP) allows the construction of complex yet
interpretable probability models through the usage of Logic Programming.
Several probabilistic logic languages based on the Distribution Semantics (DS)
[22] have been proposed during the years, such as PRISM [22], and ProbLog
[10]. Basically, they allow the definition of probabilistic facts, i.e., facts
that can be true or false with a certain probability.

Following the ProbLog syntax [10], each probabilistic fact has the form $p::a$
where $p\in]0,1]$, meaning that $a$ is true with probability $p$ and false
with probability $1-p$. If $p=1$, the fact is deterministic. An atomic choice
indicates whether a grounding $a\theta$ of a probabilistic fact $p::a$ is
selected or not, and it is represented with the triple $(a,\theta,k)$.
$k\in\\{0,1\\}$, where $k=1$ means that the fact is selected and $k=0$ that it
is not. If a set of atomic choices does not contain two atomic choices in
which the fact is both selected and not, it is termed consistent. A consistent
set $\kappa$ of atomic choices is called composite choice and its probability
can be computed as:

$\displaystyle
P(\kappa)=\prod_{(a_{i},\theta,1)\in\kappa}p_{i}\cdot\prod_{(a_{i},\theta,0)\in\kappa}(1-p_{i})$

A total composite choice contains one atomic choice for every grounding of
every probabilistic fact. A world is a logic program identified by a total
composite choice, and it is created by selecting the atoms corresponding to
each atomic choice with $k=1$. Its probability is given by the probability of
the total composite choice. Here we consider only programs where every world
has a two-valued Well-Founded Model (WFM). A semantics for programs without a
two-valued WFM [23] based on credal sets is discussed in [8]. If a program
does not contain function symbols, the set of groundings for every
probabilistic fact is finite, and so the set of worlds for a program is
finite.

######  Example 1 (Card)

Consider the following toy example that models a deck of three cards composed
of ace of spades, ace of clubs, and ace of hearts. The three cards can be
drawn with equal probability. This simple scenario can be modelled using the
following program:

    1/3 :: spades(X).
    1/2 :: clubs(X).
    pick(0,spades) :- spades(0).
    pick(0,clubs) :- \+ spades(0), clubs(0).
    pick(0,hearts) :- \+ spades(0), \+ clubs(0).

We use only two random variables since the third (hearts) can be represented
with both random variables negated (see 5th clause). Furthermore, note that
the probability of clubs(X) is set to $1/2$ to obtain $1/3$ for every possible
choice: P(pick(0,spades)) $=1/3$, P(pick(0,clubs)) $=2/3\cdot 1/2=1/3$, and
P(pick(0,hearts)) $=2/3\cdot 1/2=1/3$. We keep the variable X for the first
two facts, even if it may be considered a singleton variable, to better
illustrate the next concepts.

Example 1 illustrates a program without function symbols. If a program has at
least one variable, one constant, and one function symbol, its grounding is
infinite, so the previous definition must be extended [16].

The set of worlds $\omega_{\kappa}$ compatible with a composite choice
$\kappa$ is defined as $\omega_{\kappa}=\\{w_{\sigma}\in
W_{P}\mid\kappa\subseteq\sigma\\}$, where $W_{P}$ is the set of worlds for the
whole probabilistic logic program, $\sigma$ is a total composite choice, and
$w_{\sigma}$ is the world identified by the total choice. If the program does
not have function symbols, the probability of a composite choice is equal to
the sum of the probabilities of the worlds in $\omega_{\kappa}$. If the
program has function symbols, $\omega_{\kappa}$ may be uncountable and the
probability of each world is 0 (infinite product of values $\in]0,1[$).
However, the probability of a composite choice can still be computed. The set
of worlds compatible with a set of composite choices $K$ is given by the union
of the worlds identified by each composite choice in the set. Two composite
choices are incompatible if their union is not consistent. A set of composite
choices is pairwise incompatible if every pair of different composite choices
is incompatible. $P(K)$ is still well-defined for programs with function
symbols and can be computed by constructing pairwise incompatible equivalent
sets. A composite choice $\kappa$ is an explanation for a query $q$ (a
conjunction of ground atoms) if, for all worlds $w\in\omega_{\kappa}$,
$w\models q$. Furthermore, a set of composite choices is covering with respect
to a query $q$ if $\forall w$ in which $q$ is true, $w\in\omega_{\kappa}$. We
can also define as explanation a set of worlds and define the covering
property for sets of worlds. The probability of a query $q$ is then computed
by defining a probability measure over the set of worlds identified by
countable sets of countable composite choices:
$\mu_{P}(\omega_{K})=\lim_{n\to\infty}\mu(K^{{}^{\prime}}_{n})$, where
$K^{{}^{\prime}}_{n}=\\{\kappa^{{}^{\prime}}_{1},\dots,\kappa^{{}^{\prime}}_{n}\\}$
is a pairwise incompatible set of composite choices equivalent to
$K_{n}=\\{\kappa_{1},\dots,\kappa_{n}\\}$,
$K=\\{\kappa_{1},\kappa_{2},\dots\\}$ is a covering set of explanations for
$q$ and $\mu_{P}(K_{n})=\sum_{i=1}^{n}P(\kappa^{\prime}_{i})$. Programs with
function symbols have a well-defined semantics [18].

If a probabilistic logic program is range restricted, i.e., every variable in
the head appears also in a positive literal in the body, the answers of a
query are always ground instantiations of it [14, 21], and probabilities are
always assigned to ground atoms. However, in some cases, probabilistic facts
may not be range restricted, and can contain variables (see, for instance, the
probabilistic facts spades/1 and clubs/1 in Example 1). In this situation,
queries’ answers are still ground instantiations and the program can be well-
defined provided that the variables in probabilistic facts are ground when the
fact is called: they can be directly bound to a constant (as in Example 1,
where X is bound to 0), or they must appear in a previous literal in the body.

######  Theorem 1 (Ground queries with well-defined probability)

Given a Probabilistic Logic Program $P$ and a query $q$, if $P$ is range
restricted and all the variables in probabilistic facts appearing in the body
of clauses of $P$ are present in a previous positive literal, the answers to
$q$ will be ground instantiation of it, with an associated well-defined
probability.

Before illustrating the proof, we introduce some basic concepts following
[14]. Given two atoms, $a$ and $b$, a substitution $\theta$ (a replacement of
variables with terms) is a unifier for $a$ and $b$ if $a\theta=b\theta$. A
substitution is a grounding if all the involved terms are ground. Furthermore,
a unifier $\sigma$ is the most general unifier (mgu) for $a$ and $b$ if, for
all unifiers $\mu$ of both $a$ and $b$, exists a substitution $\rho$ such that
($a\sigma)\rho=a\mu$.

The main step of resolution is the following: given two clauses $A$ and $B$
with no common variables, $l\in A$ and $\neg l^{\prime}\in B$, and $\theta$
being the mgu of $l$ and $l^{\prime}$, the resolvent of $A$ and $B$ is defined
as $((A\setminus\\{l\\})\cup(B\setminus\\{l^{\prime}\\}))\theta$; for a
program $P$ and a query $q$, resolution is linear when $B$ represents only
clauses in $P$ and $A$ is the query or the resolvent of another linear
resolution. We suppose that literals in clauses are ordered and are chosen
following this order during the resolution. A resolution proof for SLDNF can
be represented as a sequence of clauses $\langle
q,C_{1},C_{2},\dots,C_{n}\rangle$. If $C_{n}$ is the empty clause, the proof
is a refutation. The substitution for the query $q$ is given by
$\theta=\theta_{1}\theta_{2}\dots\theta_{n}$, where each $\theta_{i}$ is the
substitution correspondent to clause $C_{i}$. Moreover, if $P$ is range
restricted, $\theta$ is a grounding.

###### Proof 1

Let us now prove Theorem 1 by induction. In the base case, the SLDNF
resolution consists of only one step: the query $q$ unifies with a
deterministic fact, since there is not a previous positive literal in the body
that will allow the presence of a probabilistic fact. The program is range
restricted, $\theta_{1}$ is computed, and the variables in the query are all
grounded. Suppose now that the theorem is true at step $n$. The current set of
substitutions is $\\{\theta_{1},\theta_{2},\dots,\theta_{n}\\}$. There are two
possible cases: if the selected literal of the query matches a deterministic
fact, substitution $\theta_{n+1}$ grounds the literal, as deterministic facts
are already ground. If the selected literal of $q$ matches with a
probabilistic fact, this fact cannot be the first of the body of the
correspondent clause, by assumption: all the variables appearing in the
probabilistic fact are already grounded by preceding substitutions, so the
literal is ground when called, and a new substitution $\theta_{n+1}$ is
computed and added to the list. Eventually, all the variables will be grounded
and the answer to the query will be a ground instantiation.

If variables appear for the first time in probabilistic facts, the query could
still eventually be ground, but probabilistic facts will not be ground when
called, and thus the semantics will be ill-defined.

Let us now extend Example 1 by introducing new clauses to obtain a program
with an infinite covering set of explanations.

######  Example 2 (Card with an infinite number of explanations)

Consider a game of card where a player needs to pick one out of three possible
cards: ace of spades, ace of clubs or ace of hearts. The game stops when the
player picks the ace of hearts. This can be modelled by adding the following
clauses to Example 1.

    pick(s(X),spades):- \+ pick(X,hearts), spades(s(X)).
    pick(s(X),clubs):- \+ pick(X,hearts), \+ spades(s(X)), clubs(s(X)).
    pick(s(X),hearts):- \+ pick(X,hearts), \+ spades(s(X)), \+ clubs(s(X)).

    at_least_once_spades :- pick(_,spades).
    never_spades :- \+ at_least_once_spades.

We can ask for the probability that the player picks at least once spades or
that that he/she never picks spades (respectively P(at_least_once_spades) and
P(never_spades)).

For conciseness, let us replace spades(X) with $f_{1}$ and clubs(X) with
$f_{2}$. Consider query
at_least_once_spades: it has the pairwise incompatible covering set of
explanations $K=\\{\kappa_{0},\kappa_{1},\ldots\\}$ with

$\displaystyle\kappa_{0}={}$ $\displaystyle\\{(f_{1},\\{X/0\\},1)\\}$
$\displaystyle\kappa_{1}={}$
$\displaystyle\\{(f_{1},\\{X/0\\},0),(f_{2},\\{X/0\\},1),(f_{1},\\{X/s(0)\\},1)\\}$
$\displaystyle\ldots$ $\displaystyle\kappa_{i}={}$
$\displaystyle\\{(f_{1},\\{X/0\\},0),(f_{2},\\{X/0\\},1),\ldots,(f_{1},\\{X/s^{i-1}(0)\\},0),$
$\displaystyle(f_{2},\\{X/s^{i-1}(0)\\},1),(f_{1},\\{X/s^{i}(0)\\},1)\\}$
$\displaystyle\ldots$

In other words, in $\kappa_{0}$ the player picked spades at round $0$, in
$\kappa_{1}$ he/she does not piked spades and picked clubs at round $0$ and
picked spades in round $s(0)$, and so on.

$K$ is countable and infinite and the explanations in it are pairwise
incompatible, so the probability of at_least_once_spades can be computed as

$\displaystyle P(at\\_least\\_once\\_spades)={}$
$\displaystyle\frac{1}{3}+\frac{1}{3}\cdot\left(\frac{2}{3}\cdot\frac{1}{2}\right)+\frac{1}{3}\cdot\left(\frac{2}{3}\cdot\frac{1}{2}\right)^{2}+\ldots$
$\displaystyle={}$
$\displaystyle\frac{1}{3}+\frac{1}{3}\cdot\left(\frac{1}{3}\right)+\frac{1}{3}\cdot\left(\frac{1}{3}\right)^{2}+\ldots$
$\displaystyle={}$
$\displaystyle\frac{1}{3}\cdot\frac{1}{1-\frac{1}{3}}=\frac{1}{3}\cdot\frac{3}{2}=\frac{1}{2}$

since it is the sum of a geometric series. The probability for the query
never_spades can be computed in a similar way. After the computations we get
$P(at\\_least\\_once\\_spades)=1-P(never\\_spades)=1/2$.

In the next section we introduce the syntax for hybrid probabilistic logic
programs, i.e., programs with both discrete and continuous random variables.

##  3 Hybrid Programs

When a probabilistic logic program contains both discrete and continuous
random variables, it is called hybrid probabilistic logic program. Here, we
also consider constraints involving continuous random variables’ values,
obtaining probabilistic constraint logic programs [4, 13]. In this paper we
use hybrid probabilistic logic program and probabilistic constraint logic
program interchangeably, since usually in PLP the goal is to compute the
probability of a query (and thus we always have constraints to define the
range of the continuous random variables), rather than the joint probability
density induced by the program.

A probabilistic constraint logic program is composed of a set of rules, a set
of Boolean probabilistic facts, and a countable set of continuous random
variables. The authors in [4] define $X=\\{X_{1},X_{2},\dots\\}$ the countable
set of continuous random variables, each one associated with its range (that
can be either $\mathbb{R}$ or $\mathbb{R}^{n}$), and $F$ the set containing
discrete probabilistic facts. These discrete facts form a countable set of
Boolean random variables $Y=\\{Y_{1},Y_{2},\dots\\}$. The sample space of the
whole program ($W_{\mathcal{P}}$) is defined as the product of the spaces for
continuous ($W_{X}$) and discrete ($W_{Y}$) random variables, i.e,
$W_{\mathcal{P}}=W_{X}\times W_{Y}$.

The probabilistic facts associated with every random variable $Y_{i}$ set to
1, and the grounding of the rules (with the constraints removed) whose
constraints are satisfied given a valuation of the continuous random
variables, define a ground normal logic program. The probability of a query
can be computed by considering a pairwise incompatible covering set of
explanations. As before, an explanation for a query of a probabilistic
constraint logic program is a set of worlds $\omega_{i}$ such that the query
is true in every element of this set.

A concrete syntax for this type of programs is given by cplint hybrid programs
[19]. In cplint hybrid programs, logical variables are partitioned into two
disjoint sets: those that can assume terms as values and those that can assume
continuous values. Let us call the first term variables and the latter
continuous variables.

Continuous random variables are encoded with probabilistic facts of the form

$A:Density$

where $A$ is an atom with a continuous variable $Var$ as argument and
$Density$ is a special atom identifying a probability density on variable
$Var$. For example,

    p(X) : gaussian(X,0,1).

indicates that X in atom p(X) is a continuous variable that follows a Gaussian
distribution with mean 0 and variance 1. Each predicate $p/n$ has a signature
that specifies which arguments hold continuous values. Only these arguments
can contain continuous variables. Continuous values (and variables) can appear
inside a term built on a function symbol $f/n$. Each function symbol $f/n$
also has a signature that specifies which arguments hold continuous values.
Again, only these arguments can contain continuous variables. While discrete
random variables are identified by ground atoms (that form a countable set),
continuous random variables are identified by predicates and by the ground
terms present in atoms with arguments that can hold continuous random
variables.

ProbLog probabilistic facts of the form $p::f$ can also be encoded as $f:p$
for uniformity with Logic Programs with Annotated Disjunctions [25] and CP-
Logic [24].

Atoms in clauses and probabilistic facts can have both term and continuous
variables. However, we impose the constraint that in every world of the
program the values taken by term variables in a ground atom for a predicate
$p/n$ that is true in the world uniquely determine the values taken by the
continuous variables.

Continuous variables are introduced by probabilistic facts for continuous
random variables and by the special predicate =:=/2 that is used to define a
new variable based on a formula involving existing continuous variables (see
Example 5). Constraints are represented by Prolog comparison predicates. The
semantics assigns a probability of being true to any ground atom not having
continuous values as arguments. Atoms with continuous values have probability
0, as the probability that a continuous random variable takes a specific value
is 0. However, there is one special case when a continuous value is admitted
as an argument. This will be discussed in Section 4.

Inference in cplint hybrid programs can be performed using MCINTYRE [2, 5,
20], an algorithm based on Monte Carlo sampling. cplint hybrid programs can
also be translated into probabilistic constraint logic programs [13] by
removing the continuous variables from the arguments of predicates and by
replacing constraints with their probabilistic constraint logic program form.

Consider an extension to Example 2, where we also add a continuous random
variable and clauses with constraints.

######  Example 3 (Card extended with a continuous random variable)

Suppose that the previous game is extended, and the player also needs to spin
a wheel. In addition to the previous rules, if the axis of the wheel is
between 0 and $\pi$ degrees (approximated to $3.14$ for convenience), the game
stops. In this example, we have a continuous random variable with uniform
distribution that indicates the angle of the wheel, plus a constraint on its
value in the clauses for the pick/2 predicate:

    1/3 :: spades(_).
    1/2 :: clubs(_).
    angle(_,X) : uniform_dens(X,0,6.28).

    pick(0,spades) :- spades(0), angle(0,V), V > 3.14.
    pick(0,clubs) :- \+ spades(0), clubs(0), angle(0,V), V > 3.14.
    pick(0,hearts) :- \+ spades(0), \+ clubs(0), angle(0,V), V > 3.14.

    pick(s(X),spades):- \+ pick(X,hearts), spades(s(X)),
        angle(s(X),V), V > 3.14.
    pick(s(X),clubs):- \+ pick(X,hearts), \+ spades(s(X)),
        clubs(s(X)), angle(s(X),V), V > 3.14.
    pick(s(X),hearts):- \+ pick(X,hearts), \+ spades(s(X)),
        \+ clubs(s(X)), angle(s(X),V), V > 3.14.

    at_least_once_spades :- pick(_,spades).
    never_spades :- \+ at_least_once_spades.

With the query at_least_once_spades we can compute the probability that the
player picks at least one time spades when the axis of the wheel is between
$\pi$ (3.14) and $2\pi$ (6.28). The continuous random variables are
represented by the second argument of predicate $angle(T,X)$: they form a
countable set, and there is a continuous random variable for each value of T
(0, $s(0)$, $s(s(0))$, …). The set $Y$ of discrete Boolean random variables is
composed of $\\{Y_{i}^{c},Y_{i}^{s}\mid i=1,2,\dots\\}$. $Y^{c}_{i}$
($Y^{s}_{i}$) represents $clubs(s^{i}(0))$ ($spades(s^{i}(0))$), and
$y^{c}_{i}$ ($y^{s}_{i}$) are values for $Y_{i}^{c}$ ($Y_{i}^{s}$). Similarly,
the set $X$ of continuous random variables is composed of $\\{X_{i}\mid
i=1,2,\dots\\}$. Each $X_{i}$ has a range $[0,2\pi]$, and its value is denoted
with $x_{i}$. To compute the probability of this query, we can consider the
mutually disjoint covering set of worlds
$\omega=\omega_{0}\cup\omega_{1}\cup\dots$ where

$\displaystyle\omega_{0}={}$
$\displaystyle\\{(w_{\mathrm{X}},w_{\mathrm{Y}})\mid
w_{\mathrm{X}}=(x_{1},x_{2},\ldots),w_{\mathrm{Y}}=(y^{c}_{1},y^{s}_{1},y^{c}_{2},y^{s}_{2},\ldots),$
$\displaystyle x_{1}\in[\pi,2\pi],y^{s}_{1}=1\\}$ $\displaystyle\omega_{1}={}$
$\displaystyle\\{(w_{\mathrm{X}},w_{\mathrm{Y}})\mid
w_{\mathrm{X}}=(x_{1},x_{2},\ldots),w_{\mathrm{Y}}=(y^{c}_{1},y^{s}_{1},y^{c}_{2},y^{s}_{2},\ldots),$
$\displaystyle
x_{1}\in[\pi,2\pi],y^{s}_{1}=0,y^{c}_{1}=1,x_{2}\in[\pi,2\pi],y^{s}_{1}=1\\}$
$\displaystyle\ldots$

That is, for explanation $\omega_{0}$ spades was selected at round 0
($y^{s}_{1}=1$) and the wheel ($x_{1}$) in the same round was in the range
$[\pi,2\pi]$. For explanation $\omega_{1}$, spades was not selected at round 0
($y^{s}_{1}=0$), clubs was selected at round 0 ($y^{c}_{1}=0$), the wheel
($x_{1}$) was in the range $[\pi,2\pi]$ at round 0, spades was selected at
round $s(0)$ ($y^{s}_{2}=1$) and the wheel ($x_{2}$) was in the range
$[\pi,2\pi]$ at round $s(0)$. The probability for $\omega_{0}$ can be computed
as [4, 7]:

$\displaystyle\mu(\omega_{0})={}$
$\displaystyle\int_{\pi}^{2\pi}\mu_{\mathrm{Y}}(\\{(y^{c}_{1},y^{s}_{1},y^{c}_{2},y^{s}_{2},\ldots)\mid
y^{c}_{1}=1\\})\ d\mu_{\mathrm{X}}$ $\displaystyle={}$
$\displaystyle\int_{\pi}^{2\pi}\frac{1}{3}\cdot\frac{1}{2\pi}dx_{1}=\frac{1}{3}\cdot\frac{1}{2}=\frac{1}{6}.$

where $\frac{1}{3}$ is the contribution of the discrete random variable
(spades) and $\frac{1}{2\pi}$ is the contribution of the continuous one
(angle). The probability for the other $\omega_{i}$ can be computed in a
similar way. Overall, considering the limits, we get
$\frac{1}{3}\cdot\frac{1}{2}\cdot\sum_{i=0}^{\infty}(\frac{2}{3}\cdot\frac{1}{2}\cdot\frac{1}{2})^{i}=\frac{1}{6}\cdot\sum_{i=0}^{\infty}(\frac{1}{6})^{i}=\frac{1}{6}\cdot\frac{6}{5}=\frac{1}{5}$
as probability for the query at_least_once_spades.

######  Example 4 (Gaussian mixture)

A Gaussian mixture model is a way to generate values of a continuous random
variable: a discrete random variable is sampled and, depending on the sampled
value, a different Gaussian distribution is selected for sampling the value of
the continuous variable.

A Gaussian mixture model with two components can be expressed in cplint hybrid
programs as 111http://cplint.eu/e/gaussian_mixture.pl:

    h : 0.6.
    heads :- h.
    tails :- \+ h.
    g(X) : gaussian(X, 0, 1).
    h(X) : gaussian(X, 5, 2).
    mix(X) :- heads, g(X).
    mix(X) :- tails, h(X).
    mix :- mix(X), X > 2.

The argument X of mix(X) follows a distribution that is a mixture of two
Gaussians, one with mean 0 and variance 1 with probability 0.6, and one with
mean 5 and variance 2 with probability 1 - 0.6 = 0.4. We can then ask for the
probability of mix.

Here, predicates g/1, h/1, and mix/1 have a single argument which can hold
continuous variables, so overall there is a finite set of continuous random
variables. Since there are no term variables, each atom for these predicates
in a world univocally determines its argument. For predicate mix/1 this is not
obvious as there are two clauses for it. However, the two clauses have
mutually exclusive bodies, i.e., in each world only one of them is true. This
property is further discussed in Section 4.

######  Example 5 (Gaussian mixture and constraints, from [12])

Consider a factory with two machines, a and b. Each machine produces a widget
with a continuous feature. A widget is produced by machine a with probability
0.3 and by machine b with probability 0.7. If the widget is produced by
machine a, the feature is distributed as a Gaussian with mean 2 and variance
1. If the widget is produced by machine b, the feature is distributed as a
Gaussian with mean 3 and variance 1. The widget then is processed by a third
machine that adds a random quantity to the feature. The quantity is
distributed as a Gaussian with mean 0.5 and variance 1.5. This can be encoded
by in cplint hybrid programs as 222http://cplint.eu/e/widget.pl:

    machine(a) : 0.3.
    machine(b) :- \+ machine(a).
    st(a,Z) : gaussian(Z, 2, 1).
    st(b,Z) : gaussian(Z, 3, 1).
    pt(Y) : gaussian(Y, 0.5, 1.5).
    widget(X) :- machine(M), st(M,Z), pt(Y), X =:= Y + Z.
    ok_widget :- widget(X), X > 1.0.

We can then ask the probability of ok_widget.

Here, X, Y, and Z are continuous variables and M is a term variable. Since X
is a continuous variable, in every world there should be a single value for X
that makes widget(X) true. Predicate widget/1 has a single clause but the
clause has two groundings, one for M = a, and one for M = b, so in principle
there could be two values for X in true groundings of widget(X). However, as
in Example 4, the two groundings of the rule have mutually exclusive bodies,
as in each world either machine(a) is true or machine(b) is true, but not
both.

######  Example 6 (Estimation of the mean of a Gaussian)

Consider now this example
333http://cplint.eu/example/inference/gauss_mean_est.pl:

    mean(M) : gaussian(M,1,5).
    value(_,M,X) : gaussian(X,M,2).
    value(I,X) :- mean(M), value(I,M,X).

The program states that, for an index I, the continuous variable X is sampled
from a Gaussian whose variance is 2 and whose mean M is sampled from a
Gaussian with mean 1 and variance 5.

This program can be used to estimate the mean of a Gaussian by querying
mean(M) given observations for atom value(I,X) for different values of I.

Here, the first argument of value/3 can hold a term variable while its second
and third arguments can hold a continuous variable. The second argument is
used as a parameter in the probability density of the third argument. It is
not immediate to see whether this program has a well-defined semantics or not.
In fact, the semantics does not allow specifying the parameters of continuous
distributions with values computed by the program, but we can consider
continuous variables M and X as specified by a joint density (see Section 4).
Since a Gaussian density with a Gaussian mean is still a Gaussian, the joint
density will be a multivariate Gaussian.

In the next Section, we illustrate more in detail the syntactic requirements
that hybrid probabilistic logic programs, and thus probabilistic constraint
logic programs, must follow to have a well-defined semantics.

##  4 Syntactic Requirements

To preserve the well-definedness, we require that the set of random variables
must be countable. That is, every random variable must be associated with a
ground logical atom. The discrete arguments of this logical atom can contain
terms and cannot be real values. Let us now analyse several possible
situations.

In the fourth clause of Example 3, (reported here for clarity)

    pick(0,spades) :- spades(0), angle(0,V), V > 3.14.

the first argument of the continuous random variable angle is a constant.
Similarly happens with the clauses with angle(s(X),V). Variable X appears in
the head, so it is ground when angle(s(X),V) is called (during the recursion
or by directly set a value for X in the query), and the continuous random
variable V is associated with term s(X) by predicate angle/2. In both cases,
s(X) is a ground logical term (integer in case of 0), so the semantics is
well-defined.

Consider now Example 4: in this program there are two random variables,
identified respectively by g(X) and h(X). Predicate mix/1 is composed of two
clauses with the same head but different bodies, a situation that may lead to
an ill-defined program where both clauses are true and the random variable X
is defined by two different distributions. However, the two bodies are
mutually exclusive: the discrete random variable h is used to discriminate
between the two. When h is true, heads is true, and thus the first clause of
mix/1 is considered. When it is false, the second mix/1 clause is considered.
This mutual exclusivity guarantees the well-definedness of the program.
Algorithmically, this property can be verified performing clause unfolding,
obtaining two clauses with the same head with bodies mutually exclusive, since
they contain at least one atom in common, but in one it is positive, in the
other it is negated. The same happens for Example 5. This idea can be extended
to the case where there are $n$ clauses with the same head defining $n$
different distributions for the same variable, provided that all the clauses
are mutually exclusive.

To see a counterexample, if we modify the clauses for predicate mix/1 of
Example 4 in this way:

    mix(X) :- g(X).
    mix(X) :- h(X).

the program is ill-defined, since the two clauses are not mutually exclusive,
and there is not a single distribution for variable X.

Focus now on Example 6. Here, atom value/3 has a continuous random variable
(M) as input. To keep the well-definedness, this input must be defined in
another continuous probabilistic fact and must be used only as a parameter for
another distribution. In fact, M is obtained from a Gaussian distribution and
it is used as a parameter for the mean for another Gaussian distribution. In
this way, variables M and X are specified by a joint density. Note that the
first variable of value/3 is a term variable and can be associated with a
ground atom (integer, for example), preserving the well-definedness: the value
of the term variable uniquely identifies the value of the continuous random
variable. In this way, the position in the sequence of random variables is
exactly identified.

This situation can be straightforwardly extended to programs in which there
are multiple continuous random variables as input: for instance, in Example 6,
the variance for X in clause value/3 can be sampled from another distribution
instead of being fixed to 2.

In other cases, when the value of a continuous random variable is used as
variable for another term, but not as a parameter for a distribution, the
semantics is ill-defined. Consider the following modification to Example 6:

    value(X) : gaussian(X,1,5).
    value(_,M) : gaussian(M,2,2).
    res(M) :- value(X), value(X,M).

Here, X in clause res/1 is the value of a continuous random variable but it is
not used as parameter for the distribution of M. The continuous random
variable M cannot be associated with a term that uniquely identifies it, so
the program is ill-defined. The difference with Example 6 is that in Example 6
the term variable used as index (I) can be associated with a ground logical
term, while here it is associated with a real value (X). If in this last
example the variable X is a discrete random variable rather than a continuous
one, the program would be well-defined, since X is associated with a ground
logical atom.

To see how to compute the probability in this situation, consider the
following simple program (a simplification of Example 6, where Gaussian
distribution are replaced by Uniform distributions identified by
uniform_dens/3):

###### Example 7

    angle_a(_,X,Y) : uniform_dens(Y,X,2).
    angle_b(X) : uniform_dens(X,0,1).
    success(I) :- angle_b(X), angle_a(I,X,Y), Y < 1.5.

Here, the variable X in angle_b/1 follows a uniform distribution between 0 and
1. For variable Y in angle_a/1, its lower bound is sampled from another
uniform distribution. The probability distribution defined by the two random
variables can be considered as a joint distribution between the two, so they
can be treated as one multivariate random variable in the sequence of random
variables, indexed by the logical term unified with I. Their joint density
function is given by

$f_{XY}(x,y)=\frac{1}{1-0}\cdot\frac{1}{2-x}=\frac{1}{2-x}$

and thus the probability of the query success(0) is

$\displaystyle P(success(0))=P[Y<1.5]$
$\displaystyle=\int_{0}^{1}\frac{1}{2-x}\left(\int_{x}^{1.5}1\ dy\right)dx=$
$\displaystyle=\int_{0}^{1}\frac{1.5-x}{2-x}\ dx=1-\frac{ln(2)}{2}\approx
0.653$

As said in Section 2, the range-restrictedness property ensures that the
answers of a query are always ground instantiations of it. However, there can
be some exceptions. Consider the first three lines of Example 3, reported here
for clarity:

    1/3 :: spades(_).
    1/2 :: clubs(_).
    angle(_,X) : uniform_dens(X,0,6.28).

Both discrete and continuous probabilistic facts are not range restricted,
since they contain an input variable (anonymous) that is not ground. To ensure
that the answers of a query are always ground instantiations of it, we need to
make sure that, when these types of facts are called, all the input variables
are ground. We then impose that the arguments of probabilistic facts that are
not ground must be variables that appear in previous literal in the body: the
preceding calls will bind these variables, so the probabilistic facts will be
called with all input arguments ground, and a probability value will be
assigned to a ground query. The order of the terms is fundamental: the
variables in a probabilistic fact must appear in a preceding literal, to
ensure the well-definedness. Consider the following clarifying example:

    a(1).
    f(_,X) : uniform_dens(X,0,6).

    g0(X):- a(X), f(X,V), V > 1.
    g1(X):- f(X,V), V > 1, a(X).

For the query g0(X), when f(X,Y) is called, X has already been substituted
with 1, and f/2 has a ground input variable. This does not hold for query
g1(X) (even if the terms in the body are the same, they are in different
order), since f/2 is called with the input variable not instantiated, thus
violating the syntactic requirements.

To sum up, if we consider the following simple program:

    ud(_,V) : uniform_dens(V,0,6).
    1/2 :: a(_).
    b(X,1):- a(X).
    b(X,2):- \+a(X).
    n(1).
    f_0(X):- a(X).
    f_1:- a(1).
    f_2(X):- n(X), a(X).
    f_3:- a(_).
    f_4(X):- b(X,V), a(V),
    f_5:- ud(1,V), V > 2.

the answers of the queries f_0(1), f_1, f_2(X), and f_5 are ground
instantiations of them, since the input term variable for probabilistic facts
a/1 and ud/2 is bounded to 1 in all the queries. Similarly happens for f_4(1),
where V can unify with 1 or 2, depending on the truth value of a. On the
contrary, the answers for f_0(_), f_3, and f_4(_) are not ground
instantiations, since the term variable of the probabilistic fact a is not
ground (for the first two queries) or the input variable X of b/2 is not
ground (for the third query).

##  5 Conclusions

In this paper, following the semantics proposed in [4], we delineated a
precise syntax and explained the necessary syntactic conditions to maintain
the well-definedness for hybrid probabilistic logic programs. We covered
several cases in which ill-definedness may arise, providing examples and
counterexamples. As future work, we plan to develop algorithms to
automatically check these properties as well as developing new inference
algorithms that can manage infinite domains [6].

## References

  * [1]
  * [2] Marco Alberti, Elena Bellodi, Giuseppe Cota, Fabrizio Riguzzi & Riccardo Zese (2017): _cplint on SWISH: Probabilistic Logical Inference with a Web Browser_.  Intelligenza Artificiale 11(1), pp. 47–64, 10.3233/IA-170105.
  * [3] Damiano Azzolini, Fabrizio Riguzzi & Evelina Lamma (2019): _Studying Transaction Fees in the Bitcoin Blockchain with Probabilistic Logic Programming_.  Information 10(11), p. 335, 10.3390/info10110335.
  * [4] Damiano Azzolini, Fabrizio Riguzzi & Evelina Lamma (2021): _A Semantics for Hybrid Probabilistic Logic Programs with Function Symbols_.  Artificial Intelligence 294, p. 103452, 10.1016/j.artint.2021.103452.
  * [5] Damiano Azzolini, Fabrizio Riguzzi, Evelina Lamma & Franco Masotti (2019): _A Comparison of MCMC Sampling for Probabilistic Logic Programming_.  In Mario Alviano, Gianluigi Greco & Francesco Scarcello, editors: Proceedings of the 18th Conference of the Italian Association for Artificial Intelligence (AI*IA2019), Rende, Italy 19-22 November 2019, Lecture Notes in Computer Science, Springer, Heidelberg, Germany, 10.1007/978-3-030-35166-3_2.
  * [6] Vaishak Belle (2020): _Symbolic Logic Meets Machine Learning: A Brief Survey in Infinite Domains_.  In Jesse Davis & Karim Tabia, editors: Scalable Uncertainty Management, Springer International Publishing, Cham, pp. 3–16, 10.1007/978-3-030-58449-8_1.
  * [7] Y.S. Chow & H. Teicher (2012): _Probability Theory: Independence, Interchangeability, Martingales_.  Springer Texts in Statistics, Springer.
  * [8] Fabio Gagliardi Cozman & Denis Deratani Mauá (2017): _On the Semantics and Complexity of Probabilistic Logic Programs_.  Journal of Artificial Intelligence Research 60, pp. 221–262, 10.1613/jair.5482.
  * [9] Luc De Raedt & Angelika Kimmig (2015): _Probabilistic (Logic) Programming Concepts_.  Machine Learning 100(1), pp. 5–47, 10.1007/s10994-015-5494-z.
  * [10] Luc De Raedt, Angelika Kimmig & Hannu Toivonen (2007): _ProbLog: A Probabilistic Prolog and Its Application in Link Discovery_.  In Manuela M. Veloso, editor: 20th International Joint Conference on Artificial Intelligence (IJCAI 2007), 7, AAAI Press/IJCAI, pp. 2462–2467.  Available at http://www.ijcai.org/papers07/Papers/IJCAI07-396.pdf.
  * [11] Bernd Gutmann, Manfred Jaeger & Luc De Raedt (2011): _Extending ProbLog with Continuous Distributions_.  In Paolo Frasconi & Francesca A. Lisi, editors: 20th International Conference on Inductive Logic Programming (ILP 2010), LNCS 6489, Springer, pp. 76–91, 10.1007/978-3-642-21295-6_12.
  * [12] Muhammad Asiful Islam, CR Ramakrishnan & IV Ramakrishnan (2012): _Inference in probabilistic logic programs with continuous random variables_.  Theory and Practice of Logic Programming 12, pp. 505–523, 10.1017/S1471068412000154.
  * [13] Steffen Michels, Arjen Hommersom, Peter J. F. Lucas & Marina Velikova (2015): _A new probabilistic constraint logic programming language based on a generalised distribution semantics_.  Artificial Intelligence 228, pp. 1–44, 10.1016/j.artint.2015.06.008.
  * [14] Stephen Muggleton (2000): _Learning Stochastic Logic Programs_.  In Lise Getoor & David Jensen, editors: Learning Statistical Models from Relational Data, Papers from the 2000 AAAI Workshop, AAAI Workshops WS-00-06, AAAI Press, pp. 36–41.
  * [15] Arnaud Nguembang Fadja & Fabrizio Riguzzi (2017): _Probabilistic Logic Programming in Action_.  In Andreas Holzinger, Randy Goebel, Massimo Ferri & Vasile Palade, editors: Towards Integrative Machine Learning and Knowledge Extraction, LNCS 10344, Springer, 10.1007/978-3-319-69775-8_5.
  * [16] David Poole (1997): _The Independent Choice Logic for Modelling Multiple Agents Under Uncertainty_.  Artificial Intelligence 94(1-2), pp. 7–56, 10.1016/S0004-3702(97)00027-1.
  * [17] Fabrizio Riguzzi (2013): _MCINTYRE: A Monte Carlo System for Probabilistic Logic Programming_.  Fundamenta Informaticae 124(4), pp. 521–541, 10.3233/FI-2013-847.
  * [18] Fabrizio Riguzzi (2016): _The Distribution Semantics for Normal Programs with Function Symbols_.  International Journal of Approximate Reasoning 77, pp. 1–19, 10.1016/j.ijar.2016.05.005.
  * [19] Fabrizio Riguzzi (2018): _Foundations of Probabilistic Logic Programming_.  River Publishers, Gistrup, Denmark.
  * [20] Fabrizio Riguzzi, Elena Bellodi, Evelina Lamma, Riccardo Zese & Giuseppe Cota (2016): _Probabilistic Logic Programming on the Web_.  Software: Practice and Experience 46(10), pp. 1381–1396, 10.1002/spe.2386.
  * [21] Fabrizio Riguzzi & Terrance Swift (2013): _Well-Definedness and Efficient Inference for Probabilistic Logic Programming under the Distribution Semantics_.  Theory and Practice of Logic Programming 13(2), pp. 279–302, 10.1017/S1471068411000664.
  * [22] Taisuke Sato (1995): _A Statistical Learning Method for Logic Programs with Distribution Semantics_.  In Leon Sterling, editor: Logic Programming, Proceedings of the Twelfth International Conference on Logic Programming, Tokyo, Japan, June 13-16, 1995, MIT Press, pp. 715–729.
  * [23] A. Van Gelder, K. A. Ross & J. S. Schlipf (1991): _The Well-founded Semantics for General Logic Programs_.  Journal of the ACM 38(3), pp. 620–650, 10.1145/116825.116838.
  * [24] J. Vennekens, Marc Denecker & Maurice Bruynooghe (2009): _CP-logic: A language of causal probabilistic events and its relation to logic programming_.  Theory and Practice of Logic Programming 9(3), pp. 245–308, 10.1017/S1471068409003767.
  * [25] Joost Vennekens, Sofie Verbaeten & Maurice Bruynooghe (2004): _Logic Programs With Annotated Disjunctions_.  In Bart Demoen & Vladimir Lifschitz, editors: 20th International Conference on Logic Programming (ICLP 2004), LNCS 3131, Springer, pp. 431–445, 10.1007/978-3-540-27775-0_30.
  * [26] Pedro Zuidberg Dos Martires, Anton Dries & Luc De Raedt (2018): _Knowledge Compilation with Continuous Random Variables and its Application in Hybrid Probabilistic Logic Programming_.  CoRR abs/1807.00614.


# Improving Topic Relevance Model by Mix-structured Summarization and LLM-
based Data Augmentation

Yizhu Liu Ran Tao Shengyu Guo Yifan Yang liuyizhu, taoran04, shengyu.guo,
<EMAIL_ADDRESS>Meituan DianpingShanghaiChina

(2018)

###### Abstract.

Topic relevance between query and document is a very important part of social
search, which can evaluate the degree of matching between document and user’s
requirement. In most social search scenarios such as Dianping, modeling search
relevance always faces two challenges. One is that many documents in social
search are very long and have much redundant information. The other is that
the training data for search relevance model is difficult to get, especially
for multi-classification relevance model. To tackle above two problems, we
first take query concatenated with the query-based summary and the document
summary without query as the input of topic relevance model, which can help
model learn the relevance degree between query and the core topic of document.
Then, we utilize the language understanding and generation abilities of large
language model (LLM) to rewrite and generate query from queries and documents
in existing training data, which can construct new query-document pairs as
training data. Extensive offline experiments and online A/B tests show that
the proposed approaches effectively improve the performance of relevance
modeling.

Topic Relevance Model, Mix-structured Summarization, Data Augmentation, Large
Language Model

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation emai; June 03–05, 2018; Woodstock, NY††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Information systems Relevance
assessment

##  1\. Introduction

In social search scenarios, topic relevance model aims at estimating the
relevance between user query and document, which reflects the matching degree
between document and user’s requirements. In order to differentiate the degree
to which user needs are satisfied, the topic relevance can be divided into 3
categories: strong relevance, weak relevance and irrelevance. Strong relevance
indicates that the document predominantly consists of information directly
aligned with the query. Weak relevance implies that the document contains only
a limited amount of query-related information, like confined to a single
sentence, with the remaining content being unrelated to the query. Irrelevance
signifies that the document’s content is entirely unrelated or contradictory
to the query. The input of neural-based topic relevance model consists of
query input and document input. Since the documents in social search are
always very long and contain a variety of parts that can meet different
queries, topic relevance model with whole document as input cannot effectively
learn the alignment between query and document. In addition, topic relevance
models suffer from lacking reliable training data consisting of query and
document pairs. Because data collected from online click behavior is not
accurate enough, and human annotation is difficult and costly. Thus, document
input optimization and training data augmentation are two challenges of topic
relevance modeling in social search scenarios.

Query |  Document |  Rel.
---|---|---
sakura |  Doc 1: March is the perfect time to visit Yuyuantan Park in Beijing, a stunning spot to capture sakura. The best viewing time for sakura is usually in the middle to late March, with only a week of full bloom that takes your breath away. Yuyuantan Park is particularly suitable for photography. The combination of the TV tower and sakura creates incredibly beautiful photos. |  Strong
Doc 2: Strolling through the hutongs in Beijing is an endlessly enjoyable activity, as it allows you to witness the ordinary lives of old Beijing while also experiencing a touch of artistic and cultural trends. The charming soul of these hutongs lies in the mix of taverns, restaurants, and small shops. From March to May, many flowers are in bloom, and the sakura in Yuyuantan Park are particularly beautiful. |  Weak
(a) Examples of query-document pairs with relevance label in test set.

Query |  Optimized Document |  Predicted Rel.
---|---|---
sakura |  Query-focused: From March to May, many flowers are in bloom, and the sakura in Yuyuantan Park are particularly beautiful. |  Strong
Mix-structured: Strolling through the hutongs in Beijing is an endlessly enjoyable activity, (document summary) the sakura in Yuyuantan Park are particularly beautiful. (query-focused summary) |  Weak
(b) Optimized document input for Doc 2 from different approaches and relevance
results predicted by relevance models trained with different inputs.

Table 1. (a) shows the examples of strong relevance and weak relevance. (b)
shows the performance of topic relevance models with document inputs of (a)
optimized by different methods. The bold sentens are related to query. “Rel.”
denotes relevance.

Query |  Document |  Rel.
---|---|---
Beef hot pot |  My favorate Beef hot pot. I love her hot pot in mini form, the pot base is only 10 yuan, and the flavor of the pot base is delicious and spicy, very satisfying! Overall, it’s great. |  Strong

Type |  Query |  Rel.
---|---|---
Syn. rewrite |  hot pot |  Strong
Ant. rewrite |  beef snacks |  Irrelevant
Generation |  mini hot pot¿pot base¿spicy |  Strong
Generation |  mini hot pot¿pot base¿spicy |  Weak

Table 2. Examples of rewritten and generated queries with their corresponding
documents. Queries from query generation are document keywords sorted
according to their importance. Relevance label (Rel.) corresponds to the bold
keyword.

The common method of document input optimization is extracting query-focused
summary from document as document input of relevance model (Zou et al., 2021).
Such methods retain the coarse-grained relevant contents and discard the rest
before measuring the semantic relevance between query and document, which can
better assist the model in determining whether the document contains
information that meets the user’s needs. However, only using query-focused
summaries as document input cannot teach the model to distinguish between
strong relevance and weak relevance, because query-focused summaries cannot
reflect the proportion of information in the document that is relevant to the
query. As shown in Table 1, given one query “sakura”, both Doc 1 and Doc 2
contain the information of sakura. The entire text of Doc 1 is describing
sakura, representing strong relevance. Doc 2 only has one sentence describing
the sakura, while other content describes the amusement facilities,
representing weak relevance. But the similar query-focused summaries for Doc 1
and Doc 2 make the topic relevance model view them as strong relevance to
query when using these summaries as modified document input. To solve above
problem, we propose a mix-structured summarization method to improve document
input of topic relevance model by connecting query-focused summary and
document summary without considering query. As shown in Table 1 (b), mix-
structured summaries refer to the information related to query in the document
and the core information of the document, leading to a clear differentiation
between strong and weak relevance.

As a neural-based model, topic relevance model requires sufficient data to
train. The Existing method of collecting training data of topic relevance
model involves manual labeling 111The data used to manually annotated is from
the search logs of online search engines. and click-through data filtering
(Yao et al., 2021). The former is costly, and the latter has a lot of noise.
Meanwhile, it is rare to find completely irrelevant query and document pairs
in real-world search engines, which leads to the trouble of introducing
irrelevant training samples (Jiang et al., 2019). Recently, Large Language
Models (LLMs) excel at handling general instructions and have shown promise in
data generation tasks (Wang et al., 2023; Schneider and Turchi, 2023). We
explore the methods for data augmentation in topic relevance model by using
LLM. We leverage the capability of LLMs in Nature Langauge Understanding (NLU)
and Nature Langauge Generation (NLG) to accomplish query rewriting and query
generation. For query rewriting, we design prompts to obtain synonyms and
antonyms of queries. Synonyms (Syn.) have the same semantics as query but use
different words, which will be paired with the corresponding document and
labeled. Antonyms (Ant.) have the opposite semantics to query but use similar
words. For query generating, we design a prompt to generate keywords from
document and take extracted keywords as new queries. As shown in Table 2, the
rewritten and generated queries will be combined with the corresponding
documents and labeled with reasonable relevance. LLM-based data augmentation
can generate sufficient training samples for different relevance categories,
which can help model training and enable the model to learn more knowledge.

In summary, this paper makes the following contributions:

  1. (1)

We propose a simple but effective method, Mix-structured Summarization, to
optimize the document input of topic relevance model. Mix-structured
Summarization is composed of query-based summary and general summary of
document, which can better differentiate the degree of relevance between query
and document. (Section 2.1)

  2. (2)

We explore the application of large language models in data augmentation for
topic relevance model. Through LLM-based data augmentation, the topic
relevance model can be better trained to learn alignment between query and
document in different relevance categories. (Section 2.2)

  3. (3)

Offline and online experiments show that the proposed approaches can improve
the performance of relevance scoring in social search. (Section 3.4 and
Section 3.5)

Both mix-structured summarization and LLM-based data augmentation can be
easily replicated and do not add to the burden of deploying models, making
them well-suited for long document relevance tasks in real social search.

##  2\. Approach

In this section, we first introduce our proposed mix-structured summarization
method which combines query-focused summary and general summary as document
input of topic relevance model, and then explain the LLM-based data
augmentation in detail.

To specify the degree to which user needs are met, our goal is to categorize
the relevance between queries and documents into three classes, i.e., strong
relevance, weak relevance and irrelevance. Strong relevance means that most of
the information in the document is a description of the query. Weak relevance
denotes that the document contains only a small amount of information related
to the query and the rest of the information is not relevant to the query.
Irrelevance represents that the information in the document is completely
inconsistent with the query. We take a 3-class BERT model as our topic
relevance model.

###  2.1. Mix-structure Summarization

Let $Q$ and $D$ denote the query and document separately. To assess the
relevance of query and document, it is necessary to consider both whether
there is information related to the query in the document and the proportion
of query-related information in the document. Thus, we first extract query-
focused summaries and common document summmaries from $D$, and then connect
them via the special token $SEP$ as mix-structured summary, as illustrated in
Figure 1. Taking query and mix-structured summary as relevance model input can
discern whether document is relevant and distinguish between strong relevance
and weak relevance.

Figure 1. Illustration of mix-structured summarization.

####  2.1.1. Query-focused Summarization

Following Zou et al. (2021), we extract summary $S_{query}$ from $D$ with
respect to a given query $Q$. We tokenize $Q$ to obtain a set of query-based
tokens $W_{q}$ We segment $D$ into sentences and tokenize each sentence. Each
sentence $s$ has a set of tokenized words $W_{s}$. To cover different tokens
in $W_{q}$,each token of $W_{q}$ will be signed a sentence, the $W_{s}$ of
which includes the token. Besides, we sequentially combine the selected
sentences with their adjacent sentences in the document, until the length of
the merged sequence meets the predetermined threshold. In this way, we can get
a query-focused summary, expressing query related information and its
contextual information in the document.

####  2.1.2. Document Summarization

In order to distinguish between strong and weak relevance documents that both
contain query related information, we extract common summary $S_{doc}$ from
$D$ without considering query. $S_{doc}$ should be comprehensive, capable of
representing all the key points in $D$. Inspired by Lead-3 222Lead-3 (Zhu et
al., 2023) is a commonly-used extractive summarization baseline that selects
first three sentences of document as summary. Although it is the simplest
method, it can yield a fairly good result., we select the first 3 sentences of
each paragraph in document and concatenate all selected sentences as summary.
Such extractive summary comprehensively abstracts the main information of the
document.

###  2.2. LLM-based Data Augmentation

It is difficult and costly to obtain sufficient and correct training data for
topic relevance model. The training data collected from real-world search
engines lacks weak relevance and irrelevance data. Therefore, we explore the
use of LLM to augment training data.

Each sample of topic relevance training data consists of three parts: query,
document and label. As shown in Figure LABEL:fig:llm, we expend relevance
training data of different categories by query rewriting and query generation.

####  2.2.1. Query Rewriting

We design two prompts (as shown in Figure 3) to generate synonyms and antonyms
of queries in the existing dataset. To increase the complexity of the
synthetic data, our prompts contain some constraints for generation. We also
provide some specific examples in the prompts, which help LLM understand our
requirements better.

Figure 2. The process of LLM-based query rewriting. Tuples of (Q, D, label)
are new samples.

For synonyms query rewriting, given a query $Q$, the prompt informs the LLM
that it is a synonym generator and asks it to generate synonyms for $Q$. The
synonyms of $Q$, together with the corresponding document of $Q$, constitute
new training samples. Their labels align with the label of $Q$ and its
associated document. This method is able to construct relevant data across
various categories. Furthermore, in order to enhance the robustness of the
training set, the prompt also limits LLM to generate synonyms that have the
same semantics as query but use different words and shows some hard-to-
distinguish synonyms. We expect LLM to generate high-quality and challenging
training sets, which can assist in improving the robustness of relevance
model.

For antonyms query rewriting, we select strong relevance and weak relevance
training data. Similar to synonyms query rewriting, LLM is asked as an antonym
generator, which needs to generate antonyms that have the opposite semantics
to query but use similar words. The antonyms of query $Q$, together with the
corresponding document of $Q$, should be labeled as irrelevance.

####  2.2.2. Query Generation

Document in strong relevance and weak relevance data must contain query
related information. Given a document $D$, we ask LLM to extract some keywords
from the document, which can be combined with the given document as strong
relevance or weak relevance samples. In the keyword generation prompt, we
require LLM to extract 3 keywords and output them in order of their importance
in the document. Keywords with the most importance and the given document form
a strong relevance sample. Keywords with the lowest importance and the given
document form a weak relevance sample.

Figure 3. The process of LLM-based query generation. Tuples of (Q, D, label)
are new samples.

By incorporating above new training samples, we can effectively enhance the
diversity and complexity of the training set, thereby improving the model’s
ability to handle a wider range of language scenarios.

##  3\. Evaluation

To demonstrate the effectiveness of our proposed method, we conduct
experiments on an industrial social search engine. Experimental results show
that for social search with long documents, the proposed mix-structured
summarization and LLM-based data augmentation can significantly improve the
performance of topic relevance modeling.

###  3.1. Datasets

In this experiment, the relevance dataset consists of human-labeled query-
document pairs from historical search results from Dianping Social Search.
Training set contains 46,306 samples, validation set contains 2,500 samples,
and test set contains 8514 samples. With the help of Dianping’s crowd-sourcing
platform, each of these query-document pairs is annotated as one of the three
relevance categories: strong relevance, weak relevance, and irrelevant.
Specifically, there are 33,453 strong relevant samples, 6,530 weak relevant
samples, and 6,323 irrelevant samples in training set.

###  3.2. Experimental Setup

We take BERT-based cross-encoder (Kenton and Toutanova, 2019) as basic model,
as it is widely used for relevance tasks in industrial search engines. The
methods compared in this experiment are listed below.

  * •

Baseline is a cross-encoder model with query-focused summary as document
input. Zou et al. (2021) showed that relevance modeling can be benefitted by
query-focused summary.

  * •

MSD-CE is a cross-encoder model with mix-structured summary as document input.

  * •

LLM-CE is a cross-encoder model with query-focused summary as document input,
which is trained on data augmented by query rewriting and query generation.

  * •

MSD-LLM-CE is a cross-encoder model with mix-structured summary as document
input, which is trained on data augmented by query rewriting and query
generation.

To be fair, the max input length of all relevance models is set as $192$. For
models with mix-structured summaries as document input, the max length of
query-focused summaries is $128$ and the max length of comment document
summaries is $64$. LLM used in our experiments is GPT-3.5-turbo.

###  3.3. Evaluation Metrics

We use the following evaluation metrics in this experiment.

Area Under Curve (AUC)  (Jiang et al., 2019; Yao et al., 2021) is widely used
for evaluating relevance models. It is an offline metric, which is used on
test set. As our relevance task has three categories, Inspired by Zan et al.
(2023), we compute multiclass AUC in our experiments as follows:

(1) $f(a,b)=\left\\{\begin{array}[]{rcl}1&&{a>b}\\\ 0&&{others}\\\
\end{array}\right.$

(2) $AUC=\frac{\sum_{j}\sum_{k}f(y_{j},y_{k})\cdot
f(p_{j},p_{k})}{\sum_{j}\sum_{k}f(y_{j},y_{k})}$

where $y_{j}$ and $p_{j}$ are the reference relevant score and predicted
relevant score for the $j^{th}$ query and document pair respectively. The
Relevant score for strong relevance, weak relevance, and irrelevant are 1,
0.7, and 0.

Good vs. Same vs. Bad (GSB) (Zhao et al., 2011) is a professional human
evaluation metric, which compares the search results from two same search
engines with different topic relevance models. Given a user query and two
returned documents from different relevance models, the annotators, who do not
know which relevance model the result belongs to, are required to label the
result with better relevance as “Good” and the one with poor relevance as
“Bad”. If these two results are equally good or bad, both of them should be
labeled as “Same”. As the query and return document pairs evaluated by GSB are
obtained from online sources, Following Yao et al. (2021), we use $\Delta GSB$
to reflect real user satisfaction.

(3) $\Delta GSB=\frac{\\#Good-\\#Bad}{\\#Good+\\#Same+\\#Bad}$

###  3.4. Offline Experimental Results

In this section, we analyze the performance of our proposed methods on an
offline test set. As shown in Table 3, after taking mix-structured summaries
as document input and augmenting training data by LLM, MSD-LLM-CE outperforms
all other models.

Model | AUC (%)
---|---
Baseline | 73.64
MSD-CE | 75.71
LLM-CE | 75.94
MSD-LLM-CE | 77.41
Table 3. Performance of different topic relevance models.

####  3.4.1. Query-focused Summarization vs. Mix-structured Summarization

Baseline and LLM-CE only take query-focused summaries as document input, which
ensures that the model can identify information in the document that matches
the query. MSD-CE and MSD-LLM-CE take mix-structured summaries as document
input, which provides more comprehensive information of document. As shown in
Table 3, MSD-CE and MSD-LLM-CE perform better than Baseline and LLM-CE
respectively, demonstrating that mix-structured summaries can assist the model
in identifying query-related information in the document and determining the
proportion of query-related information in the document. Table 1 shows that
query-focused summary may cause the model to confuse strong relevance and weak
relevance. Compared with query-focused summaries, mix-structured summaries can
guide the model to better distinguish between strong relevance and weak
relevance.

####  3.4.2. With LLM-based Data Augmentation vs. Without LLM-based Data
Augmentation

As shown in Table 4, after applying LLM-based data augmentation method on
training dataset, the total number of training samples has tripled. In
particular, the quantity of weak relevance and irrelevance training samples is
significantly increased. In Table 3, LLM-CE and MSD-LLM-CE outperform Baseline
and MSD-CE respectively, indicating that it is feasible to use LLM for data
augmentation. LLM-based data augmentation enhances the quantity and diversity
of training data, which helps improve the generalization ability and
robustness of the model.

Training set | Strong | Weak | Irrelevance | Overall
---|---|---|---|---
w/o Augmented | 33,453 | 6,530 | 6,323 | 46,306
with Augmented | 41,946 | 39,564 | 42,386 | 123,896
Table 4. Training data statistics for different relevance categories.

###  3.5. Online Experimental Results

Online A/B test between old online relevance model and new online relevance
model using our proposed approach shows that the new model can improve the
overall user experience in social search scenarios. Moreover, randomly
sampling 100 user queries, $\Delta GSB$ between old online relevance model and
new online relevance model is 0.1, which means new online model benefits from
our proposed approach.

##  4\. Related Work

In social search scenarios, the query and document (query-doc) are very
different in length, and the query may match only part of the document. Neural
methods (Dai et al., 2018; Huang et al., 2013; Mitra et al., 2017; Wu et al.,
2017; Kenton and Toutanova, 2019; Zan et al., 2023) have been flourishing for
their better capability to model semantic similarity.

Typical query-document search relevance modeling (Xu et al., 2018; Guo et al.,
2016; Xiong et al., 2017; Dai et al., 2018; Zou et al., 2021) can be divided
into two classes: global distribution of matching signals and local
distribution of matching signal. Models based on global distribution first
calculate the matching signals among the document for each query and then
aggregate the distribution. Guo et al. (2016) summarized the word-level
similarities by pyramid pooling. Xiong et al. (2017) introduced kernel-pooling
to summarize the translation matrix and provide soft match signals. Dai et al.
(2018) used CNN to represent n-grams of different lengths and softly match
them in a unified embedding space. As for models based on local distribution,
they first extract the local context of each query among document and conduct
matching between query and local context, and then aggregate the local
matching signals for relevance. Pang et al. (2017) proposed DeepRank to
extract relevant contexts and determine relevance score of each context first
and then aggregate the distribution of all local contexts. Zou et al. (2021)
extracted local relevance context of query from document and input the
concatenation of query and local context to the BERT model to compute their
relevance. Above two typical query-doc relevance models are not good at multi-
class relevance tasks, becase they cannot effectively balance the recognition
of query related information and the proportion of query related information.
Therefore, we propose to generate a mix-structure summary consisting of query-
focused summary and common-document summary as document input of relevance
model.

In recent years, BERT-based models (Kenton and Toutanova, 2019) get great
success in relevance tasks. However, such BERT-based models demand large-scale
training data. Collecting data by human annotation is costly and inefficient.
Annotated query and document according to user click is not accurate enough,
as user click is not only affected by relevance. It is common to encounter an
imbalanced distribution of training data across different relevance
categories, since completely irrelevant query and document pairs are difficult
to be found in real-world search engines (Jiang et al., 2019). Data
augmentation is an intuitive method to deal with above problems. Recent
research shows that LLMs demonstrate excellent generalization performance
across various natural language understanding and natural language generation
tasks in few-shot or zero-shot scenarios (OpenAI, 2023). Thus, we explore
generating query-doc training samples through LLM-based data augmentation,
such as query rewriting and query generation.

##  5\. Conclusion

In this work, we proposed two methods to enhance the multi-classification
topic relevance model: 1) took query-based summary and document summary
without query as document input of the topic relevance model, and 2) rewrite
and generate query from query and document separately by large language model
to construct new training data. The results showed that our methods can make
full use of document information to learn the relevance between query and
document and improve topic relevance model through data augmentation.

## References

  * (1)
  * Dai et al. (2018) Zhuyun Dai, Chenyan Xiong, Jamie Callan, and Zhiyuan Liu. 2018\.  Convolutional neural networks for soft-matching n-grams in ad-hoc search. In _Proceedings of the eleventh ACM international conference on web search and data mining_. 126–134.
  * Guo et al. (2016) Jiafeng Guo, Yixing Fan, Qingyao Ai, and W Bruce Croft. 2016\.  A deep relevance matching model for ad-hoc retrieval. In _Proceedings of the 25th ACM international on conference on information and knowledge management_. 55–64.
  * Huang et al. (2013) Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013\.  Learning deep structured semantic models for web search using clickthrough data. In _Proceedings of the 22nd ACM international conference on Information & Knowledge Management_. 2333–2338.
  * Jiang et al. (2019) Yunjiang Jiang, Yue Shang, Rui Li, Wen-Yun Yang, Guoyu Tang, Chaoyi Ma, Yun Xiao, and Eric Zhao. 2019\.  A unified neural network approach to e-commerce relevance learning. In _Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data_. 1–7.
  * Kenton and Toutanova (2019) Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of naacL-HLT_ , Vol. 1. 2.
  * Mitra et al. (2017) Bhaskar Mitra, Fernando Diaz, and Nick Craswell. 2017\.  Learning to match using local and distributed representations of text for web search. In _Proceedings of the 26th international conference on world wide web_. 1291–1299.
  * OpenAI (2023) OpenAI. 2023.  GPT-4 Technical Report.  arXiv:2303.08774 [cs.CL]
  * Pang et al. (2017) Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Jingfang Xu, and Xueqi Cheng. 2017\.  Deeprank: A new deep architecture for relevance ranking in information retrieval. In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management_. 257–266.
  * Schneider and Turchi (2023) Felix Schneider and Marco Turchi. 2023.  Team Zoom @ AutoMin 2023: Utilizing Topic Segmentation And LLM Data Augmentation For Long-Form Meeting Summarization. In _Proceedings of the 16th International Natural Language Generation Conference: Generation Challenges_ , Simon Mille (Ed.). 101–107.
  * Wang et al. (2023) Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023.  Self-Instruct: Aligning Language Models with Self-Generated Instructions. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). 13484–13508.
  * Wu et al. (2017) Yu Wu, Wei Wu, Chen Xing, Ming Zhou, and Zhoujun Li. 2017.  Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-Based Chatbots. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 496–505.
  * Xiong et al. (2017) Chenyan Xiong, Zhuyun Dai, Jamie Callan, Zhiyuan Liu, and Russell Power. 2017.  End-to-end neural ad-hoc ranking with kernel pooling. In _Proceedings of the 40th International ACM SIGIR conference on research and development in information retrieval_. 55–64.
  * Xu et al. (2018) Jun Xu, Xiangnan He, and Hang Li. 2018.  Deep learning for matching in search and recommendation. In _The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_. 1365–1368.
  * Yao et al. (2021) Shaowei Yao, Jiwei Tan, Xi Chen, Keping Yang, Rong Xiao, Hongbo Deng, and Xiaojun Wan. 2021.  Learning a product relevance model from click-through data in e-commerce. In _Proceedings of the Web Conference 2021_. 2890–2899.
  * Zan et al. (2023) Wen Zan, Yaopeng Han, Xiaotian Jiang, Yao Xiao, Yang Yang, Dayao Chen, and Sheng Chen. 2023.  SPM: Structured Pretraining and Matching Architectures for Relevance Modeling in Meituan Search. In _Proceedings of the 32nd ACM International Conference on Information and Knowledge Management_. 4923–4929.
  * Zhao et al. (2011) Shiqi Zhao, Haifeng Wang, Chao Li, Ting Liu, and Yi Guan. 2011.  Automatically generating questions from queries for community-based question answering. In _Proceedings of 5th international joint conference on natural language processing_. 929–937.
  * Zhu et al. (2023) Tianyu Zhu, Wen Hua, Jianfeng Qu, Saeid Hosseini, and Xiaofang Zhou. 2023.  Auto-regressive extractive summarization with replacement.  _World Wide Web_ 26, 4 (2023), 2003–2026.
  * Zou et al. (2021) Lixin Zou, Shengqiang Zhang, Hengyi Cai, Dehong Ma, Suqi Cheng, Shuaiqiang Wang, Daiting Shi, Zhicong Cheng, and Dawei Yin. 2021.  Pre-trained language model based ranking in Baidu search. In _Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining_. 4014–4022.


# Can exist a function, that transforms electromagnetic potentials from one to
other gauge?

Vladimir Onoochin

###### Abstract.

In this article, it is analyzed a problem of existence of a function which is
able to transform electromagnetic potentials defined in one gauge to
corresponding potentials defined in the other gauge. It is shown that such a
function cannot exist.

##  1\. Introduction

Despite electromagnetic potentials are treated within the classical
electrodynamics as a mathematical tool to calculate the physical quantities,
the EM fields, computation of these potentials is mostly forced procedure. The
Maxwell equations are solved by means of the wave equations for the scalar
$\varphi$ and vector ${\bf A}$ potentials. As it had been shown by Lorenz
still in 1867, solutions of these wave equations plus the condition connecting
the potentials (in the Gauss units),

(1.1) $\bm{\nabla}\cdot{\bf A}+\frac{1}{c}\frac{\partial\varphi}{\partial
t}=0\,,$

fully correspond to the Maxwell equations [1].

According to [2], Lorentz re-opened the gauge condition in 1904 by writing
that in order to have the potentials satisfy the ordinary wave equations they
must be related by (1.1). He discusses the arbitrariness in the potentials and
then states ’every other admissible pair ${\bf A}^{\prime}$ and
$\varphi^{\prime}$ can be related to the first pair ${\bf A}^{\prime}$ and
$\varphi^{\prime}$ via the transformations

(1.2)
$\begin{split}\varphi^{\prime}=\varphi-\frac{1}{c}{\frac{\partial{\chi_{L}}}{\partial
t}}\,,\\\ {\bf A}^{\prime}={\bf A}+\bm{\nabla}\chi_{L}\,,\end{split}$

where the function $\chi_{L}({\bf r},t)$ can be determined by subjecting ${\bf
A}^{\prime}$ and $\varphi^{\prime}$ to the condition (1.1) (Note 5 of [3])

(1.3) $\nabla\cdot{\bf
A}^{\prime}+\frac{1}{c}\frac{\partial\varphi^{\prime}}{\partial
t}=0\,,\quad\to\quad\nabla^{2}\chi_{L}-\frac{1}{c^{2}}\frac{\partial^{2}\chi_{L}}{\partial
t^{2}}=\nabla\cdot{\bf A}+\frac{1}{c}\frac{\partial\varphi}{\partial t}\,.$

So the gauge function in the Lorenz gauge obeys the homogeneous wave equation.

After these results of Lorentz, the function $\chi_{L}$ is called the gauge
function. Let us note that Lorentz introduced transformations (1.2) for the
potentials those obey the wave equations (p. 157 of [4]), or defined in the
Lorenz gauge.

Essential property of transformations (1.2) is that both sets of the
potentials, $\varphi,\,{\bf A}$ and $\varphi^{\prime},\,{\bf A}^{\prime}$,
give the same values of the EM fields since the derivatives of the gauge
function mutually compensate each other in the Maxwell equations.

Later these transformations have been extended to other gauges, first of all,
the Coulomb gauge. The latter is associated with Maxwell who derived the wave
equation for the magnetic field using the condition (in modern notation)

(1.4) $\bm{\nabla}\cdot\mathbf{A}=0\,.$

that was later called as the Coulomb gauge. Let us apply the transformations
of the same form as for the gauge function in the Lorenz gauge,

(1.5) ${\bf A}_{C}^{\prime}={\bf A}_{C}+\nabla\chi_{C}\,,$

where ${\bf A}_{C},\,{\bf A}_{C}^{\prime}$ are the vector potentials in the
Coulomb gauge. Using the gauge condition (1.4), one has

(1.6) $\bm{\nabla}\cdot{\bf A}_{C}^{\prime}=\bm{\nabla}\cdot{\bf
A}_{C}+\nabla^{2}\chi_{C}=0\,,$

Since both $\bm{\nabla}\cdot{\bf A}_{C}^{\prime}$ and $\bm{\nabla}\cdot{\bf
A}_{C}$ should be equal to zero in this gauge, one obtains that the Coulomb
gauge function obeys the Poisson equation $\nabla^{2}\chi_{C}=0$.

Although most problems in electrodynamics are solved in the Lorenz gauge,
where the wave equations for the potentials are separated, there are some
problems that are solved in another gauge. For example, due to the
difficulties of quantizing scalar photons, some problems of quantum
electrodynamics are solved in the Coulomb gauge, where the scalar potential is
considered as a static non-quantizable quantity. The use of different gauges
raises the question, are the electromagnetic fields calculated from the
potentials determined in different gauges equal? The study of this problem
raises another question: is it possible to find a function that transforms
electromagnetic potentials defined in one gauge into electromagnetic
potentials defined in another gauge. It will be shown below that such a
function cannot exist in the general case.

Since the most used gauges are the Lorenz and Coulomb ones, it is appropriate
to find a function that transforms the potentials of these gauges.

##  2\. The condition of the gauge function existence.

If one intends to seek such a gauge function, first of all one should be sure
that the derivatives of this function do not enter into the Maxwell equations.
So it is reasonable to choose the same form for transformations of the
potentials as it was introduced by Lorentz, or

(2.1)
$\displaystyle\varphi_{C}-\varphi_{L}=-\frac{1}{c}{\dfrac{\partial{\chi}}{\partial
t}}\,,$ (2.2) $\displaystyle{\bf A}_{C}-{\bf A}_{L}=\bm{\nabla}\chi\,,$

where the indices ”C” and ”L” refer to the potentials defined in the Coulomb
and Lorentz gauges, and $\chi$ is the required function. This function must
satisfy the main requirement for the gauge function, namely, in the Maxwell
equations, the derivatives of $\chi$ will eliminate each other and the gauge
function cannot be included in the Maxwell equations.

At the same time, fulfillment of Eqs. (2.1) and (2.2) provides the equivalence
of the EM fields calculated in both gauges. Actually,

$\begin{split}\bm{\nabla}\varphi_{C}-\bm{\nabla}\varphi_{L}=-\frac{1}{c}\bm{\nabla}{\frac{\partial{\chi}}{\partial
t}}\,,\\\ \frac{1}{c}\frac{\partial{\bf A}_{C}}{\partial
t}-\frac{1}{c}\frac{\partial{\bf A}_{L}}{\partial
t}=\frac{1}{c}\bm{\nabla}{\frac{\partial{\chi}}{\partial t}}\,.\end{split}$

Term-by-term summation of the above equations gives

$\bm{\nabla}\varphi_{C}+\frac{1}{c}\frac{\partial{\bf A}_{C}}{\partial
t}-\bm{\nabla}\varphi_{L}-\frac{1}{c}\frac{\partial{\bf A}_{L}}{\partial
t}=0\,\,\to\,\,{\bf E}_{L}-{\bf E}_{C}=0\,.$

In order for the above equation to be valid, one needs to find the functions
$\chi$, or at least show that this function exists. To realize this statement,
let us analyze the restrictions that are imposed on $\chi$ by Eqs. (2.1) and
(2.2).

Calculation of the divergence of both parts of Eq. (2.1) gives,

(2.3) $\bm{\nabla}\cdot{\bf A}_{C}-\bm{\nabla}\cdot{\bf
A}_{L}=\bm{\nabla}\cdot\bm{\nabla}\chi\,\,\quad{\rm
or}\,\,\quad-\bm{\nabla}\cdot{\bf A}_{L}=\nabla^{2}\chi\,,$

and after application of the Lorenz gauge condition to the lhs of the above
equation, one has

(2.4) $\frac{1}{c}\dot{\varphi_{L}}=\nabla^{2}\chi\,.$

The gauge function can be reconstructed from Eq. (2.4), so one obtains,

(2.5) $\chi=\frac{1}{4\pi}\int\frac{\nabla^{2}\chi({\bf r}^{\prime})}{|{\bf
r}-{\bf r}^{\prime}|}\mathtt{d}^{3}r^{\prime}=\frac{1}{4\pi
c}\int\frac{\dot{\varphi_{L}}({\bf r}^{\prime},t)}{|{\bf r}-{\bf
r}^{\prime}|}\mathtt{d}^{3}r^{\prime}\,.$

The last integral in (2.5) diverges as $\ln r$ since $\dot{\varphi_{L}}\simeq
1/r^{2}$ but let us skip this defect for a while.

Since the difference ${\bf A}_{C}-{\bf A}_{L}$ is given by the solution of the
wave equation with the rhs equal to
$\dfrac{1}{c}\bm{\nabla}\dfrac{\partial\varphi_{C}}{\partial t}$ (Eq. (3.3) of
[5]), and ${\bf A}_{C}-{\bf A}_{L}=\bm{\nabla}\chi$, one has,

(2.6) $\nabla\chi({\bf r},t)=\frac{1}{4\pi
c}\int\frac{[\bm{\nabla}\dot{\varphi}_{C}]_{ret}}{|{\bf r}-{\bf
r}^{\prime}|}\mathtt{d}^{3}r^{\prime}\,.$

Thus, gradient of the gauge function should be equal to the integral in rhs of
(2.5) and at the same time this gradient should be equal to the integral in
rhs of (2.6). Therefore, these integrals should be equal one other,

$\frac{1}{4\pi c}\int\frac{[\bm{\nabla}_{r^{\prime}}\dot{\varphi}_{C}({\bf
r}^{\prime},t^{\prime}]_{ret}}{|{\bf r}-{\bf
r}^{\prime}|}\mathtt{d}^{3}r^{\prime}=\frac{1}{4\pi
c}\bm{\nabla}_{r}\int\frac{\dot{\varphi_{L}}({\bf r}^{\prime})}{|{\bf r}-{\bf
r}^{\prime}|}\mathtt{d}^{3}r^{\prime}\,.$

where the argument $t^{\prime}$ in $\varphi_{C}$ corresponds to the retarded
time – actually, the change $t^{\prime}\,\to\,t_{r}=t-|{\bf r}-{\bf
r}^{\prime}|/c$ should be made after preforming all operations on $\Phi_{C}$

Typically, the scalar potential in the Lorenz gauge is expressed via the
retarded time (for example, the Liénard-Wiechert potentials). However, if
$\varphi_{L}$ is expressed in the present time coordinates, Eq. (2.7) is
simplified. Since in this case $\dot{\varphi_{L}}$ is function only of
$r^{\prime}$ and $t$,

$\bm{\nabla}_{r}\int\frac{\dot{\varphi_{L}}({\bf r}^{\prime},t)}{|{\bf r}-{\bf
r}^{\prime}|}\mathtt{d}^{3}r^{\prime}=\int\frac{\bm{\nabla}_{r^{\prime}}\dot{\varphi_{L}}({\bf
r}^{\prime},t)}{|{\bf r}-{\bf r}^{\prime}|}\mathtt{d}^{3}r^{\prime}\,,$

and

(2.7) $\frac{1}{4\pi
c}\int\frac{[\bm{\nabla}_{r^{\prime}}\dot{\varphi}_{C}({\bf
r}^{\prime},t]_{ret}}{|{\bf r}-{\bf
r}^{\prime}|}\mathtt{d}^{3}r^{\prime}=\frac{1}{4\pi
c}\int\frac{\bm{\nabla}_{r^{\prime}}\dot{\varphi_{L}}({\bf
r}^{\prime},t)}{|{\bf r}-{\bf r}^{\prime}|}\mathtt{d}^{3}r^{\prime}\,.$

The above relation must be fulfilled with necessity. If it is not fulfilled,
it means that there is no function providing the transformation of the
potentials from the Coulomb to Lorenz gauge, and otherwise.

It is difficult to verify this relation, since the integrals in it are too
cumbersome. However, these integrals can be computed in simplest case of
uniform motion of a single classical charge. Let us show how the integrals of
Eq. (2.7) can be computed.

##  3\. Computation of the integrals.

Verification of the fulfillment of relation (2.7), for example, for the $x$
component of $\bm{\nabla}\Phi$, can be made by consideration of potentials
created by a classical charge which is at rest until $t=0$, and then begins to
move with constant velocity $v$ along the $x$ axis.

In this case, only the $x$ component of integrals can have non-zero values.
Due to symmetry, the $y$ and $z$ components should be equal to zero.

If the charge starts its motion from the point $r=0$, the scalar potentials in
two gauges will be

$\Phi_{C}({\bf
r},t)=\left\\{\begin{array}[]{ccc}\dfrac{q\Theta(-t)}{\sqrt{x^{2}+y^{2}+z^{2}}}\,;\\\
\dfrac{q\Theta(t)}{\sqrt{(x-vt)^{2}+y^{2}+z^{2}}}\,;\end{array}\right.;\,\,\Phi_{L}({\bf
r},t)=\left\\{\begin{array}[]{ccc}\dfrac{q\Theta(-t)}{\sqrt{x^{2}+y^{2}+z^{2}}}\,;\\\
\dfrac{q\Theta(t)}{\sqrt{(x-vt)^{2}+[1-(v/c)^{2}][y^{2}+z^{2}]}}\,;\end{array}\right.$

where $q$ is a charge of the particle and $\Theta(t)$ is the step function.
Calculation of $\partial_{x}\dot{\Phi}$ for these potentials gives

(3.1) $\begin{split}\partial_{x}\dot{\Phi}_{C}({\bf
r},t)=\dfrac{vq\Theta(t)\left[y^{2}+z^{2}-2(x-vt)^{2}\right]}{\left[(x-vt)^{2}+y^{2}+z^{2}\right]^{5/2}}\,,\\\
\partial_{x}\dot{\Phi}_{L}({\bf
r},t)=\dfrac{vq\Theta(t)\left[[1-(v/c)^{2}](y^{2}+z^{2})-2(x-vt)^{2}\right]}{\left[(x-vt)^{2}+[1-(v/c)^{2}](y^{2}+z^{2})\right]^{5/2}}\,.\end{split}$

Here, it should be noted some feature of establishing the scalar potentials in
space after change of motion of the charge at $t=0$. If $\Phi_{C}({\bf r},t)$
is establishing in all space instantaneously, it is not the case for
$\Phi_{L}({\bf r},t)$.

’Switching on’ the potential of moving charge at a point $O$ (fig 1) with
${\bf r}=0$ at the instant $t=0$ means that a spherical ’wave of establishing’
of this potential in space begins to extend from $O$ and at an instant $t$ a
front of this wave will be at $R=ct$.

Let us choose the origin of the coordinate system is located at $r^{\prime}=0$
and a value of the external variable $r=0$. Then in spherical coordinates,
$r^{\prime},\,\theta,\,\phi$ the region of integration is the same for both
integrals (fig. 1). For $\Phi_{L}$, this region is determined by condition
that at the interval $[0;t]$ the potential of the moving charge can be
established inside the sphere with the radius $r^{\prime}=R=ct$. For
$\Phi_{C}$, this region is determined by the argument of the $\Theta$
function, $t-r^{\prime}/c>0$.

Figure 1. The sphere of integration. The radius of the sphere $R=ct$. The
charge begins its motion from a point $O(0,0,0)$, and its position at the
instant $t$ is at the point $A(vt,0,0)$, so $OA=vt$.

The symbol $[\,\,\,]_{ret}$ in Eq. (2.7) means that the nominator in its lhs
depends on the retarded time, or $t_{r}=t-r^{\prime}/c$. Then in the spherical
coordinates, the integrals are written as 111Due to axial symmetry (the $x$
axis is the axis of symmetry), integration over $\phi$ gives $2\pi$.

$\displaystyle\frac{q}{2c}\int\limits_{0}^{ct}r^{\prime}\mathtt{d}r^{\prime}\int\limits_{0}^{\pi}\dfrac{v\left[r^{\prime
2}(1-3\cos^{2}\theta)+4vr^{\prime}t_{r}\cos\theta-2v^{2}t_{r}^{2}\right]\sin\theta\mathtt{d}\theta}{\left[r^{\prime
2}+v^{2}t_{r}^{2}+2vr^{\prime}t_{r}\cos\theta\right]^{5/2}}=$ (3.2)
$\displaystyle=\frac{q}{2c}\int\limits_{0}^{ct}r^{\prime}\mathtt{d}r^{\prime}\int\limits_{0}^{\pi}\dfrac{v\left[r^{\prime
2}(1-v^{2}/c^{2})(1-\cos^{2}\theta)-2(r^{\prime}\cos\theta-
vt)^{2}\right]\sin\theta\mathtt{d}\theta}{\left[r^{\prime
2}(1-v^{2}/c^{2})(1-\cos^{2}\theta)+(r^{\prime}\cos\theta-
vt)^{2}\right]^{5/2}}\,,$

Both integrands contain the regular and singular terms, or the terms which
make the above integrals belong to improper ones. The latter appear due to the
second derivative of the scalar potential, and special procedure has been
developed for processing divergent (singular) terms (Ch. IV.5.5 of [6]). Thus,
computation of the integrals with these terms gives

$I_{C,{\rm sing}}=\frac{q}{3c}\int\frac{1}{\sqrt{x^{\prime 2}+y^{\prime
2}+z^{\prime
2}}}\dfrac{\delta(x^{\prime}-\tilde{x})\delta(y^{\prime})\delta(z^{\prime})}{\dfrac{c+v}{c}}\dfrac{\mathtt{d}x^{\prime}\mathtt{d}y^{\prime}\mathtt{d}z^{\prime}}{\tilde{x}-\dfrac{cvt}{c+v}}=\frac{q}{3ct}\,,$
$I_{L,{\rm
sing}}=\frac{q}{3c}\int\frac{\delta(x^{\prime}-vt)\delta(y^{\prime})\delta(z^{\prime})}{\sqrt{x^{\prime
2}+y^{\prime 2}+z^{\prime
2}}}\mathtt{d}x^{\prime}\mathtt{d}y^{\prime}\mathtt{d}z^{\prime}=\frac{q}{3ct}\,.$

So account of the singular terms of the integrands is equal for both
integrals. Now it is necessary to compute accounts of regular terms. After
change of variable $\cos\theta=\xi$, the first of integrals (3.2) becomes,

$\frac{q}{2c}\int\limits_{0}^{ct}r^{\prime}\mathtt{d}r^{\prime}\int\limits_{-1}^{1}\dfrac{v\left[r^{\prime
2}(1-3\xi)+4vr^{\prime}t_{r}\xi-2v^{2}t_{r}^{2}\right]\mathtt{d}\xi}{\left[r^{\prime
2}+v^{2}t_{r}^{2}+2vr^{\prime}t_{r}\xi\right]^{5/2}}=-\frac{q}{2c}\int\limits_{0}^{ct}r^{\prime}\mathtt{d}r^{\prime}\frac{2(1-{\rm
sgn}(r^{\prime}-vt_{r}))}{v^{2}t^{3}}$

where ${\rm sgn}(z)$ is a signum function of the real argument $z$. Thus,
calculation of this integral with respect to $r^{\prime}$ gives

(3.3)
$-\frac{q}{2c}\int\limits_{0}^{ct}r^{\prime}\mathtt{d}r^{\prime}\frac{2\left(1-{\rm
sgn}\left[r^{\prime}-v(t-r^{\prime}/c)\right]\right)}{v^{2}t^{3}}=-\frac{2q}{v^{2}\,c}\int\limits_{0}^{vct/(c+v)}\frac{r^{\prime}\mathtt{d}r^{\prime}}{(t-r^{\prime}/c)^{3}}=-\frac{q}{ct}\,.$

Calculation of the second integral in (3.2) with respect to $\xi$ by means of
Mathematica and then with respect to $r^{\prime}$ gives

(3.4)
$\begin{split}&\frac{q}{2c}\int\limits_{0}^{ct}r^{\prime}\mathtt{d}r^{\prime}\int\limits_{-1}^{1}\dfrac{v\left[r^{\prime
2}(1-v^{2}/c^{2})(1-\xi^{2})-2(r^{\prime}\xi-
vt)^{2}\right]\mathtt{d}\xi}{\left[r^{\prime
2}(1-v^{2}/c^{2})(1-\xi^{2})+(r^{\prime}\xi-vt)^{2}\right]^{5/2}}=\\\
&=-\frac{q}{2c}\int\limits_{0}^{ct}\mathtt{d}r^{\prime}\frac{4c^{4}tr^{\prime}\Theta(vt-r^{\prime})}{v^{2}(r^{\prime
2}-c^{2}t^{2})^{2}}=-\frac{q}{\left(1-{v^{2}}/{c^{2}}\right)ct}\,.\end{split}$

Since the results of integrations of Eqs. (3.3) and (3.4) are not equal each
other, and since the gauge function should correspond to both these
expressions, such a gauge function cannot exist in the general case.

##  4\. Conclusions.

In this work it is considered a problem of existence of so-called gauge
function that is able to transform the electromagnetic potentials defined in
one gauge to the corresponding potentials defined in another gauge. Except the
only requirement caused by the ’gauge invariance’, namely, that the Maxwell
equations cannot contain any gauge function including its derivatives, no
other specification of the form of the gauge function is used in the presented
analysis.

Meanwhile, the requirement of the gauge invariance, which determines the
relationship of potentials with derivatives of $\chi$–function, is
sufficiently strong to determine that this function must be defined through
scalar potentials. As a result, it turns out that this function is expressed
in terms of the scalar potential in the Coulomb gauge, Eq. (2.6) and at the
same time it is expressed through the scalar potential in the Lorenz gauge,
Eq. (2.5). The compatibility of these two conditions leads to Eq. (2.7), which
means that if this equation holds, then the gauge function can be found from
any of Eqs. (2.6) and (2.5). But if Eq. (2.7) does not hold, then the gauge
function cannot exist.

However, there is one obstacle to verify a validity of Eq. (2.7) in the
general case. The reason for this is that for such a verifying one needs to
know the explicit form of the potentials of the charge in an arbitrary motion,
written in the present time variables222$\Phi_{L}$ in Eq. (2.7) is a function
of $t$ but not of $t_{ret}$.. This explicit form is known, the expressions for
the Liénard–Wiechert potentials. But these expressions are written in terms of
retarded variables. To check the fulfillment of Eq. (2.7), it is necessary to
know the expressions for the scalar potentials written in the present time
variables.

But even if Eq. (2.7) does not hold in one particular case, this is sufficient
to conclude that the needed connection between $\Phi_{C}$ and $\Phi_{L}$ does
not exist. Such a special case is considered in Sec. III of this work and it
is shown that connection between the scalar potentials defined in two gauges
actually does not exist. Thus, even whether a problem of divergence of the
integral (2.5), detrmining the gauge function, is ignored, the obtained result
allows us to conclude that the required connection is generally invalid and,
therefore, there is no gauge function that provides the transformation of
electromagnetic potentials.

## References

  * [1] A. O’Rahilly. Electromagnetic theory, Vol. 1. Ch. VI. (Dover Publ. N.Y. 1965.)
  * [2] Jackson J. D. , and Okun L.B. Historical roots of gauge invariance. Rev. Mod. Phys. 73, 663, 2001.
  * [3] Lorentz, H. A., Theory of Electrons (G. B. Teubner, Leipzig and G. E. Stechert, New York. 1909).[2nd ed., 1916; reprinted by Dover Publications, New York. 1952]
  * [4] Lorentz, H. A., Encykl. Math. Wissen., Band V:2, Heft 1, V. 14, ‘Weiterbildung der Maxwellischen Theorie. Elektronentheorie,’ p. 145-280.
  * [5] Onoochin V. On non-equivalence of Lorentz and Coulomb gauges within classical electrodynamics. https://arxiv.org/abs/physics/0111017
  * [6] A. N. Tikhonov and A. A. Samarski, Equations of Mathematical Physics, (Dover Publications, Inc., New York, 1990).