Logic Tensor Networks

1,2]Samy Badreddinecor1

3]Artur d'Avila Garcez

4]Luciano Serafini

1,2]Michael Spranger

[cor1]Corresponding author
[1]organization=Sony Computer Science Laboratories Inc,
addressline=3-14-13 Higashigotanda,

[2]organization=Sony AI Inc,
addressline=1-7-1 Konan,

[3]
organization=City, University of London,
addressline=Northampton Square,
postcode=EC1V 0HB,
country=United Kingdom

[4]organization=Fondazione Bruno Kessler,
addressline=Via Sommarive 18,

Attempts at combining logic and neural networks into neurosymbolic approaches have been on the increase in recent years.
In a neurosymbolic system, symbolic knowledge assists deep learning, which typically uses a sub-symbolic distributed representation, to learn and reason at a higher level of abstraction.
We present Logic Tensor Networks (LTN), a neurosymbolic framework that supports querying, learning and reasoning with both rich data and abstract knowledge about the world.
LTN introduces a fully differentiable logical language, called Real Logic, whereby the elements of a first-order logic signature are grounded onto data using neural computational graphs and first-order fuzzy logic semantics.
We show that LTN provides a uniform language to represent and compute efficiently many of the most important AI tasks
such as multi-label classification, relational learning, data clustering, semi-supervised learning, regression, embedding learning and query answering.
We implement and illustrate each of the above tasks with several simple explanatory examples using TensorFlow 2. The results indicate that LTN can be a general and powerful framework for neurosymbolic AI.

Neurosymbolic AI Deep Learning and Reasoning Many-valued Logics.

§ INTRODUCTION

Artificial Intelligence (AI) agents are required to learn from their surroundings and reason about what has been learned to make decisions, act in the world, or react to various stimuli.
The latest Machine Learning (ML) has adopted mostly a pure sub-symbolic learning approach.
Using distributed representations of entities, the latest ML performs quick decision-making without building a comprehensible model of the world.
While achieving impressive results in computer vision, natural language, game playing, and multimodal learning, such approaches are known to be data inefficient and to struggle at out-of-distribution generalization. Although the use of appropriate inductive biases can alleviate such shortcomings, in general, sub-symbolic models lack comprehensibility. By contrast, symbolic AI is based on rich, high-level representations of the world that use human-readable symbols.
By rich knowledge, we refer to logical representations which are more expressive than propositional logic or propositional probabilistic approaches, and which can express knowledge using full first-order logic, including universal and existential quantification ($\forall x$ and $\exists y$), arbitrary $n$-ary relations over variables, e.g. $R(x,y,z,\dots)$, and function symbols,
e.g. $\mathrm{fatherOf}(x)$, $x+y$, etc.
Symbolic AI has achieved success at theorem proving, logical inference, and verification.
However, it also has shortcomings when dealing with incomplete knowledge. It can be inefficient with large amounts of inaccurate data and lack robustness to outliers. Purely symbolic decision algorithms usually have high computational complexity making them impractical for the real world.
It is now clear that the predominant approach to ML, where learning is based on recognizing the latent structures hidden in the data, is insufficient and may benefit from symbolic AI [17]. In this context, neurosymbolic AI, which stems from neural networks and symbolic AI, attempts to combine the strength of both paradigms (see [16, 40, 54] for recent surveys).
That is to say, combine reasoning with complex representations of knowledge (knowledge-bases, semantic networks, ontologies, trees, and graphs) with learning from complex data (images, time series, sensorimotor data, natural language).
Consequently, a main challenge for neurosymbolic AI is the grounding of symbols, including constants, functional and relational symbols, into real data, which is akin to the longstanding symbol grounding problem [30].

Logic Tensor Networks (LTN) are a neurosymbolic framework and computational model that supports learning and reasoning about data with rich knowledge.
In LTN, one can represent and effectively compute the most important tasks of deep learning with a fully differentiable first-order logic language, called Real Logic, which adopts infinitely many truth-values in the interval [0,1] [22, 25].
In particular, LTN supports the specification and computation of the following AI tasks uniformly using the same language:
data clustering,
relational learning,
query answering,
semi-supervised learning,
regression, and
embedding learning.

LTN and Real Logic were first introduced in [62].
Since then, LTN has been applied to different AI tasks involving perception, learning, and reasoning about relational knowledge.
In [18, 19], LTN was applied to semantic image interpretation whereby relational knowledge about objects was injected into deep networks for object relationship detection.
In [6], LTN was evaluated on its capacity to perform reasoning about ontological knowledge.
Furthermore, [7] shows how LTN can be used to learn an embedding of concepts into a latent real space by taking into consideration ontological knowledge about such concepts.
In [3], LTN is used to annotate a reinforcement learning environment with prior knowledge and incorporate latent information into an agent.
In [42], authors embed LTN in a state-of-the-art convolutional object detector.
Extensions and generalizations of LTN have also been proposed in the past years, such as LYRICS [47] and Differentiable Fuzzy Logic (DFL) [68, 69].
LYRICS provides an input language allowing one to define background knowledge using a first-order logic where predicate and function symbols are grounded onto any computational graph.
DFL analyzes how a large collection of fuzzy logic operators behave in a differentiable learning setting.
DFL also introduces new semantics for fuzzy logic implications called sigmoidal implications, and it shows that such semantics outperform other semantics in several semi-supervised machine learning tasks.

This paper provides a thorough description of the full formalism and several extensions of LTN. We
show using an extensive set of explanatory examples, how LTN can be applied to solve many ML tasks with the help of logical knowledge.
In particular, the earlier versions of LTN have been extended with:
(1) Explicit domain declaration: constants, variables, functions and predicates are now domain typed (e.g. the constants John and Paris can be from the domain of person and city, respectively). The definition of structured domains is also possible (e.g. the domain couple can be defined as the Cartesian product of two domains of persons); (2) Guarded quantifiers: guarded universal and existential quantifiers now allow the user to limit the quantification to the elements that satisfy some Boolean condition, e.g. $\forall x : \mathrm{age}(x) < 10 \: (\mathrm{playsPiano}(x) \rightarrow \mathrm{enfantProdige}(x))$ restricts the quantification to the cases where $age$ is lower than 10;
(3) Diagonal quantification: Diagonal quantification allows the user to write statements about specific tuples extracted in order from $n$ variables. For example, if the variables $\mathrm{capital}$ and $\mathrm{country}$ both have $k$ instances such that the $i$-th instance of $\mathrm{capital}$ corresponds to the $i$-th instance of $\mathrm{country}$, one can write $\forall \mathrm{Diag}(\mathrm{capital},\mathrm{country}) \: \mathrm{capitalOf}(\mathrm{capital},\mathrm{country})$.

Inspired by the work of [69], this paper also extends the product t-norm configuration of LTN with the generalized mean aggregator, and it introduces solutions to the vanishing or exploding gradient problems.
Finally, the paper formally defines a semantic approach to refutation-based reasoning in Real Logic to verify if a statement is a logical consequence of a knowledge base.
Example <ref> proves that this new approach can better capture logical consequences compared to simply querying unknown formulas after learning (as done in [6]).

The new version of LTN has been implemented in TensorFlow 2 [1]. Both the LTN library and the code for the examples used in this paper are available at <https://github.com/logictensornetworks/logictensornetworks>.

The remainder of the paper is organized as follows:
In Section <ref>, we define and illustrate Real Logic as a fully-differentiable first-order logic. In Section <ref>, we specify learning and reasoning in Real Logic and its modeling into deep networks with Logic Tensor Networks (LTN).
In Section <ref>, we illustrate the reach of LTN by investigating a range of learning problems from clustering to embedding learning.
In Section <ref>, we place LTN in the context of the latest related work in neurosymbolic AI. In Section
<ref> we conclude and discuss directions for future work. The Appendix contains information about the implementation of LTN in TensorFlow 2, experimental set-ups, the different options for the differentiable logic operators, and a study of their relationship with gradient computations.

§ THE REACH OF LOGIC TENSOR NETWORKS

The objective of this section is to show how the language of Real Logic can be used to specify a number of tasks that involve learning from data and reasoning. Examples of such tasks are classification, regression, clustering, and link prediction.
The solution of a problem specified in Real Logic is obtained by
interpreting such a specification in Logic Tensor Networks. The LTN library implements Real Logic in Tensorflow 2 [1] and is available from GitHub[https://github.com/logictensornetworks/logictensornetworks]. Every logical operator is grounded using Tensorflow primitives such that LTN implements directly a Tensorflow graph. Due to Tensorflow built-in optimization, LTN is relatively efficient while providing the expressive power of first-order logic. Details on the implementation of the examples
described in this section are reported in <ref>. The implementation of the examples presented here is also available from the LTN repository on GitHub.
Except when stated otherwise, the results reported are the average result over 10 runs using a 95% confidence interval.
Every example uses a stable real product configuration to approximate the Real Logic operators and the Adam optimizer [35] with a learning rate of $0.001$. Table <ref> in the Appendix gives an overview of the network architectures used to obtain the results reported in this section.

§.§ Binary Classification

The simplest machine learning task is binary classification.
Suppose that one wants to learn a binary classifier $A$ for a set of points in $[0,1]^2$.
Suppose that a set of positive and negative training examples is given.
LTN uses the following language and grounding:

$\mathrm{points}$ (denoting the examples).

$x_+$ for the positive examples.

$x_-$ for the negative examples.

$x$ for all examples.

$\domof(x) = \domof(x_+) = \domof(x_-) = \mathrm{points}$.

$A(x)$ for the trainable classifier.

$\domofin(A) = \mathrm{points}$.

\begin{align}
\forall x_+ & \ A(x_+) \\
\forall x_- & \ \neg A(x_-)
\end{align}

$\G(\mathrm{points}) = [0,1]^2$.

$\G(x) \in [0,1]^{m\times 2}$ ($\G(x)$ is a sequence of $m$ points, that is, $m$ examples).

$\G(x_+) = \left< d \in \G(x) \mid \lVert d - (0.5,0.5) \rVert < 0.09 \right>$.[$\G(x_+)$ are, by definition in this example, the training examples with Euclidean distance to the center $(0.5,0.5)$ smaller than the threshold of $0.09$.]

$\G(x_-) = \left< d \in \G(x) \mid \lVert d - (0.5,0.5) \rVert \geq 0.09 \right>$.[
$\G(x_-)$ are, by definition, the training examples with Euclidean distance to the centre $(0.5,0.5)$ larger or equal to the threshold of $0.09$.]

$\G(A \mid \theta) : x \mapsto \mathtt{sigmoid}(\mathtt{MLP}_\theta(x))$, where $\mathtt{MLP}$ is a Multilayer Perceptron with a single output neuron, whose parameters $\theta$ are to be learned[

Let us define $\bm D$ the data set of all examples.
The objective function with $\K=\{\forall x_+ \ A(x_+), \forall x_- \ \neg A(x_-)\}$ is given by $\argmax_{\bm\theta\in\bm\Theta}\ \Sat_{\phi\in\K}\G_{\bm\theta, x \leftarrow \bm D}(\phi)$.
The notation $\G_{x \leftarrow \bm D}(\phi(x))$ means that the variable $x$ is grounded with the data $\bm D$ (that is, $\G(x) := \bm D$) when grounding $\phi(x)$.
In practice, the optimizer uses the following loss function:

\bm L = (1 - \Sat_{\phi\in\K}\G_{\bm\theta,x \leftarrow \bm B}(\phi))

where $\bm B$ is a mini-batch sampled from $\bm D$.[
As usual in ML, while it is possible to compute the loss function and gradients over the entire data set, it is preferred to use mini-batches of the examples.
The objective and loss functions depend on the following hyper-parameters:

* the choice of fuzzy logic operator semantics used to approximate each connective and quantifier,

* the choice of hyper-parameters underlying the operators, such as the value of the exponent $p$ in any generalized mean,

* the choice of formula aggregator function.

Using the stable product configuration to approximate connectives and quantifiers, and $p=2$ for every occurrence of $\ApmeanError$, and using for the formula aggregator also $\ApmeanError$ with $p=2$, yields the following satisfaction equation:

\begin{align}
\Sat_{\phi \in \K}\G_\theta(\phi)
= & 1 - \frac{1}{2} \Big( 1 - \Big( 1 - \Big( \frac{1}{\lvert \G(x_+)\rvert} \sum\limits_{v \in \G(x_+)} \big(1-\mathtt{sigmoid}(\mathtt{MLP_\theta}(v))\big)^2 \Big)^{\frac{1}{2}\cdot 2}\Big) \notag \\
& \qquad + 1 - \Big( 1 - \Big( \frac{1}{\lvert \G(x_-)\rvert} \sum\limits_{v \in \G(x_-)} \big(\mathtt{sigmoid}(\mathtt{MLP_\theta}(v))\big)^2 \Big)^{\frac{1}{2}\cdot 2}\Big)
\Big)^{\frac{1}{2}} \notag
\end{align}

The computational graph of Figure <ref> shows $\Sat_{\phi\in\K}\G_{\bm\theta}(\phi))$ as used with the above loss function.

We are therefore interested in learning the parameters $\theta$ of the $\mathtt{MLP}$ used to model the binary classifier.
We sample 100 data points uniformly from $[0,1]^2$ to populate the data set of positive and negative examples.
The data set was split into 50 data points for training and 50 points for testing.
The training was carried out for a fixed number of 1000 epochs using backpropagation with the Adam optimizer [35] with a batch size of $64$ examples.
Figure <ref> shows the classification accuracy and satisfaction level of the LTN on both training and test sets averaged over 10 runs using a 95% confidence interval.
The accuracy shown is the ratio of examples correctly classified, with an example deemed as being positive if the classifier outputs a value higher than $0.5$.

Notice that a model can reach an accuracy of $100\%$ while satisfaction of the knowledge base is yet not maximized.
For example, if the threshold for an example to be deemed as positive is $0.7$, all examples may be classified correctly with a confidence score of $0.7$. In that case, while the accuracy is already maximized, the satisfaction of $\forall x_+ A(x_+)$ would still be $0.7$, and can still improve until the confidence for every sample reaches $1.0$.

This first example, although straightforward, illustrates step-by-step the process of using LTN in a simple setting.
Notice that, according to the nomenclature of Section <ref>, measuring accuracy amounts to querying the truth query (respectively, the generalization truth query) $A(x)$ for all the examples of the training set (respectively, test set) and comparing the results with the classification threshold.
In Figure <ref>, we show the results of such queries $A(x)$ after optimization.
Next, we show how the LTN language can be used to solve progressively more complex problems by combining learning and reasoning.

Symbolic Tensor Computational Graph for the Binary Classification Example. In the figure, $\G{x_+}$ and $\G{x_-}$ are inputs to the network $\G_\theta(A)$ and the dotted lines indicate the propagation of activation from each input through the network, which produces two outputs.

Binary Classification task (training and test set performance): Average accuracy (left) and satisfiability (right). Due to the random initializations, accuracy and satisfiability start on average at $0.5$ with performance increasing rapidly after a few epochs.

Binary Classification task (querying the trained predicate $A(x)$): It is interesting to see how $A(x)$ could be appropriately named as denoting the inside of the central region shown in the figure, and therefore $\neg A(x)$ represents the outside of the region.

§.§ Multi-Class Single-Label Classification

The natural extension of binary classification is a multi-class classification task.
We first approach multi-class single-label classification, which assumes
that each example is assigned to one and only one label.

For illustration purposes, we use the Iris flower data set [20], which consists of classification into three mutually exclusive classes; call these $A$, $B$, and $C$.
While one could train three unary predicates $A(x)$, $B(x)$ and $C(x)$, it turns out to be more effective if this problem is modeled by a single binary predicate $P(x,l)$, where $l$ is a variable denoting a multi-class label, in this case, classes $A$, $B$ or $C$.
This syntax allows one to write statements quantifying over the classes, e.g. $\forall x ( \exists l ( P(x,l)))$.
Since the classes are mutually exclusive, the output layer of the $\mathtt{MLP}$ representing $P(x,l)$ will be a $\mathtt{softmax}$ layer, instead of a $\mathtt{sigmoid}$ function, to ensure the exclusivity constraint on satisfiability scores.[$\texttt{softmax}(x)=e^{x_i}/\sum_j e^{x_j}$].
The problem can be specified as follows:

$\mathrm{items}$, denoting the examples from the Iris flower data set.

$\mathrm{labels}$, denoting the class labels.

$x_A$, $x_B$, $x_C$ for the positive examples of classes $A$, $B$, $C$.

$x$ for all examples.

$\domof(x_A) = \domof(x_B) = \domof(x_C) = \domof(x) = \mathrm{items}$.

$l_A$, $l_B$, $l_C$, the labels of classes $A$ (Iris setosa), $B$ (Iris virginica), $C$ (Iris versicolor),

$\domof(l_A) = \domof(l_B) = \domof(l_C) = \mathrm{labels}$.

$P(x,l)$ denoting the fact that item $x$ is classified as $l$.

$\domofin(P) = \mathrm{items},\mathrm{labels}$.

\begin{align}
\forall x_A &\ P(x_A,l_A)\\
\forall x_B &\ P(x_B,l_B)\\
\forall x_C &\ P(x_C,l_C)
\end{align}

Notice that rules about exclusiveness such as $\forall x (P(x,l_A) \rightarrow (\lnot P(x,l_B) \wedge \lnot P(x,l_C)))$ are not included since such constraints are already imposed by the grounding of $P$ below, more specifically the $\mathtt{softmax}$ function.

$\G(\mathrm{items}) = \R^4$, items are described by $4$ features: the length and the width of the sepals and petals, in centimeters.

$\G(\mathrm{labels}) = \mathbb{N}^3$, we use a one-hot encoding to represent classes.

$\G(x_A) \in \R^{m_1 \times 4}$, that is, $\G(x_A)$ is a sequence of $m_1$ examples of class $A$.

$\G(x_B) \in \R^{m_2 \times 4}$, $\G(x_B)$ is a sequence of $m_2$ examples of class $B$.

$\G(x_C) \in \R^{m_3 \times 4}$, $\G(x_C)$ is a sequence of $m_3$ examples of class $C$.

$\G(x) \in \R^{(m_1+m_2+m_3)\times 4}$, $\G(x)$ is a sequence of all the examples.

$\G(l_A) = [1,0,0]$, $\G(l_B) = [0,1,0]$, $\G(l_C) = [0,0,1]$.

$\G(P \mid \theta) : x,l \mapsto l^\top \cdot \mathtt{softmax}(\mathtt{MLP}_\theta(x))$, where the $\mathtt{MLP}$ has three output neurons corresponding to as many classes, and
$\cdot$ denotes the dot product as a way of selecting an output for $\G(P \mid \theta)$; multiplying the $\mathtt{MLP}$'s output by the one-hot vector $l^\top$ gives the truth degree corresponding to the class denoted by $l$.

The logical operators and connectives are approximated using the stable product configuration with $p=2$ for $\ApmeanError$.
For the formula aggregator,  $\ApmeanError$ is used also with $p=2$.

The computational graph of Figure <ref> illustrates how $\Sat_{\phi \in \K} \G_\theta(\phi)$ is obtained.
If $\bm U$ denotes batches sampled from the data set of all examples, the loss function (to minimize) is:

$$\bm L = 1-\Sat_{\phi \in \K }\G_{\theta,x\leftarrow \bm B}(\phi).$$

Figure <ref> shows the result of training with the Adam optimizer with batches of 64 examples.
Accuracy measures the ratio of examples correctly classified, with example $x$ labeled as $\argmax_{l}(P(x,l))$.[This is also known as top-1 accuracy, as proposed in [39]. Cross-entropy results $\sum(t \ log(y))$ could have been reported here as is common with the use of softmax, although it is worth noting that, of course, the loss function used by LTN is different.]
Classification accuracy reaches an average value near $1.0$ for both the training and test data after some 100 epochs. Satisfaction levels of the Iris flower predictions continue to increase for the rest of the training (500 epochs) to more than $0.8$.

It is worth contrasting the choice of using a binary predicate $(P(x,l))$ in this example with the option of using multiple unary predicates $(l_A(x),l_B(x),l_C(x))$, one for each class. Notice how each predicate is normally associated with an output neuron. In the case of the unary predicates, the networks would be disjoint (or modular), whereas weight-sharing takes place with the use of the binary predicate. Since $l$ is instantiated into $l_A,l_B,l_C$, in practice $P(x,l)$ becomes $P(x,l_A)$, $P(x,l_B)$, $P(x,l_C)$, which is implemented via three output neurons to which a softmax function applies.

Symbolic Tensor Computational Graph for the Multi-Class Single-Label Problem. As before, the dotted lines in the figure indicate the propagation of activation from each input through the network, in this case producing three outputs.

Multi-Class Single-Label Classification: Classification accuracy (left) and satisfaction level (right).

§.§ Multi-Class Multi-Label Classification

We now turn to multi-label classification, whereby multiple labels can be assigned to each example. As a first example of the reach of LTNs, we shall see how the previous example can be extended naturally using LTN to account for multiple labels, not always a trivial extension for most ML algorithms.
The standard approach to the multi-label problem is to provide explicit negative examples for each class. By contrast, LTN can use background knowledge to relate classes directly to each other, thus becoming a powerful tool in the case of the multi-label problem when typically the labeled data is scarce.
We explore the Leptograpsus crabs data set [10] consisting of 200 examples of 5 morphological measurements of 50 crabs. The task is to classify the crabs according to their color and sex. There are four labels: blue, orange, male, and female.
The color labels are mutually exclusive, and so are the labels for sex. LTN will be used to specify such information logically.

$\mathrm{items}$ denoting the examples from the crabs dataset.

$\mathrm{labels}$ denoting the class labels.

$x_{\mathrm{blue}}$, $x_{\mathrm{orange}}$,  $x_{\mathrm{male}}$, $x_{\mathrm{female}}$ for the positive examples of each class.

$x$, used to denote all the examples.

$\domof(x_{\mathrm{blue}}) = \domof(x_{\mathrm{orange}}) = \domof(x_{\mathrm{male}}) = \domof(x_{\mathrm{female}}) = \domof(x) = \mathrm{items}$.

$l_{\mathrm{blue}}$,  $l_{\mathrm{orange}}$, $l_{\mathrm{male}}$, $l_{\mathrm{female}}$ (the labels for each class).

$P(x,l)$, denotes the fact that item $x$ is labelled as $l$.

$\domofin(P) = \mathrm{items},\mathrm{labels}$.

\begin{align}
&\forall x_{\mathrm{blue}} \ P(x_{\mathrm{blue}},l_{\mathrm{blue}})\\
&\forall x_{\mathrm{orange}} \ P(x_{\mathrm{orange}},l_{\mathrm{orange}})\\
&\forall x_{\mathrm{male}} \ P(x_{\mathrm{male}},l_{\mathrm{male}})\\
&\forall x_{\mathrm{female}} \ P(x_{\mathrm{female}},l_{\mathrm{female}})\\
&\forall x \ \lnot(P(x,l_{\mathrm{blue}}) \land P(x,l_{\mathrm{orange}})) \label{ex:axiom:exclusivecolors}\\
&\forall x \ \lnot(P(x,l_{\mathrm{male}}) \land P(x,l_{\mathrm{female}})) \label{ex:axiom:exclusivegenders}
\end{align}

Notice how logical rules <ref> and <ref> above represent the mutual exclusion of the labels on colour and sex, respectively. As a result, negative examples are not used explicitly in this specification.

$\G(\mathrm{items})=\R^5$; the examples from the data set are described using 5 features.

$\G(\mathrm{labels})=\mathbb{N}^4$; one-hot vectors are used to represent class labels.[There are two possible approaches here: either each item is labeled with one multi-hot encoding or each item is labeled with several one-hot encodings. The latter approach was used in this example.]

$\G(x_\mathrm{blue}) \in \R^{m_1 \times 5}$, $\G(x_\mathrm{orange}) \in \R^{m_2 \times 5}$, $\G(x_\mathrm{male}) \in \R^{m_3 \times 5}$, $\G(x_\mathrm{female}) \in \R^{m_4 \times 5}$.
These sequences are not mutually-exclusive, one example can for instance be in both $x_\mathrm{blue}$ and $x_\mathrm{male}$.

$\G(l_\mathrm{blue}) = [1,0,0,0]$, $\G(l_\mathrm{orange}) = [0,1,0,0]$,$\G(l_\mathrm{male}) = [0,0,1,0]$,$\G(l_\mathrm{female}) = [0,0,0,1]$.

$\G(P \mid \theta) : x,l \mapsto l^\top \cdot \mathtt{sigmoid}(\mathtt{MLP}_\theta(x))$, with the $\mathtt{MLP}$ having four output neurons corresponding to as many classes. As before, $\cdot$ denotes the dot product which selects a single output.
By contrast with the previous example, notice the use of a $\mathtt{sigmoid}$ function instead of a $\mathtt{softmax}$ function.

As before, the fuzzy logic operators and connectives are approximated using the stable product configuration with $p=2$ for $\ApmeanError$, and for the formula aggregator, $\ApmeanError$ is also used with $p=2$.

Figure <ref> shows the result of the Adam optimizer using backpropagation trained with batches of 64 examples.
This time, the accuracy is defined as $1-\mathrm{HL}$, where $\mathrm{HL}$ is the average Hamming loss, i.e. the fraction of labels predicted incorrectly, with a classification threshold of $0.5$ (given an example $u$, if the model outputs a value greater than $0.5$ for class $C$ then $u$ is deemed as belonging to class $C$). The rightmost graph in Figure <ref> illustrates how LTN learns the constraint that a crab cannot have both blue and orange color, which is discussed in more detail in what follows.

To illustrate the learning of constraints by LTN, we have queried three formulas that were not explicitly part of the knowledge-base, over time during learning:

\begin{align}
\phi_1 &:& \forall x \ &( P(x, l_{\mathrm{blue}}) \imp \lnot P(x, l_{\mathrm{orange}})) \\
\phi_2 &:& \forall x \ &( P(x, l_{\mathrm{blue}}) \imp P(x, l_{\mathrm{orange}})) \\
\phi_3 &:& \forall x \ &( P(x, l_{\mathrm{blue}}) \imp P(x, l_{\mathrm{male}}))
\end{align}

For querying, we use $p=5$ when approximating the universal quantifiers with $\ApmeanError$.
A higher $p$ denotes a stricter universal quantification with a stronger focus on outliers (see Section <ref>).
Training should usually not focus on outliers, as optimizers would struggle to generalize and tend to get stuck in local minima.
However, when querying $\phi_1$,$\phi_2$,$\phi_3$, we wish to be more careful about the interpretation of our statement.
See also <ref>.
We should expect $\phi_1$ to hold true (every blue crab cannot be orange and vice-versa[Notice how, strictly speaking, other colours remain possible since the prior knowledge did not specify the bi-conditional:
$\forall x \ (P(x,l_{\mathrm{blue}}) \leftrightarrow \lnot P(x,l_{\mathrm{orange}}))$], and we should expect $\phi_2$ (every blue crab is also orange) and $\phi_3$ (every blue crab is male) to be false.
The results are reported in the rightmost plot of Figure <ref>.
Prior to training, the truth-values of $\phi_1$ to $\phi_3$ are non-informative. During training one can see, with the maximization of the satisfaction of the knowledge-base, a trend towards the satisfaction of $\phi_1$, and an opposite trend of $\phi_2$ and $\phi_3$ towards false.

Multi-Class Multi-Label Classification: Classification Accuracy (left), Satisfiability level (middle), and Querying of Constraints (right).

§.§ Semi-Supervised Pattern recognition

§.§ Regression

Another important problem in Machine Learning is regression where a relationship is estimated between one independent variable $X$ and a continuous dependent variable $Y$.
The essence of regression is, therefore, to approximate a function $f(x)=y$ by a function $f^*$, given examples $(x_i, y_i)$ such that $f(x_i) = y_i$. In LTN one can model a regression task by defining $f^*$ as a learnable function whose parameter values are constrained by data.
Additionally, a regression task requires a notion of equality. We, therefore, define the predicate $\mathrm{eq}$ as a smooth version of the symbol $=$ to turn the constraint $f(x_i)=y_i$ into a smooth optimization problem.

In this example, we explore regression using a problem from a real estate data set[https://www.kaggle.com/quantbruce/real-estate-price-prediction] with
414 examples, each described in terms of 6 real-numbered features:
the transaction date (converted to a float),
the age of the house, the distance to the nearest station, the number of convenience stores in the vicinity, and the latitude and longitude coordinates. The model has to predict the house price per unit area.

$\mathrm{samples}$, denoting the houses and their features.

$\mathrm{prices}$, denoting the house prices.

$x$ for the samples.

$y$ for the prices.

$f^*(x)$, the regression function to be learned.

$\domofin(f^*)=\mathrm{samples}$, $\domofout(f^*)=\mathrm{prices}$.

$\mathrm{eq}(y_1,y_2)$, a smooth equality predicate that measures how similar $y_1$ and $y_2$ are.

$\domofin(\mathrm{eq}) = \mathrm{prices},\mathrm{prices}$.

\begin{align}
\forall \mathrm{Diag}(x,y) \ \mathrm{eq}(f^*(x),y)
\end{align}

Notice again the use of $\mathrm{Diag}$:
when grounding $x$ and $y$ onto sequences of values, this is done by obeying a one-to-one correspondence between the sequences. In other words, we aggregate pairs of corresponding $\mathrm{samples}$ and $\mathrm{prices}$, instead of any combination thereof.

$\G(x) \in \R^m \times 6$, $\G(y) \in \R^m \times 1$.
Notice that this specification refers to the same number $m$ of examples for $x$ and $y$ due to the above one-to-one correspondence obtained with the use of  $\mathrm{Diag}$.

$\G(\mathrm{eq}(\mathbf{u},\mathbf{v})) = \exp\big(-\alpha\sqrt{\sum_j(u_j-v_j)^2}\big)$,
where the hyper-parameter $\alpha$ is a real number that scales how strict the smooth equality is.
Intuitively, the smooth equality is $\exp(-\alpha \  d(\mathbf{u},\mathbf{v}))$, where $d(\mathbf{u},\mathbf{v})$ is the Euclidean distance between $\mathbf{u}$ and $\mathbf{v}$.
It produces a $1$ if the distance is zero;
as the distance increases, the result decreases exponentially towards $0$.
In case an exponential decrease is undesirable, one can adopt the following alternative equation:  $\mathrm{eq}(\mathbf{u},\mathbf{v})=\frac{1}{1+\alpha d(\mathbf{u},\mathbf{v})}$.]
In our experiments, we use $\alpha=0.05$.

$\G(f^*(x) \mid \theta) = \mathtt{MLP}_\theta(x)$, where $\mathtt{MLP}_\theta$ is a multilayer perceptron which ends in one neuron corresponding to a price prediction, with a linear output layer (no activation function).

The theory is constrained by the parameters of the model of $f^*$. LTN is used to estimate such parameters by maximizing the satisfaction of the knowledge-base, in the usual way.
Approximating $\forall$ using $\ApmeanError$ with $p=2$, as before, we randomly split the data set into 330 examples for training and 84 examples for testing.
Figure <ref> shows the satisfaction level over 500 epochs.
We also plot the Root Mean Squared Error (RMSE) between the predicted prices and the labels (i.e. actual prices, also known as target values).
We visualize in Figure <ref> the strong correlation between actual and predicted prices at the end of one of the runs.

Regression task: RMSE and satisfaction level over time.

Visualization of LTN solving a regression problem.

§.§ Unsupervised Learning (Clustering)

In unsupervised learning, labels are either not available or are not used for learning. Clustering is a form of unsupervised learning whereby, without labels, the data is characterized by constraints alone.
LTN can formulate such constraints, such as:

* clusters should be disjoint,

* every example should be assigned to a cluster,

* a cluster should not be empty,

* if the points are near, they should belong to the same cluster,

* if the points are far, they should belong to different clusters, etc.

$\mathrm{points}$, denoting the data to cluster.

$\mathrm{points\_pairs}$, denoting pairs of examples.

$\mathrm{clusters}$, denoting the cluster.

$x$, $y$ for all points.

$\domof(x) = \domof(y) = \mathrm{points}$.

$\domof(c) = \mathrm{clusters}$.

$C(x,c)$, the truth degree of a given point belonging in a given cluster.

\begin{align}
\forall x &\ \exists c \ C(x,c) \\
\forall c &\ \exists x \ C(x,c) \\
\label{eq:clustering_close} \forall (c, x, y : \lvert x - y \rvert < \mathrm{th_{close}} ) & \ \ (C(x,c) \leftrightarrow C(y,c)) \\
\label{eq:clustering_dist} \forall (c, x, y : \lvert x - y \rvert > \mathrm{th_{distant}}) & \ \ \lnot(C(x,c) \land C(y,c))
\end{align}

Notice the use of guarded quantifiers: all the pairs of points with Euclidean distance lower (resp. higher) than a value $\mathrm{th_{close}}$ (resp. $\mathrm{th_{distant}}$) should belong in the same cluster (resp. should not).
$\mathrm{th_{close}}$ and $\mathrm{th_{distant}}$ are arbitrary threshold values that define some of the closest and most distant pairs of points. In our example, they are set to, respectively, $0.2$ and $1.0$.

As done in the example of Section <ref>, the clustering predicate has mutually exclusive satisfiability scores for each cluster using a $\mathtt{softmax}$ layer.
Therefore, there is no explicit constraint about clusters being disjoint.

$\G(\mathrm{points}) = [-1,1]^2$.

$\G(\mathrm{clusters}) = \mathbb{N}^4$, we use one-hot vectors to represent a choice of 4 clusters.

$\G(x) \in [-1,1]^{m \times 2}$, that is, $x$ is a sequence of $m$ points. $\G(y) = \G(x)$.

$\mathrm{th_{close}} = 0.2$, $\mathrm{th_{distant}} = 1.0$.

$\G(c) = \left< [1,0,0,0],[0,1,0,0],[0,0,1,0],[0,0,0,1]\right>$.

$\G(C \mid \theta) : x,c \mapsto c^\top \cdot \mathtt{softmax}(\mathtt{MLP}_\theta(x))$, where $\mathtt{MLP}$ has 4 output neurons corresponding to the 4 clusters.

We use the stable real product configuration to approximate the logical operators.
For $\forall$, we use $\ApmeanError$ with $p=4$.
For $\exists$, we use $\Apmean$ with $p=1$ during the first 100 epochs, and $p=6$ thereafter, as a simplified version of the schedule used in Section <ref>.
The formula aggregator is approximated by $\ApmeanError$ with $p=2$.
The model is trained for a total of 1000 epochs using the Adam optimizer, which is sufficient for LTN to solve the clustering problem shown in Figure <ref>. Ground-truth data for this task was generated artificially by creating 4 centers, and generating $50$ random samples from a multivariate Gaussian distribution around each center.
The trained LTN achieves a satisfaction level of the clustering constraints of $0.857$.

LTN solving a clustering problem by constraint optimization: ground-truth (top) and querying of each cluster $C0$, $C1$, $C2$ and $C3$, in turn.

§.§ Learning Embeddings with LTN

A classic example of Statistical Relational Learning is the  smokers-friends-cancer example introduced in [55]. Below, we show how this example can be formalized in LTN using semi-supervised embedding learning.

There are 14 people divided into two groups $\{a,b,\dots,h\}$ and $\{i,j,\dots,n\}$.
Within each group, there is complete knowledge about smoking habits.
In the first group, there is complete knowledge about who has and who does not have cancer.
Knowledge about the friendship relation is complete within each group only if symmetry is assumed, that is, $\forall x,y \ (friends(x,y) \rightarrow friends(y,x))$.
Otherwise, knowledge about friendship is incomplete in that it may be known that e.g. $a$ is a friend of $b$, and it may be not known whether $b$ is a friend of $a$.
Finally, there is general knowledge about smoking, friendship, and cancer, namely that smoking causes cancer, friendship is normally symmetric and anti-reflexive, everyone has a friend, and smoking propagates (actively or passively) among friends.
All this knowledge is represented in the axioms further below.

$\mathrm{people}$, to denote the individuals.

$a,b,\dots,h,i,j,\dots,n$, the 14 individuals. Our goal is to learn an adequate embedding for each constant.

$x$,$y$ ranging over the individuals.

$S(x)$ for smokes, $F(x,y)$ for friends, $C(x)$ for cancer.

$\domof(S)=\domof(C)=\mathrm{people}$. $\domof(F)=\mathrm{people},\mathrm{people}$.

Let $\mathcal{X}_1 = \{a,b,\dots,h\}$ and $\mathcal{X}_2 = \{i,j,\dots,n\}$ be the two groups of individuals.

Let $\mathcal{S} = \{a,e,f,g,j,n\}$ be the smokers; knowledge is complete in both groups.

Let $\mathcal{C} = \{a,e\}$ be the individuals with cancer; knowledge is complete in $\mathcal{X}_1$ only.

Let $\mathcal{F} = \{(a,b),(a,e),(a,f),(a,g),(b,c),(c,d),(e,f),(g,h),(i,j),(j,m),(k,l),(m,n)\}$ be the set of friendship relations; knowledge is complete if assuming symmetry.

These facts are illustrated in Figure <ref>.

We have the following axioms:

\begin{align}
\qquad \qquad & F(u,v) && \text{for } (u,v) \in \mathcal{F} \\
& \lnot F(u,v) && \text{for } (u,v) \notin \mathcal{F},\ u > v \\
& S(u) && \text{for } u \in \mathcal{S} \\
& \lnot S(u) && \text{for } u \in (\mathcal{X}_1 \cup \mathcal{X}_2 ) \setminus \mathcal{S} \\
& C(u) && \text{for } u \in \mathcal{C} \\
& \lnot C(u) && \text{for } u \in \mathcal{X}_1 \setminus \mathcal{C}\\
\label{eq:smokes_axiom_F_antireflexive}
\forall x \ & \lnot F(x,x) && \\
\label{eq:smokes_axiom_F_symmetric}
\forall x,y \ & (F(x,y) \imp F(y,x)) && \\
\forall x \exists y \ & F(x,y) && \\
\forall x,y \ & ((F(x,y) \land S(x)) \imp S(y)) && \label{eq:smokes_axiom_propagate} \\
\forall x \ & (S(x) \imp C(x)) && \label{eq:smokes_axiom_causecancer}\\
\forall x \ & (\lnot C(x) \imp \lnot S(x)) &&
\end{align}

Notice that the knowledge base is not satisfiable in the strict logical sense of the word.
For instance, $f$ is said to smoke but not to have cancer, which is inconsistent with the rule $\forall x \ (S(x) \imp C(x))$.
Hence, it is important to adopt a fuzzy approach as done with MLN or a many-valued fuzzy logic interpretation as done with LTN.

$\G(\mathrm{people}) = \R^5$. The model is expected to learn embeddings in $\R^5$.

$\G(a \mid \theta) = \mathbf{v}_\theta(a)$, …, $\G(n \mid \theta) = \mathbf{v}_\theta(n)$.
Every individual is associated with a vector of 5 real numbers.
The embedding is initialized randomly uniformly.

$\G(x \mid \theta) = \G(y \mid \theta) = \left< \mathbf{v}_\theta(a), \dots, \mathbf{v}_\theta(n) \right>$.

$\G(S \mid \theta) : x \mapsto \mathtt{sigmoid}(\mathtt{MLP\_S}_\theta(x))$, where $\mathtt{MLP\_S}_\theta$ has 1 output neuron.

$\G(F \mid \theta) : x,y \mapsto \mathtt{sigmoid}(\mathtt{MLP\_F}_\theta(x,y))$, where $\mathtt{MLP\_F}_\theta$ has 1 output neuron.

$\G(C \mid \theta) : x \mapsto \mathtt{sigmoid}(\mathtt{MLP\_C}_\theta(x))$, where $\mathtt{MLP\_C}_\theta$ has 1 output neuron.

The $\mathtt{MLP}$ models for $S$, $F$, $C$ are kept simple, so that most of the learning is focused on the embedding.

We use the stable real product configuration to approximate the operators.
For $\forall$, we use $\ApmeanError$ with $p=2$ for all the rules, except for rules (<ref>) and (<ref>), where we use $p=6$.
The intuition behind this choice of $p$ is that no outliers are to be accepted for the friendship relation since it is expected to be symmetric and anti-reflexive, but outliers are accepted for the other rules.
For $\exists$, we use $\Apmean$ with $p=1$ during the first 200 epochs of training, and $p=6$ thereafter, with the same motivation as that of the schedule used in Section <ref>.
The formula aggregator is approximated by $\ApmeanError$ with $p=2$.

Figure <ref> shows the satisfiability over 1000 epochs of training.
At the end of one of these runs, we query $S(x)$, $F(x,y)$, $C(x)$ for each individual; the results are shown in Figure <ref>.
We also plot the principal components of the learned embeddings [51] in Figure <ref>.
The friendship relations are learned as expected: (<ref>) "smoking implies cancer" is inferred for group 2 even though such information was not present in the knowledge base.
For group 1, the given facts for smoking and cancer for the individuals $f$ and $g$ are slightly altered, as these were inconsistent with the rules.
(the rule for smoking propagating via friendship (<ref>) is incompatible with many of the given facts).
Increasing the satisfaction of this rule would require decreasing the overall satisfaction of the knowledge base, which explains why it is partly ignored by LTN during training.
Finally, it is interesting to note that the principal components for the learned embeddings seem to be linearly separable for the smoking and cancer classifiers (c.f. Figure <ref>, top right and bottom right plots).

To illustrate querying in LTN,
we query over time two formulas that are not present in the knowledge-base:

\begin{align}
\phi_1 &:& \forall p : & C(p) \imp S(p) \\
\phi_2 &:& \forall p,q : & (C(p) \lor C(q)) \imp F(p,q)
\end{align}

We use $p=5$ when approximating $\forall$ since the impact of an outlier at querying time should be seen as more important than at learning time.
It can be seen that as the grounding approaches satisfiability of the knowledge-base, $\phi_1$ approaches true, whereas $\phi_2$ approaches false (c.f. Figure <ref>).

Smoker-Friends-Cancer example: Satisfiability levels during training (left) and truth-values of queries $\phi_1$ and $\phi_2$ over time (right).

Incomplete facts in the knowledge-base: axioms for smokers and cancer for individuals $a$ to $n$ (left), friendship relations in group 1 (middle), and friendship relations in group 2 (right).

Querying all the truth-values using LTN after training: smokers and cancer (left), friendship relations (middle and right).

Smoker-Friends-Cancer example: Illustration of the facts before and after training.

Smoker-Friends-Cancer example: learned embeddings showing the result of applying PCA on the individuals (top left); truth-values of smokes and cancer predicates for each embedding (top and bottom right); illustration of the friendship relations which are satisfied after learning (bottom left).

§.§ Reasoning in LTN

The essence of reasoning is to find out if a closed formula $\phi$ is the logical consequence of a knowledge-base $(\K,\G_\theta,\Theta)$.
Section <ref> introduced two approaches to this problem in LTN:

* By simply querying after learning[Here, learning refers to Section <ref>, which is optimizing using the satisfaction of the knowledge base as an objective.]
one seeks to verify if for the grounded theories that maximally satisfy $\K$,
the grounding of $\phi$ gives a truth-value greater than a threshold $q$.
This often requires checking an infinite number of groundings.
Instead, the user approximates the search for these grounded theories by running the optimization a fixed number of times only.

* Reasoning by refutation one seeks to find out a counter-example: a grounding that satisfies the knowledge-base $\K$ but not the formula $\phi$ given the threshold $q$.
A search is performed here using a different objective function.

We now demonstrate that reasoning by refutation is the preferred option using a simple example where we seek to find out whether $(A \lor B) \models_q A$.

Propositional Variables:

The symbols $A$ and $B$ denote two propositionial variables.

\begin{equation}
A \lor B
\end{equation}

$\G(A)=a$, $\G(B)=b$, where $a$ and $b$ are two real-valued parameters.
The set of parameters is therefore $\theta=\{a,b\}$.
At initialization, $a=b=0$.

We use the probabilistic-sum $\Sprod$ to approximate $\lor$, resulting in the following satisfiability measure:[
We use the notation $\G(\K) := \Sat_{\phi \in \K}(\K,\G)$.

\begin{equation}
\G_\theta(\K) = \G_\theta(A \lor B) = a + b - ab.
\end{equation}

There are infinite global optima maximizing the satisfiability of the theory,
as any $\G_\theta$ such that $\G_\theta(A)=1$ (resp. $\G_\theta(B)=1$) gives a satisfiability $\G_\theta(\K)=1$ for any value of $\G_\theta(B)$ (resp. $\G_\theta(A)$).
As expected, the following groundings are examples of global optima:

*  $\G_1(A)=1$, $\G_1(B)=1$, $\G_1(\K)=1$,

* $\G_2(A)=1$, $\G_2(B)=0$, $\G_2(\K)=1$,

* $\G_3(A)=0$, $\G_3(B)=1$, $\G_3(\K)=1$.

$(A \lor B) \models_q A$?
That is, given the threshold $q=0.95$, does every $\G_\theta$ such that $\G_\theta(\K) \geq q$ verify $\G_\theta(\phi) \geq q$.
Immediately, one can notice that this is not the case.
For instance, the grounding $\G_3$ is a counter-example.

If one simply reasons by querying multiple groundings after learning with the usual objective $\argmax_{(\G_\theta)}\G_\theta(\K)$,
the results will all converge to $\G_1$:
$\frac{\partial \G_\theta(\K)}{\partial a} = 1 - b$ and $\frac{\partial \G_\theta(\K)}{\partial b} = 1 - a$.
Every run of the optimizer will increase $a$ and $b$ simultaneously until they reach the optimum $a=b=1$.
Because the grid search always converges to the same point, no counter-example is found and the logical consequence is mistakenly assumed true.
This is illustrated in Figure <ref>.

Reasoning by refutation, however, the objective function has an incentive to find a counter-example with $\neg A$, as illustrated in Figure <ref>.
LTN converges to the optimum $\G_3$, which refutes the logical consequence.

Querying after learning: 10 runs of the optimizer with objective $\G^* = \argmax_{\G_\theta} (\G_\theta(\K))$. All runs converge to the optimum $\G_1$; the grid search misses the counter-example.

Reasoning by refutation: one run of the optimizer with objective $\G^* = \argmin_{\G_\theta} (\G_\theta(\phi) + \mathtt{elu}(\alpha,\beta (q-\G_\theta(\K)))$,
$q=0.95$, $\alpha=0.05$, $\beta=10$.
In the first training epochs, the directed search prioritizes the satisfaction of the knowledge base.
Then, the minimization of $\G_\theta(\phi)$ starts to weigh in more and the search focuses on finding a counter-example.
Eventually, the run converges to the optimum $\G_3$, which refutes the logical consequence.

§ RELATED WORK

The past years have seen considerable work aiming to integrate symbolic systems and neural networks.
We shall focus on work whose objective is to build
computational models that integrate deep learning and logical reasoning into a so-called end-to-end (fully differentiable) architecture. We summarize a categorization in Figure <ref> where
the class containing LTN is further expanded into three
sub-classes. The sub-class highlighted in red is the one that contains LTN. The reason why one may wish to combine symbolic AI and neural networks into a neurosymbolic AI system may vary, c.f. [17] for a recent comprehensive overview of approaches and challenges for neurosymbolic AI.

§.§ Neural architectures for logical reasoning

These use
neural networks to perform (probabilistic) inference on logical
theories. Early work in this direction has shown correspondences
between various logical-symbolic systems and neural network models
[27, 32, 52, 63, 65].
They have also highlighted the limits of current neural networks as
models for knowledge representation. In a nutshell, current neural
networks (including deep learning) have been shown capable of
representing propositional logic, nonmonotonic logic programming,
propositional modal logic, and fragments of first-order logic, but
not full first-order or higher-order logic. Recently, there has been
a resurgence of interest in the topic with many proposals emerging
[13, 48, 53].
In [13], each clause of a Stochastic Logic Program is converted into a factor graph with reasoning becoming
differentiable so that it can be implemented by deep networks. In
[49], a differentiable unification algorithm
is introduced with theorem proving sought to be carried out inside
the neural network. Furthermore, in
[11, 49]
neural networks are used to learn reasoning strategies and logical
rule induction.

Reasoning with LTN (Section <ref>) is reminiscent of this category,
given that knowledge is not represented in a traditional logical language but in Real Logic.

§.§ Logical specification of neural network architectures

Here the goal is to use a logical language to specify the architecture of a neural network. Examples include
[13, 24, 26, 56, 66].
In [26], the languages of extended logic programming
(logic programs with negation by failure) and answer set programming
are used as background knowledge to set up the initial architecture
and set of weights of a recurrent neural network, which is
subsequently trained from data using backpropagation. In [24], first-order logic programs in the
form of Horn clauses are used to define a neural network that can
solve Inductive Logic Programming tasks, starting from the most
specific hypotheses covering the set of examples.
Lifted relational neural
networks [66] is a declarative framework where a
Datalog program is used as a compact specification of a diverse range of existing advanced neural
architectures, with a particular focus on Graph Neural Networks
(GNNs) and their generalizations.
In [56] a
weighted Real Logic is introduced and used to specify neurons in a
highly modular neural network that resembles a tree structure,
whereby neurons with different activation functions are used to
implement the different logic operators.

To some extent, it is also possible to specify neural architectures using logic in LTN.
For example, a user can define a classifier
$P(x,y)$ as the formula $P(x,y) = (Q(x,y) \land R(y)) \lor S(x,y)$.
$\G(P)$ becomes a computational graph that combines the sub-architectures $\G(Q)$, $\G(R)$, and $\G(S)$ according to the syntax of the logical formula.

§.§ Neurosymbolic architectures for the integration of inductive
learning and deductive reasoning

These architectures seek to
enable the integration of inductive and deductive reasoning in a
unique fully differentiable framework
[15, 23, 41, 46, 47].
The systems that belong to this class combine a neural component
with a logical component. The former consists of one or more neural
networks, the latter provides a set of algorithms for performing
logical tasks such as model checking, satisfiability, and logical
consequence. These two components are tightly integrated so that
learning and inference in the neural component are influenced by
reasoning in the logical component and vice versa. Logic Tensor
Networks belong to this category. Neurosymbolic
architectures for integrating learning and reasoning can be further separated into three

* Approaches that introduce additional layers to the
neural network to encode logical constraints which modify the
predictions of the network. This sub-class includes Deep
Logic Models [46] and Knowledge Enhanced Neural
Networks [15].

* Approaches that integrate
logical knowledge as additional constraints in the objective function or loss function used to train the neural network (LTN and
[23, 33, 47]).

* Approaches that apply
(differentiable) logical inference to compute the consequences
of the predictions made by a set of base neural networks. Examples of
this sub-class are DeepProblog [41] and Abductive Learning

[sibling distance=10em,
every node/.style = shape=rectangle, rounded corners,
draw, align=center,text width=10em,
top color=white, bottom color=blue!20,level distance=5em,sibling distance=12em]
Neuro-symbolic architectures
child  node Neural architectures for logical reasoning
[13, 27, 31, 48, 53, 57, 71]

child node Architectures integrating learning and reasoning
child[level distance=6em]   node Modify the predictions of a neural network using logical
knowledge [15, 46]

child[level distance=6em]  node[bottom color=red!20] Encode the satisfiability of logical knowledge
in the loss function: LTN and
[23, 33, 47, 72]

child[level distance=6em]   node Use of logical knowledge in addition to neural predictions [14, 41]
child  node Logical specification of neural networks: LTN and
[24, 26, 56, 66]

Three classes of neurosymbolic approaches with Architectures Integrating Learning and Reasoning further sub-divided into three sub-classes, with LTN belonging to the sub-class highlighted in red.

In what follows, we revise recent neurosymbolic
architectures in the same class as LTN: Integrating learning and reasoning.

Systems that modify the predictions of a base neural
network: Among the approaches that modify the predictions of the neural network
using logical constraints are Deep Logic Models
[46] and Knowledge Enhanced Neural Networks
[15]. Deep Logic Models (DLM) are a general
architecture for learning with constraints. Here, we will consider the
special case where constraints are expressed by logical formulas. In
this case, a DLM predicts the truth-values of a set of $n$
ground atoms of a domain $\Delta=\{a_1,\dots,a_k\}$. It consists of two models: a neural network $f(\bm x\mid\bm w)$ which takes
as input the features $\bm x$ of the elements of $\Delta$ and produces
as output an evaluation $\bm f$ for all the ground atoms, i.e.
$\bm f\in[0,1]^n$, and a probability distribution
$p(\bm y\mid \bm f,\bm \lambda)$ which is modeled by an undirected graphical model of the exponential family with each logical constraint characterized by a clique that contains the ground
atoms, rather similarly to GNNs. The model returns the assignment to the atoms that maximize the weighted truth-value of the constraints and minimize the
difference between the prediction of the neural network and a target value $\bm y$. Formally:

\text{DLM}(\bm x\mid\bm\lambda,\bm w) =
\argmax_{\bm y}\left(\sum_c\lambda_c\Phi_c(\bm y_c)-\frac12||\bm y -
\bm f(\bm x\mid\bm w)||^2\right)

Each $\Phi_c(\bm y_c)$ corresponds to a ground propositional formula which is
evaluated w.r.t. the target truth assignment $\bm y$, and $\lambda_c$ is the
weight associated with formula $\Phi_c$. Intuitively, the upper
model (the undirected graphical model) should modify the
prediction of the lower model (the neural network) minimally
to satisfy the constraints. $\bm f$ and $\bm y$
are truth-values of all the ground atoms obtained from the constraints
appearing in the upper model in the domain specified by the data

Similar to LTN, DLM evaluates constraints using fuzzy semantics. However, it considers only propositional connectives, whereas universal and existential quantifiers are supported in LTN.

Inference in DLM requires maximizing the prediction of the
model, which might be prohibitive in the presence of a large number of instances. In LTN, inference involves only a forward pass through the
neural component which is rather simple and can be carried out in parallel. However, in DLM
the weight associated with constraints can be
learned, while in LTN they are specified in the background

The approach taken in Knowledge Enhanced Neural Networks (KENN)
[15] is similar to that of DLM. Starting from
the predictions $\bm y = f_{nn}(\bm x\mid\bm w)$ made by a base neural
network $f_{nn}(\cdot\mid\bm w)$, KENN adds a knowledge enhancer, which is a function that modifies $\bm y$
based on a set of weighted constraints formulated in terms of
clauses. The formal model can be specified as follows:

\text{KENN}(\bm x\mid \bm\lambda,\bm w) =
\sigma(f'_{nn}(\bm x\mid \bm w) +
\sum_c\lambda_c\cdot
(\mathrm{softmax}(\mathrm{sign}(c) \odot f'_{nn}(\bm x\mid \bm w))
\odot \mathrm{sign}(c)))

where $f'_{nn}(\bm x\mid \bm w)$ are the pre-activations of
$f_{nn}(\bm x\mid\bm w)$, $\mathrm{sign}(c)$ is a vector
of the same dimension of $\bm y$ containing $1,-1$ and
$-\infty$, such that $\mathrm{sign}(c)_i=1$
(resp. $\mathrm{sign}(c)_i=-1$) if the $i$-th atom occurs positively (resp. negatively) in $c$, or $-\infty$ otherwise, and $\odot$ is the
element-wise product. KENN learns the weights $\bm \lambda$ of the
clauses in the background
knowledge and the base network parameters $\bm w$ by minimizing some
standard loss, (e.g. cross-entropy) on a set of training data.
If the training data is
inconsistent with the constraint, the weight of the constraint will be
close to zero. This intuitively implies that the latent knowledge
present in the data is preferred to the knowledge specified in the
constraints. In LTN, instead, training data and logical constraints are
represented uniformly with a formula, and we require that they are
both satisfied. A second difference between KENN and LTN is the
language: while LTN supports constraints written in full first-order
logic, constraints in KENN are limited to universally quantified

Systems that add knowledge to a neural network by adding a term to the loss function: In [33], a framework is proposed that learns simultaneously from labeled data and logical rules. The proposed architecture is made of a student
network $f_{nn}$ and a teacher network, denoted by $q$. The student network is trained to
do the actual predictions, while the teacher network encodes the
information of the logical rules. The transfer of information from the
teacher to the student network is done by defining a joint loss
$\mathscr{L}$ for both networks as a convex combination of the loss of the student and the teacher.
If $\tilde{\bm y}=f_{nn}(\bm x\mid\bm w)$ is the prediction of the
student network for input $\bm x$, the loss is defined as:

(1-\pi)\cdot\mathscr{L}(\bm y,\tilde{\bm y}) +
\pi\cdot\mathscr{L}(q(\tilde{\bm y}\mid\bm x),\tilde{\bm y})

where $q(\tilde{\bm y}\mid \bm x) =\exp\left(-\sum_c\lambda_c(1-\phi_c(\bm
x,\tilde{\bm y}))\right)$ measures how much the predictions
$\tilde{\bm y}$ satisfy the constraints encoded in the set of
clauses $\{\lambda_c:\phi_c\}_{c\in C}$. Training is
iterative. At every iteration, the parameters of the student network
are optimized to minimize the loss that takes into account
the feedback of the teacher network on the predictions from the
previous step. The main difference between this approach and LTN
is how the constraints are encoded in the loss. LTN integrates the
constraints in the network and optimizes directly their
satisfiability with no need for additional training data. Furthermore,
the constraints proposed in [33] are universally
quantified formulas only.

The approach adopted by Lyrics [47] is
analogous to the first version of LTN [61]. Logical constraints are translated into a loss function that measures
the (negative) satisfiability level of the network. Differently from
LTN, formulas in Lyrics can be associated with weights that are
hyper-parameters. In [47], a logarithmic loss function is also used when the product t-norm is adopted. Notice that weights
can also be added (indirectly) to LTN by introducing a $0$-ary
predicate $p_w$ to represent a constraint of the form $p_w\wedge\phi$. An
advantage of this approach would be that the weights could be learned.

In [72], a neural network computes the probability of some events being true.
The neural network should satisfy a set of propositional logic constraints on its output.
These constraints are compiled into arithmetic circuits for weighted model counting, which are then used to compute a loss function.
The loss function then captures how close the neural network is to satisfying the propositional logic constraints.

Systems that apply logical reasoning on the predictions of
a base neural network:
The most notable architecture in this category is
DeepProblog [41]. DeepProblog extends the ProbLog framework for
probabilistic logic programming to allow the computation of
probabilistic evidence from neural networks. A ProbLog program is a
logic program where facts and rules can be associated with probability values. Such values can be learned. Inference in ProbLog to answer a query $q$ is performed by knowledge
compilation into a function $p(q \mid \bm\lambda)$
that computes the probability that $q$ is
true according to the logic program with relative frequencies $\bm \lambda$. In DeepProbLog, a neural network $f_{nn}$ that outputs a probability distribution
$\bm t=(t_1,\dots,t_n)$ over a set of atoms $\bm a=(a_1,\dots,a_n)$ is
integrated into ProbLog by extending the logic program with $\bm a$ and the respective probabilities $\bm t$. The probability of a query $q$ is then given by $p'(q\mid\bm \lambda,f_{nn}(x\mid \bm w))$, where $x$
is the input of $f_{nn}$ and $p'$ is the function corresponding to the logic program extended with
$\bm a$. Given a set of
queries $\bm q$, input vectors $\bm x$ and
ground-truths $\bm y$ for all the queries,
training is performed by minimizing a loss function that measures
the distance between the probabilities predicted by the logic
program and the ground-truths, as follows:

\mathscr{L}(\bm y, p'(\bm q\mid\bm
\lambda,f_{nn}(\bm x\mid \bm w)))

The most important difference between DeepProbLog and LTN concerns the
logic on which they are based. DeepProbLog adopts probabilistic
logic programming. The output of the base neural network is interpreted as the
probability of certain atoms being true. LTN instead is based on many-valued logic.
The predictions of the base neural network are interpreted as fuzzy truth-values
(though previous work [67] also formalizes Real Logic as handling probabilities with relaxed constraints).
This difference of logic leads to the second main difference between LTN and DeepProblog: their inference mechanism.  DeepProblog performs probabilistic inference
(based on model counting) while LTN inference consists of computing
the truth-value of a formula starting from the truth-values of its
atomic components. The two types of inference are incomparable.
However, computing the fuzzy truth-value of a formula is more efficient
than model counting, resulting in a more scalable
inference task that allows LTN to use full first-order logic with function symbols. In DeepProblog, to perform probabilistic inference, a closed-world
assumption is made and a function-free language is used. Typically, DeepProbLog clauses are compiled into
Sentential Decision Diagrams (SDDs) to accelerate inference considerably[36], although the compilation step of clauses into the SDD circuit is still costly.

An approach that extends the predictions of a base neural network using
abductive reasoning is [14]. Given a neural network
$f_{nn}(\bm x\mid\bm w)$ that produces a crisp output
$\bm y\in\{0,1\}^n$ for $n$ predicates $p_1,\dots,p_n$ and background
knowledge in the form of a logic program $p$, parameters $\bm w$ of
$f_{nn}$ are learned alongside a set of additional rules $\Delta_C$
that define a new concept $C$ w.r.t. $p_1,\dots,p_n$ such that, for
every object $o$ with features $\bm x_o$:

\begin{align}
\label{eq:abdutive-neuro-symb}
\begin{array}{l@{\ \ \ \ }l}
p \cup f_{nn}(\bm x_o\mid\bm w)\cup\Delta_C \models C(o)
& \mbox{if $o$ is an instance of $C$} \\
p \cup f_{nn}(\bm x_o\mid\bm w)\cup\Delta_C \models \neg C(o)
& \mbox{if $o$ is not an instance of $C$}
\end{array}
\end{align}

The task is solved by iterating the following three steps:

* Given the predictions of the neural network $\{f_{nn}(\bm x_o\mid\bm
w)\}_{o\in O}$ on the set $O$ of training objects, search for the best $\Delta_C$ that
maximize the number of objects for which
(<ref>) holds;

* For each object $o$, compute by abduction on $p\cup\Delta_C$,
the explanation $\bm p'(o)$;

* Retrain $f_{nn}$ with the training set $\{\bm x_o,\bm
p'(o)\}_{o\in O}$.

Differently from LTN, in [14] the optimization is done
separately in an iterative way. The
semantics of the logic is crisp, neither fuzzy nor probabilistic, and therefore not fully differentiable. Abductive reasoning is adopted, which is a potentially relevant addition for comparison with symbolic ML and Inductive Logic Programming approaches [50].

Various other loosely-coupled approaches have been proposed recently such as [44], where image classification is carried out by a neural network in combination with reasoning from text data for concept learning at a higher level of abstraction than what is normally possible with pixel data alone. The proliferation of such approaches has prompted Henry Kautz to propose a taxonomy for neurosymbolic AI in  [34] (also discussed in [17]), including recent work combining neural networks with graphical models and graph neural networks [4, 40, 58], statistical relational learning [21, 55], and even verification of neural multi-agent systems [2, 8].

§ CONCLUSIONS AND FUTURE WORK

In this paper, we have specified the theory and exemplified the reach of Logic Tensor Networks as a model and system for neurosymbolic AI. LTN is capable of combining approximate reasoning and deep learning, knowledge and data.

For ML practitioners, learning in LTN (see Section <ref>)
can be understood as optimizing under first-order logic constraints relaxed into a loss function.
For logic practitioners, learning is similar to inductive inference:
given a theory, learning makes generalizations from specific observations obtained from data.
Compared to other neuro-symbolic architectures (see Section <ref>),
the LTN framework has useful properties for gradient-based optimization (see Section <ref>)
and a syntax that supports many traditional ML tasks and their inductive biases (see Section <ref>),
all while remaining computationally efficient (see Table <ref>).

Section <ref> discussed reasoning in LTN. Reasoning is normally under-specified within neural networks. Logical reasoning is the task of proving if some knowledge follows from the facts which are currently known.
It is traditionally achieved semantically using model theory or syntactically via a proof system.
The current LTN framework approaches reasoning semantically, although it should be possible to use LTN and querying alongside a proof system.
When reasoning by refutation in LTN, to find out if a statement $\phi$ is a logical consequence of given data and knowledge-base $\K$, a proof by refutation attempts to find a semantic counterexample where $\lnot \phi$ and $\K$ are satisfied. If the search fails then $\phi$ is assumed to hold. This approach is efficient in LTN when we allow for a direct search to find counterexamples via gradient-descent optimization. It is assumed that $\phi$, the statement to prove or disprove, is known.
Future work could explore automatically inducing which statement $\phi$ to consider, possibly using syntactical reasoning in the process.

The paper formalizes Real Logic, the language supporting LTN.
The semantics of Real Logic are close to the semantics of Fuzzy FOL with the following major differences:
1) Real Logic domains are typed and restricted to real numbers and real-valued tensors,
2) Real Logic variables are sequences of fixed length, whereas FOL variables are a placeholder for any individual in a domain,
3) Real Logic relations relations are interpreted as mathematical functions, whereas Fuzzy Logic relations are interpreted as fuzzy set membership functions.
Concerning the semantics of connectives and quantifiers, some LTN implementations correspond to semantics for t-norm fuzzy logic, but not all.
For example, the conjunction operator in stable product semantics is not a t-norm, as pointed out at the end of Section <ref>.

Integrative neural-symbolic approaches are known for either seeking to bring neurons into a symbolic system (neurons into symbols) [41] or to bring symbols into a neural network (symbols into neurons) [60]. LTN adopts the latter approach but maintaining a close link between the symbols and their grounding into the neural network. The discussion around these two options - neurons into symbols vs. symbols into neurons - is likely to take center stage in the debate around neurosymbolic AI in the next decade. LTN and related approaches are well placed to play an important role in this debate by offering a rich logical language tightly coupled with an efficient distributed implementation into TensorFlow computational graphs.

The close connection between first-order logic and its implementation in LTN makes LTN very suitable as a model for the neural-symbolic cycle
[27, 29], which seeks to translate between neural and symbolic representations.
Such translations can take place at the level of the structure of a neural network, given a symbolic language [27], or at the level of the loss functions, as done by LTN and related approaches [13, 45, 46]. LTN opens up a number of promising avenues for further research:

Firstly, a continual learning approach might allow one to start with very little knowledge, build up and validate knowledge over time by querying the LTN network. Translations to and from neural and symbolic representations will enable reasoning also to take place at the symbolic level (e.g. alongside a proof system), as proposed recently in [70] with the goal of improving fairness of the network model.

Secondly, LTN should be compared in large-scale practical use cases with other recent efforts to add structure to neural networks such as the neuro-symbolic concept learner [44] and high-level capsules which were used recently to learn the part-of relation [38], similarly to how LTN was used for semantic image interpretation in [19].

Finally, LTN should also be compared with Tensor Product Representations, e.g. [59], which show that state-of-the-art recurrent neural networks may fail at simple question-answering tasks, despite achieving very high accuracy. Efforts in the area of transfer learning, mostly in computer vision, which seek to model systematicity could also be considered a benchmark [5]. Experiments using fewer data and therefore lower energy consumption, out-of-distribution extrapolation, and knowledge-based transfer are all potentially suitable areas of application for LTN as a framework for neurosymbolic AI based on learning from data and compositional knowledge.

§ ACKNOWLEDGEMENT

We would like to thank Benedikt Wagner for his comments and a number of  productive discussions on continual learning, knowledge extraction and reasoning in LTNs.

[1]
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen,
Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin,
Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard,
Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh
Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris
Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal
Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas,
Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and
Xiaoqiang Zheng.
TensorFlow: Large-scale machine learning on heterogeneous systems,
Software available from tensorflow.org.

[2]
Michael Akintunde, Elena Botoeva, Panagiotis Kouvaros, and Alessio Lomuscio.
Verifying strategic abilities of neural multi-agent systems.
In Proceedings of 17th International Conference on Principles of
Knowledge Representation and Reasoning, KR2020, Rhodes, Greece, September

[3]
Samy Badreddine and Michael Spranger.
Injecting Prior Knowledge for Transfer Learning into
Reinforcement Learning Algorithms using Logic Tensor Networks.
arXiv:1906.06576 [cs, stat], June 2019.
arXiv: 1906.06576.

[4]
Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, and Koray
Interaction networks for learning about objects, relations and
In Proceedings of the 30th International Conference on Neural
Information Processing Systems, NIPS'16, pages 4509–4517, USA, 2016. Curran
Associates Inc.

[5]
Yoshua Bengio, Tristan Deleu, Nasim Rahaman, Nan Rosemary Ke, Sebastien
Lachapelle, Olexa Bilaniuk, Anirudh Goyal, and Christopher Pal.
A meta-transfer objective for learning to disentangle causal
In International Conference on Learning Representations, 2020.

[6]
Federico Bianchi and Pascal Hitzler.
On the capabilities of logic tensor networks for deductive reasoning.
In Proceedings of the AAAI 2019 Spring Symposium on Combining
Machine Learning with Knowledge Engineering (AAAI-MAKE 2019) Stanford
University, Palo Alto, California, USA, March 25-27, 2019., Stanford
University, Palo Alto, California, USA, March 25-27, 2019., 2019.

[7]
Federico Bianchi, Matteo Palmonari, Pascal Hitzler, and Luciano Serafini.
Complementing logical reasoning with sub-symbolic commonsense.
In International Joint Conference on Rules and Reasoning, pages
161–170. Springer, 2019.

[8]
Rafael Borges, Artur d'Avila Garcez, and Luís Lamb.
Learning and representing temporal knowledge in recurrent networks.
IEEE transactions on neural networks / a publication of the IEEE
Neural Networks Council, 22:2409–21, 12 2011.

[9]
Liber Běhounek, Petr Cintula, and Petr Hájek.
Introduction to mathematical fuzzy logic.
In Petr Cintula, Petr Hájek, and Carles Noguera, editors, Handbook of Mathematical Fuzzy Logic, Volume 1, volume 37 of Studies in
Logic, Mathematical Logic and Foundations, pages 1–102. College
Publications, 2011.

[10]
N. A. Campbell and R. J. Mahon.
A multivariate study of variation in two species of rock crab of the
genus Leptograpsus.
Australian Journal of Zoology, 22(3):417–425, 1974.
Publisher: CSIRO PUBLISHING.

[11]
Andres Campero, Aldo Pareja, Tim Klinger, Josh Tenenbaum, and Sebastian Riedel.
Logical rule induction and theory learning using neural theorem
CoRR, abs/1809.02193, 2018.

[12]
Benhui Chen, Xuefen Hong, Lihua Duan, and Jinglu Hu.
Improving multi-label classification performance by label
In The 2013 International Joint Conference on Neural Networks
(IJCNN), pages 1–5. IEEE, 2013.

[13]
William W. Cohen, Fan Yang, and Kathryn Mazaitis.
Tensorlog: A probabilistic database implemented using deep-learning
J. Artif. Intell. Res., 67:285–325, 2020.

[14]
W.-Z. Dai, Q. Xu, Y. Yu, and Z.-H. Zhou.
Bridging machine learning and logical reasoning by abductive
In Proceedings of the 33rd International Conference on Neural
Information Processing Systems, NeurIPS'19, USA, 2019. Curran Associates

[15]
Alessandro Daniele and Luciano Serafini.
Knowledge enhanced neural networks.
In Pacific Rim International Conference on Artificial
Intelligence, pages 542–554. Springer, 2019.

[16]
Artur d'Avila Garcez, Marco Gori, Luís C. Lamb, Luciano Serafini,
Michael Spranger, and Son N. Tran.
Neural-symbolic computing: An effective methodology for principled
integration of machine learning and reasoning.
FLAP, 6(4):611–632, 2019.

[17]
Artur d'Avila Garcez and Luis C. Lamb.
Neurosymbolic AI: The 3rd wave, 2020.

[18]
Ivan Donadello and Luciano Serafini.
Compensating supervision incompleteness with prior knowledge in
semantic image interpretation.
In 2019 International Joint Conference on Neural Networks
(IJCNN), pages 1–8. IEEE, 2019.

[19]
Ivan Donadello, Luciano Serafini, and Artur d'Avila Garcez.
Logic tensor networks for semantic image interpretation.
In Proceedings of the Twenty-Sixth International Joint
Conference on Artificial Intelligence, IJCAI-17, pages 1596–1602, 2017.

[20]
Dheeru Dua and Casey Graff.
UCI machine learning repository, 2017.

[21]
Richard Evans and Edward Grefenstette.
Learning explanatory rules from noisy data.
J. Artif. Intell. Res., 61:1–64, 2018.

[22]
Ronald Fagin, Ryan Riegel, and Alexander Gray.
Foundations of reasoning with uncertainty via real-valued logics,

[23]
Marc Fischer, Mislav Balunovic, Dana Drachsler-Cohen, Timon Gehr, Ce Zhang, and
Martin Vechev.
Dl2: Training and querying neural networks with logic.
In International Conference on Machine Learning, pages
1931–1941, 2019.

[24]
Manoel Franca, Gerson Zaverucha, and Artur d'Avila Garcez.
Fast relational learning using bottom clause propositionalization
with artificial neural networks.
Machine Learning, 94:81–104, 01 2014.

[25]
Dov M. Gabbay and John Woods, editors.
The Many Valued and Nonmonotonic Turn in Logic, volume 8 of
Handbook of the History of Logic.
Elsevier, 2007.

[26]
Artur d'Avila Garcez, Dov M. Gabbay, and Krysia B. Broda.
Neural-Symbolic Learning System: Foundations and Applications.
Springer-Verlag, Berlin, Heidelberg, 2002.

[27]
Artur d'Avila Garcez, Lus C. Lamb, and Dov M. Gabbay.
Neural-Symbolic Cognitive Reasoning.
Springer Publishing Company, Incorporated, 1 edition, 2008.

[28]
Petr Hajek.
Metamathematics of Fuzzy Logic.
Kluwer Academic Publishers, 1998.

[29]
Barbara Hammer and Pascal Hitzler, editors.
Perspectives of Neural-Symbolic Integration, volume 77 of Studies in Computational Intelligence.
Springer, 2007.

[30]
Stevan Harnad.
The symbol grounding problem.
Physica D: Nonlinear Phenomena, 42(1-3):335–346, 1990.

[31]
Patrick Hohenecker and Thomas Lukasiewicz.
Ontology reasoning with deep neural networks.
Journal of Artificial Intelligence Research, 68:503–540, 2020.

[32]
Steffen Hölldobler and Franz J. Kurfess.
CHCL - A connectionist infernce system.
In Bertram Fronhöfer and Graham Wrightson, editors, Parallelization in Inference Systems, International Workshop, Dagstuhl
Castle, Germany, December 17-18, 1990, Proceedings, volume 590 of Lecture Notes in Computer Science, pages 318–342. Springer, 1990.

[33]
Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard Hovy, and Eric Xing.
Harnessing deep neural networks with logic rules.
In Proceedings of the 54th Annual Meeting of the Association for
Computational Linguistics (Volume 1: Long Papers), pages 2410–2420, Berlin,
Germany, August 2016. Association for Computational Linguistics.

[34]
Henry Kautz.
The Third AI Summer, AAAI Robert S. Engelmore
Memorial Lecture, Thirty-fourth AAAI Conference on Artificial
Intelligence, New York, NY, February 10, 2020.

[35]
Diederik P. Kingma and Jimmy Ba.
Adam: A Method for Stochastic Optimization.
arXiv:1412.6980 [cs], January 2017.
arXiv: 1412.6980.

[36]
Doga Kisa, Guy Van den Broeck, Arthur Choi, and Adnan Darwiche.
Probabilistic sentential decision diagrams.
In Proceedings of the 14th International Conference on
Principles of Knowledge Representation and Reasoning (KR), July 2014.

[37]
Erich Peter Klement, Radko Mesiar, and Endre Pap.
Triangular Norms, volume 8 of Trends in Logic.
Springer Netherlands, Dordrecht, 2000.

[38]
Adam Kosiorek, Sara Sabour, Yee Whye Teh, and Geoffrey E Hinton.
Stacked capsule autoencoders.
In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural
Information Processing Systems 32, pages 15512–15522. Curran Associates,
Inc., 2019.

[39]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton.
Imagenet classification with deep convolutional neural networks.
In Proceedings of the 25th International Conference on Neural
Information Processing Systems - Volume 1, NIPS'12, page 1097–1105, Red
Hook, NY, USA, 2012. Curran Associates Inc.

[40]
Luís C. Lamb, Artur d'Avila Garcez, Marco Gori, Marcelo O. R. Prates,
Pedro H. C. Avelar, and Moshe Y. Vardi.
Graph neural networks meet neural-symbolic computing: A survey and
In Christian Bessiere, editor, Proceedings of the Twenty-Ninth
International Joint Conference on Artificial Intelligence, IJCAI 2020
[scheduled for July 2020, Yokohama, Japan, postponed due to the Corona
pandemic], pages 4877–4884. ijcai.org, 2020.

[41]
Robin Manhaeve, Sebastijan Dumancic, Angelika Kimmig, Thomas Demeester, and
Luc De Raedt.
Deepproblog: Neural probabilistic logic programming.
In Proceedings of the 32nd International Conference on Neural
Information Processing Systems, NeurIPS'18, pages 3753–3763, USA, 2018.
Curran Associates Inc.

[42]
Francesco Manigrasso, Filomeno Davide Miro, Lia Morra, and Fabrizio Lamberti.
Faster-LTN: a neuro-symbolic, end-to-end object detection
arXiv:2107.01877 [cs], July 2021.

[43]
Vasco Manquinho, Joao Marques-Silva, and Jordi Planes.
Algorithms for weighted boolean optimization.
In International conference on theory and applications of
satisfiability testing, pages 495–508. Springer, 2009.

[44]
Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, and Jiajun Wu.
The neuro-symbolic concept learner: Interpreting scenes, words, and
sentences from natural supervision.
CoRR, abs/1904.12584, 2019.

[45]
Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, and Marco Gori.
Constraint-based visual generation.
In Igor V. Tetko, Vera Kurková, Pavel Karpov, and Fabian J.
Theis, editors, Artificial Neural Networks and Machine Learning -
ICANN 2019: Image Processing - 28th International Conference on Artificial
Neural Networks, Munich, Germany, September 17-19, 2019, Proceedings, Part
III, volume 11729 of Lecture Notes in Computer Science, pages
565–577. Springer, 2019.

[46]
Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, and Marco Gori.
Integrating learning and reasoning with deep logic models.
In Machine Learning and Knowledge Discovery in Databases -
European Conference, ECML PKDD 2019, Würzburg, Germany, September
16-20, 2019, Proceedings, Part II, volume 11907 of Lecture Notes in
Computer Science, pages 517–532. Springer, 2019.

[47]
Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, and Marco Gori.
Lyrics: A general interface layer to integrate logic inference and
deep learning.
In Joint European Conference on Machine Learning and Knowledge
Discovery in Databases, pages 283–298. Springer, 2019.

[48]
Giuseppe Marra and Ondřej Kuželka.
Neural markov logic networks.
arXiv preprint arXiv:1905.13462, 2019.

[49]
Pasquale Minervini, Sebastian Riedel, Pontus Stenetorp, Edward Grefenstette,
and Tim Rocktäschel.
Learning reasoning strategies in end-to-end differentiable proving,

[50]
Stephen H. Muggleton, Dianhuan Lin, Niels Pahlavi, and Alireza
Meta-interpretive learning: Application to grammatical inference.
Mach. Learn., 94(1):25–49, January 2014.

[51]
Karl Pearson.
Liii. on lines and planes of closest fit to systems of points in
The London, Edinburgh, and Dublin Philosophical Magazine and
Journal of Science, 2(11):559–572, 1901.

[52]
Gadi Pinkas.
Reasoning, nonmonotonicity and learning in connectionist networks
that capture propositional knowledge.
Artif. Intell., 77(2):203–247, 1995.

[53]
Meng Qu and Jian Tang.
Probabilistic logic neural networks for reasoning.
In Advances in Neural Information Processing Systems, pages
7712–7722, 2019.

[54]
Luc De Raedt, Sebastijan Dumančić, Robin Manhaeve, and Giuseppe Marra.
From statistical relational to neuro-symbolic artificial
intelligence, 2020.

[55]
Matthew Richardson and Pedro Domingos.
Markov logic networks.
Mach. Learn., 62(1-2):107–136, February 2006.

[56]
Ryan Riegel, Alexander Gray, Francois Luus, Naweed Khan, Ndivhuwo Makondo,
Ismail Yunus Akhalwaya, Haifeng Qian, Ronald Fagin, Francisco Barahona, Udit
Sharma, Shajith Ikbal, Hima Karanam, Sumit Neelam, Ankita Likhyani, and
Santosh Srivastava.
Logical Neural Networks.
arXiv:2006.13155 [cs], June 2020.
arXiv: 2006.13155.

[57]
Tim Rocktäschel and Sebastian Riedel.
End-to-end differentiable proving.
In Advances in Neural Information Processing Systems, pages
3788–3800, 2017.

[58]
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele
The graph neural network model.
Trans. Neur. Netw., 20(1):61–80, January 2009.

[59]
Imanol Schlag and Jürgen Schmidhuber.
Learning to reason with third-order tensor products.
CoRR, abs/1811.12143, 2018.

[60]
Imanol Schlag, Paul Smolensky, Roland Fernandez, Nebojsa Jojic, Jürgen
Schmidhuber, and Jianfeng Gao.
Enhancing the transformer with explicit relational encoding for math
problem solving.
CoRR, abs/1910.06611, 2019.

[61]
Luciano Serafini and Artur d'Avila Garcez.
Logic tensor networks: Deep learning and logical reasoning from data
and knowledge.
arXiv preprint arXiv:1606.04422, 2016.

[62]
Luciano Serafini and Artur d’Avila Garcez.
Learning and reasoning with logic tensor networks.
In Conference of the Italian Association for Artificial
Intelligence, pages 334–348. Springer, 2016.

[63]
Lokendra Shastri.
Advances in SHRUTI-A neurally motivated model of relational
knowledge representation and rapid inference using temporal synchrony.
Appl. Intell., 11(1):79–108, 1999.

[64]
Yun Shi.
A deep study of fuzzy implications.
PhD thesis, Ghent University, 2009.

[65]
Paul Smolensky and Géraldine Legendre.
The Harmonic Mind: From Neural Computation to
Optimality-Theoretic GrammarVolume I: Cognitive Architecture (Bradford
The MIT Press, 2006.

[66]
Gustav Sourek, Vojtech Aschenbrenner, Filip Zelezny, Steven Schockaert, and
Ondrej Kuzelka.
Lifted relational neural networks: Efficient learning of latent
relational structures.
Journal of Artificial Intelligence Research, 62:69–100, 2018.

[67]
Emile van Krieken, Erman Acar, and Frank van Harmelen.
Semi-Supervised Learning using Differentiable Reasoning.
arXiv:1908.04700 [cs], August 2019.
arXiv: 1908.04700.

[68]
Emile van Krieken, Erman Acar, and Frank van Harmelen.
Analyzing Differentiable Fuzzy Implications.
In Proceedings of the 17th International Conference on
Principles of Knowledge Representation and Reasoning, pages 893–903, 9

[69]
Emile van Krieken, Erman Acar, and Frank van Harmelen.
Analyzing Differentiable Fuzzy Logic Operators.
arXiv:2002.06100 [cs], February 2020.
arXiv: 2002.06100.

[70]
Benedikt Wagner and Artur d'Avila Garcez.
Neural-Symbolic Integration for Fairness in AI.
Proceedings of the AAAI Spring Symposium: Combining Machine
Learning with Knowledge Engineering 2021, page 14, 2021.

[71]
Po-Wei Wang, Priya L Donti, Bryan Wilder, and Zico Kolter.
Satnet: Bridging deep learning and logical reasoning using a
differentiable satisfiability solver.
arXiv preprint arXiv:1905.12149, 2019.

[72]
Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Broeck.
A Semantic Loss Function for Deep Learning with Symbolic
In International Conference on Machine Learning, pages
5502–5511. PMLR, July 2018.
ISSN: 2640-3498.

§ IMPLEMENTATION DETAILS

The LTN library is implemented in Tensorflow 2 [1] and is available from GitHub[https://github.com/logictensornetworks/logictensornetworks].
Every logical operator is grounded using Tensorflow primitives.
The LTN code implements directly a Tensorflow graph.
Due to Tensorflow built-in optimization, LTN is relatively efficient while providing the expressive power of FOL.

Table <ref> shows an overview of the network architectures used to obtain the results of the examples in Section <ref>.
The LTN repository includes the code for these examples.
Except if explicitly mentioned otherwise, the reported results are averaged over 10 runs using a 95% confidence interval.
Every example uses a stable real product configuration to approximate Real Logic operators, and the Adam optimizer [35] with a learning rate of $0.001$ to train the parameters.

Task     Network     Architecture

<ref>     $\mathtt{MLP}$     $\Dense(16)^\ast$, $\Dense(16)^\ast$, $\Dense(1)$

<ref>     $\mathtt{MLP}$     $\Dense(16)^\ast$, $\Dropout(0.2)$, $\Dense(16)^\ast$, $\Dropout(0.2)$,

$\Dense(8)^\ast$, $\Dropout(0.2)$, $\Dense(1)$

<ref>     $\mathtt{MLP}$     $\Dense(16)^\ast$, $\Dense(16)^\ast$, $\Dense(8)^\ast$, $\Dense(1)$

<ref>     $\mathtt{CNN}$     $\MNISTConv$, $\Dense(84)^\ast$, $\Dense(10)$

$\mathrm{baseline-SD}$     $\MNISTConv\times 2$, $\Dense(84)^\ast$, $\Dense(19)$, $\mathrm{Softmax}$

$\mathrm{baseline-MD}$     $\MNISTConv\times 4$, $\Dense(128)^\ast$, $\Dense(199)$, $\mathrm{Softmax}$

<ref>     $\mathtt{MLP}$      $\Dense(8)^\ast$, $\Dense(8)^\ast$, $\Dense(1)$

<ref>     $\mathtt{MLP}$     $\Dense(16)^\ast$, $\Dense(16)^\ast$, $\Dense(16)^\ast$, $\Dense(1)$

<ref>     $\mathtt{MLP\_S}$     $\Dense(8)^\ast$, $\Dense(8)^\ast$, $\Dense(1)$

$\mathtt{MLP\_F}$      $\Dense(8)^\ast$, $\Dense(8)^\ast$, $\Dense(1)$

$\mathtt{MLP\_C}$      $\Dense(8)^\ast$, $\Dense(8)^\ast$, $\Dense(1)$

* $^\ast$ : layer ends with an $\mathtt{elu}$ activation

* $\Dense(n)$ : regular fully-connected layer of $n$ units

* $\Dropout(r)$ : dropout layer with rate $r$

* $\Conv(f,k)$ : 2D convolution layer with $f$ filters and a kernel of size $k$

* $\mathrm{MP}(w,h)$ : max pooling operation with a $w \times h$ pooling window

* $\MNISTConv$ : $\Conv(6,5)^\ast$, $\mathrm{MP}(2,2)$, $\Conv(16,5)^\ast$, $\mathrm{MP}(2,2)$, $\Dense(100)^\ast$

Overview of the neural network architectures used in each example.
Notice that in the examples, the networks are usually used with some additional layer(s) to ground symbols.
For instance, in experiment <ref>, in $\G(P) : x,l \mapsto l^\top \mathtt{softmax}(\mathtt{MLP}(x))$,
the softmax layer normalizes the raw predictions of $\mathtt{MLP}$ to probabilities in $[0,1]$,
and the multiplication with the one-hot label $l$ selects the probability for one given class.