# A Number Sense as an Emergent Property
of the Manipulating Brain

N. Kondapaneni and P. Perona
California Institute of Technology

## Abstract

The ability to understand and manipulate numbers and quantities emerges during
childhood, but the mechanism through which humans acquire and develop this
ability is still poorly understood. In particular, it is not known whether
acquiring such a number sense is possible without supervision from a teacher.

We explore this question through a model, assuming that the learner is able to
pick and place small objects and will spontaneously engage in undirected
manipulation. We further assume that the learner’s visual system will monitor
the changing arrangements of objects in the scene and will learn to predict
the effects of each action by comparing perception with the efferent signal of
the motor system. We model perception using standard deep networks for feature
extraction and classification, and gradient descent learning.

Our main finding is that, from learning the unrelated task of action
prediction, an unexpected image representation emerges exhibiting regularities
that foreshadow the perception and representation of numbers and quantity.
These include distinct categories for zero and the first few natural numbers,
a strict ordering of the numbers, and a one-dimensional signal that correlates
with numerical quantity. As a result, our model acquires the ability to
estimate numerosity, i.e. the number of objects in the scene, as well as
subitization, i.e. the ability to recognize at a glance the exact number of
objects in small scenes. Remarkably, subitization and numerosity estimation
extrapolate to scenes containing many objects, far beyond the three objects
used during training.

We conclude that important aspects of a facility with numbers and quantities
may be learned without teacher supervision.

## Introduction

Mathematics, one of the most distinctive expressions of human intelligence, is
founded on the ability to reason about abstract entities. We are interested in
the question of how humans develop an intuitive facility with numbers and
quantities, and how they come to recognize numbers as an abstract property of
sets of objects. There is wide agreement that innate mechanisms play a strong
role in developing a number sense [1, 2, 3], that development and learning
also play an important role [2], that naming numbers is not necessary for the
perception of quantities [4, 5], and that specific brain areas are involved in
processing numbers [6, 7]. As to the role of learning, we do not yet know
whether developing a number sense requires a teacher’s supervision or whether
unsupervised learning is sufficient.

The role of supervised learning in developing abilities that relate to the
natural numbers and estimation has been recently explored using computational
models. Fang et al. [8] trained a recurrent neural network to count
sequentially and Sabathiel et al. [9] showed that a neural network can be
trained to anticipate the actions of a teacher on three counting-related tasks
– they find that specific patterns of activity in the network’s units
correlate with quantities. The ability to perceive numerosity, i.e. a rough
estimate of the number of objects in a set, was explored by Stoianov, Zorzi
and Testolin [10, 11], who trained a deep network encoder to efficiently
reconstruct patterns composed of dots, and found that the network developed
units or ‘neurons’ that were coarsely tuned to quantity, and by Nasr et al.
[12], who found the same effect in a deep neural network that was trained on
visual object classification, an unrelated task. In these models quantity-
sensitive units are an emergent property. In a recent study, Kim et al. [13]
observed that a random network with no training will exhibit quantity-
sensitive units.

It thus intuitive that supervised learning is sufficient for training
perception to estimate quantities. This is because emergent quantity-tuned
units that are observed in previous studies will be automatically recruited as
an input by a supervised classifier that is trained to estimate numerosity. Is
supervised learning, i.e. the intervention of an external teacher, also
necessary? We are interested in the question of whether the natural numbers,
as an ordered set of abstract concepts, as well as the effortless and
spontaneous perception of quantity, may instead be learned without explicit
supervision.

We explore the hypothesis that a facility with numbers and quantities may
arise without the help of a teacher – we call this unsupervised learning
following computer science nomenclature. We focus on the interplay of action
and perception as a possible avenue for this to happen. More specifically, we
explore whether perception, as it is naturally trained during object
manipulation, may develop representations that support a number sense. In
order to test this hypothesis we propose a model where perception learns how
specific actions modify the world. The model shows that perception develops a
representation of the scene which, as an emergent property, can enable the
ability to manipulate numbers and estimate quantities at a glance [14, 15].
Thus, we find that a teacher is not needed.

In order to ground intuition, consider a child who has learned to pick up
objects, one at a time, and let them go at will. Imagine the child sitting
comfortably and playing with small toys (acorns, Legos, sea shells) which may
be dropped into a bowl. We will assume that the child has already learned to
perform, and tell apart, three distinct operations. The put (P) operation
consists of picking up an object from the surrounding space and dropping it
into the bowl. The take (T) operation consists in doing the opposite: picking
up an object from the bowl and discarding it. The shake (S) operation consists
of agitating the bowl so that the objects inside change their position
randomly without falling out. Objects in the bowl may be randomly moved during
put and take as well. We hypothesize that the visual system of the child is
progressively becoming sensitive to the changes that are caused by
manipulation [16]. As a result, the visual system is progressively trained
through spontaneous play to predict (or, more accurately, post-dict) which
operation took place that changed the appearance of the bowl: was it a put, a
take or a shake?

Figure 1: Schematics of our model. (A) (Bottom-to-top) The scene changes as a
result of manipulation. The images $x_{t}$ and $x_{t+1}$ of the scene before
and after manipulation are mapped by perception into representations $z_{t}$
and $z_{t+1}$. These are compared by a classifier to predict which action took
place. Learning monitors the error between predicted action and the efferent
copy of the actual action, and updates simultaneously the weights of both
perception and the classifier to increase prediction accuracy. (B) (Bottom-to-
top) Our model of perception is a hybrid neural network composed of the
concatenation of a convolutional neural network (CNN) with a fully-connected
network (FCN 1). The classifier is implemented by a fully connected network
(FCN 2) which compares the two representations $z_{t}$ and $z_{t+1}$. The two
perception networks are actually the same network operating on distinct images
and therefore their parameters are identical and learned simultaneously in a
Siamese network configuration [17]. Details of the models are given in Fig.
S1.

A

B

Figure 2: Training image sequence samples. We trained our model using
sequences of images that were generated by randomly concatenating take (T),
put (P) and shake (S) manipulations, while limiting the number of objects to
the $\left\\{0\dots 3\right\\}$ set (see Methods - Training Sets). We
experimented with two different environment/scene statistics: (A) Identical
objects (15x15 pixel squares) with random position. (B) Objects (squares) of
variable position, size and contrast. The overall image intensity is a poor
predictor of cardinality in this dataset (statistics in Fig. S2). Note that
some of the objects in (B) have very low contrast and may not be visible on
all displays.

A number of perceptual maps are known to arise during development, each one of
which contains mechanisms that are tuned to specific scene properties, as
simple as orientation [18] and boundaries [19], and as complex as faces [20]
and objects [21, 22]. We propose that, while the child is playing, the visual
system is being trained to use one or more such maps to build a representation
that facilitates the comparison of the pair of images that are seen before and
after a manipulation. Simultaneously, a classifier network is trained to
predict the action (P,T,S) from the representation of the pair of images (see
Fig. 2). Using a simple model of this putative mechanism, we find that the
image representation that is being learned for classifying actions,
simultaneously learns to represent and perceive the first few natural numbers,
to place them in the correct order, from zero to one and beyond, as well as
estimate the number of objects in the scene.

## Results

We postulate that the efferent signals from the motor system are available to
the visual system and are used as a supervisory signal (Fig 2A). Such signals
provide information regarding the three actions of put, take and shake and,
accordingly, perception may be trained to predict these three actions. Note
that no explicit signal indicating the number of objects in the scene is
available to the visual system at any time.

We use a standard deep learning model of perception [23, 24, 25]: a feature
extraction stage is followed by a classifier (Fig. 2B). The feature extraction
stage maps the image $x$ to an internal representation $z$, often called an
embedding. It is implemented by a deep network [24] composed of convolutional
layers (CNN) followed by fully connected layers (FCN 1). The classifier,
implemented with a simple fully connected network (FCN 2), compares the
representations $z_{t}$ and $z_{t+1}$ of the before and after images to
predict which action took place. Feature extraction and classification are
trained jointly by minimizing the prediction error. We find that the embedding
dimension makes little difference to the performance of the network (Fig. S5).
Thus, for ease of visualization, we settled on two dimensions.

We carried out train-test experiments using sequences of synthetic images
containing a small number of randomly arranged objects (Fig. 2). When training
we limited the top number of objects to three (an arbitrary choice), and each
pair of subsequent images was consistent with one of the manipulations (put,
take, shake). We ran our experiments twice with different object statistics.
In the first dataset the objects were identical squares, in the second they
had variable size and contrast. In the following we refer to the model trained
on the first dataset as Model 1 and the model trained on the second dataset as
Model 2.

Figure 3: Action classification performance. The network accurately classifies
actions up to the training limit of three objects, regardless of the
statistics of the data (the x axis indicates the number of objects in the
scene before the action takes place). Error increases when the number of
objects in the test images exceeds the number of objects in the training set.
95% Bayesian confidence intervals are shown by the shaded areas (272 $\leq$ N
$\leq$ 360). The gray region highlights test cases where the number of objects
exceeds the number in the training set.

We found that models learn to predict the three actions on a test set of novel
image sequences (Fig. 3) with an error below 1% on scenes up to three objects
(the highest number during training). Performance degrades progressively for
higher numbers beyond the training range. Model 2’s error rate is higher,
consistently with the task being harder. Thus, we find that our model learns
to predict actions accurately. However, there is little ability to generalize
the task to scenes containing previously unseen numbers of objects. This will
become relevant later.

A

B

C

Figure 4: Visualizing the embedding space for Model 2. To explore the
structure of the embedding space, we generated a dataset with $\left\\{0\dots
30\right\\}$ objects, extending the number of objects far beyond the limit of
3 objects in the training task. Each image in the dataset was passed through
Model 2 and the output (the internal representation/embedding) of the image is
shown. (A) Each dot indicates an image embedding and the embeddings happen to
be arranged along a line. The number of objects in each image is color coded.
The smooth gradation of the color suggests that the embeddings are arranged
monotonically with respect to the number of objects in the corresponding
image. The inset shows that the embeddings of the images that contain only a
few objects are arranged along the line into ‘islands’. (B) We apply an
unsupervised clustering algorithm to the embeddings. Each cluster that is
discovered is denoted by a specific color. The cluster X, denoted by black
crosses, indicates points that the clustering algorithm excluded as outliers.
(C) The confusion matrix shows that the clusters that are found by the
clustering algorithm correspond to numbers. Images containing 0 - 6 objects
are neatly separated into individual clusters; after that images are collected
into a large group that is not in one-to-one correspondence with the number of
objects in the image. Note that the color scale is logarithmic (base 10).

When we examined the structure of the embedding we were intrigued to find a
number of unexpected regularities (Fig. 4). First, the images’ representations
do not spread across the embedding, filling the available dimensions, as is
usually the case. Rather, they are arranged along a one-dimensional structure.
This trait is very robust to extrapolation: after training (with up to three
objects), we computed the embedding of novel images that contained up to
thirty objects and found that the line-like structure persisted (Fig. 4A).
This embedding line is also robust with respect to the dimensions of the
embedding – we tested from two to 256 and observed it each time.

Second, images are arranged almost monotonically along the embedding line
according to the number of objects that are present (Fig. 4A). Thus, the
representation that is developed by the model contains an order. We were
curious as to whether the embedding coordinate, i.e. the position of an image
along the embedding line, may be used to estimate the number of objects in the
image. Any one of the features that make up the coordinates of the embedding
provides a handy measure for this position, measured as the distance from the
beginning of the line – the value of these coordinates may be thought of as
the firing rate of specific neurons [26]. We tested this hypothesis both in a
relative and in an absolute quantity estimation task. First, we used the
embedding coordinate to compare the number of objects in two different images
and assess which is larger, and found very good accuracy (Fig. 5). Second,
assuming that the system may self-calibrate, e.g. by using the ‘put’ action to
estimate a unit of increment, then an absolute measure of quantity may be
computed from the embedding coordinate. We tested this idea by computing such
a perceived number against the actual count of objects in images (Fig. 6). The
estimates turn out to be quite accurate, with a slight underestimate that
increases as the numbers become larger. Both relative and absolute estimates
of quantity were accurate as far as thirty objects (we did not test beyond
this number), which far exceeds the training limit of three. We looked for
image properties, other than ‘number of objects’, that might drive the
estimate of quantity and we could not find any convincing candidate (see
Methods and Fig. S3).

Third, image embeddings separate out into distinct ‘islands’ at one end of the
embedding line (Fig. 4A inset). The brain is known to spontaneously cluster
perceptual information [27, 7], and therefore we tested empirically whether
this form of unsupervised learning may be sufficient to discover distinct
categories of images/scenes from their embedding. We found that unsupervised
learning successfully discovers the clusters with very few outliers in both
Model 1 and the more challenging Model 2 (Fig. 4B).

Fourth, the first few clusters discovered by unsupervised learning along the
embedding line are in almost perfect one-to-one correspondence with groups of
images that share the same number of objects (Figs. 4C). Once such distinct
number categories are discovered, they may be used to classify images. This is
because the model maps the images to the embedding, and the unsupervised
clustering algorithm can classify points in the embedding into number
categories. Thus, our model learns the ability to carry out instant
association of images with a small set of objects with the corresponding
number category.

A fifth property of the embedding is that there is a limit to how many
distinct number categories are learned. Beyond a certain number of objects one
finds large clusters which are no longer number-specific (Fig. 4). I.e. our
model learns distinct categories for the numbers between zero and eight, and
additional larger categories for, say, “more than a few” and for “many”.

There is nothing magical in the fact that during training we limited the
number of objects to three – the regularities we observed are robust with
respect to the choice of the number of objects that are used in training the
action classifier (Fig. S7, S8).

Figure 5: Comparative estimation of quantity. Two images may be compared for
quantity [28] by computing their embedding and observing their position along
our model’s embedding line: the image that is furthest along the line is
predicted to contain more objects. Here images containing a test number of
objects (see three examples above containing N=12, 16 and 20 objects) are
compared with images containing the reference number of objects (orange line,
N=16). The number of objects in the test image is plotted along the x axis and
the proportion of comparisons that result in a ‘more’ response are plotted on
the y-axis (blue line). Human data from 10 subjects [29] is plotted in green.
Figure 6: Unsupervised numerosity perception. The position of images in the
embedding space fall along a straight line that starts with 0, and continues
monotonically with an increasing number of objects. Thus, the position of an
image in the embedding line is an estimate for the number of objects in the
scene. Here we demonstrate the outputs of such a model, where we rescale the
embedding coordinate (an arbitrary unit) so that one unit of distance matches
the distance between the “zero” and the “one” clusters. The y-axis represents
such perceived numerosity, which is not necessarily an integer value. The red
line indicates perfect prediction. Each violin plot (light blue) indicates the
distribution of perceived numerosities for a given ground-truth number of
objects. The width of the distributions for the higher counts indicates that
perception is subject to errors. There is a slight underestimation bias for
higher numbers, consistent with that seen in humans [30, 31]. In fact, Krueger
shows that human numerosity judgements (on images with 20 to 400 objects)
follow a power function with an exponent of $0.83\pm 0.2$. The green line and
its shadow depict the range of human numerosity predictions on the same task.
The orange lines are power function fits for seven models trained in the same
fashion as Model 2 with different random initializations.

## Discussion

Our model and experiments demonstrate that a number sense may be learned by an
agent that freely engages in object manipulation without supervision from a
teacher. The two mechanisms of the model, deep learning and unsupervised
clustering, are computational abstractions of mechanisms that have been
documented in the brain. The model observes the effect of manipulation on
small sets of objects and learns to predict actions. The image representation
that is developed in the process presents emergent regularities which confer
to the model a number of unexpected properties.

First, the model discovers the structure underlying the integers. The first
few numbers, from zero to six, say, emerge as categories from spontaneous
clustering of the embeddings of the corresponding images. Clustered
topographic numerosity maps observed in human cortex may be viewed as
confirming this prediction [7]. These number categories are naturally ordered
by their position on the embedding line, a fundamental property of numbers.
The ability to think about numbers may be thought as a necessary, although not
sufficient, step towards counting, addition and subtraction [32, 33]. The
dissociation between familiarity with the first few numbers and the ability to
count has been observed in hunter-gatherer societies [5] suggesting that these
are distinct steps in cognition.

Second, instant classification of the number of objects in the scene is
enabled by the emergence of number categories in the embedding. This predicts
a well-known capability of humans, commonly called subitization [14, 34],
limited to the first few numbers, a property we observe in our model as well.

Third, a linear structure, which we call embedding line, where images are
ordered according to quantity, is an emergent representation. This prediction
is strongly reminiscent of the mental number line which has been postulated in
the psychology literature [35, 36, 37, 38]. The embedding line confers to the
model the ability to estimate quantities both in relative comparisons and in
absolute judgments. The model predicts the ability to carry out relative
estimation, absolute estimation, as well as the tendency to slight
underestimation in absolute judgments. These predictions are confirmed in the
psychophysics literature [28, 30].

Fourth, subitization and numerosity estimation extend far beyond the number of
objects using in training. While the model trains itself to classify actions
using up to three objects, subitization extends to 5-8 objects and numerosity
estimation extends to at least thirty, which is as far as we tested.
Extrapolating from the training set is a hallmark of abstraction, which eludes
most supervised models [39].

The model is simple and essentially parameter-free, in that our observations
are robust with respect to large variations in the dimension of the embedding,
the number of objects in the training set and the tuning parameters of the
clustering algorithm. Yet, the model accounts qualitatively and, to some
extent, quantitatively for a disparate set of observations by psychologists,
psychophysicists and cognitive scientists.

There is a debate in the literature on whether estimation and subitization are
supported by the same mechanisms or separate ones [28, 40]. Our model suggests
a solution that will please both sides: both perceptions rely on a common
representation, the embedding. However, the two depend on different mechanisms
that take input from this common representation. Furthermore, our model
predicts that adaptation affects estimation, but not subitzation. This is
because subitization solely relies on classifiers, which allows for a direct
estimate of quantity. Estimation, however, relies on an analog variable, the
coordinate along the embedding line, which requires calibration. These
predictions are confirmed in the psychophysics literature [30, 28]. Our model
predicts the existence of summation units, which have been documented in the
physiology literature [26] and have been postulated in previous models [41].
It does not rule out the simultaneous presence of other codes, such as
population codes or labeled-line codes [42].

It is important to recognize the limitations of our model: it is designed to
explore the minimal conditions that are required for the emergence of a number
sense, and abstracts over the details of a specific implementation in the
brain. For instance, we limit the model to vision, while it is known that
multiple sensory systems may contribute, including hearing, touch and self-
produced actions [43, 44, 45]. Furthermore, the visual system serves multiple
tasks, such as face processing, object recognition, and navigation. Thus, it
is likely that multiple visual maps are simultaneously learned, and it is
possible that our ‘latent representation’ is shared with other visual
modalities [12]. Additionally, we postulate that visually-guided manipulation,
and hence the ability to detect and locate objects, is learned before numbers.
Thus, it would perhaps be more realistic to consider input from an
intermediate map where objects have been already detected and located, and are
thus represented as ‘tokens’, in visual space, and this would likely make the
model’s task easier, perhaps closer to Model 1 than to Model 2. However,
making this additional assumption is not necessary for our observations.

Our investigation adds a concrete case study to the discussion on how
abstraction may be learned without explicit supervision. While images
containing, say, five objects will look very different from each other, our
model discovers a common property, i.e. the number of items, which is not
immediately available from the brightness distribution or other scene
properties. The mechanism driving such abstraction may be interpreted as an
implicit contrastive learning signal [46], where the shake action identifies
pairs of images that ought to be considered as similar, while the put and take
actions signal pairs of images that ought to be considered dissimilar, hence
the clustering. However, there is a crucial difference between our model and
traditional contrastive learning. In contrastive learning it is the designer
who hand-crafts the similarity and dissimilarity training signals in order to
achieve an intended learning goal. I.e. the abstraction is directly built
into, not discovered by, the network. By contrast, in our model it is the
network itself that associates a meaning of ‘close’ and ‘far’ to the P,T,S
actions, and ultimately discovers the abstraction. This abstraction is
surprisingly strong – while the primary supervised task, action
classification, does not generalize well beyond the three objects used in
training, the unsupervised abstractions of number and quantity extend far
beyond it.

## Methods

### Network Structure

The network we train is a standard deep network [25] composed of two stages.
First, a feature extraction network maps the original image of the scene into
an embedding space (Fig. 2A). Second, a classification network takes the
embedding of two sequential images and predicts the action that modified the
first into the second (Fig. 2B). Given the fact that the classification
network takes the embedding of two distinct images as its input, each computed
by identical copies of the feature extraction network, the latter is trained
in a Siamese configuration [17].

The feature extraction network is a 9-layer CNN followed by two fully
connected layers (details in Fig. S1A). The first 3 layers of the feature
extraction network are from AlexNet [24] pre-trained on ImageNet [47] and are
not updated during training. The remaining four convolutional layers and two
fully connected layers are trained in our action prediction task.

The dimension of the output of the final layer is a free parameter (it
corresponds to the number of features and to the dimension of the embedding
space). In a control experiment we varied this dimension from one to 256, and
found little difference in the action classification error rates (Fig. S5). We
settled for a two-dimensional output for the experiments that are reported
here.

The classification network is a two-layer fully connected network that outputs
a three-dimensional one-hot-encoding vector indicating a put, take or shake
action (details in
Fig. S1B).

### Training procedure

The network was trained with a negative log-likelihood loss (NLL loss)
function with a learning rate of 1e-4. The NLL loss calculates error as the
-log of the probability of the correct class. Thus, if the probability of the
correct class is low (near 0), the error is higher. The network was trained
for 30 epochs with 30 mini-batches in each epoch. Each mini-batch was created
from a sequence of 180 actions, resulting in 180 image pairs. Thus, the
network saw a total of 162,000 unique pairs of images over the course of
training.

We tested for reproducibility by training Model 2 thirty times with different
random initializations of the network and different random seeds in our
dataset generation algorithm. The embeddings for these reproduced models are
shown in Figure S8.

### Training sets

We carried out separate experiments using synthetic image sequences where
objects were represented by randomly positioned squares. The images are
244x244 pixels (px) in size. Objects were positioned with uniform probability
in the image, with the exception that they were not allowed to overlap and a
margin of at least 3px clearance between them was imposed. We used two
different statistics of object appearance: identical size (15px) and contrast
(100%) in the first, and variable size (10px - 30px) and contrast (9.8% -
100%) in the second (Fig. 2). Mean image intensity statistics for the two
training sets are shown in Figure S2. The mean image intensity is highly
correlated with the number of objects in the first dataset, while it is
ambiguous and thus not very informative in the second. We studied a number of
other potential correlates and we were unable to find an image property that
would ‘explain away’ the abstraction of number (Fig.S3). We note that a
principled analysis of the information that is carried out by individual
object images is still missing from the literature [48] and this point
deserves more attention.

Each training sequence was generated starting from zero objects, and then
selecting a random action (put, take, shake) to generate the next image. The
take action is meaningless when the scene contains zero objects and was thus
not used there. We also discarded put actions when the objects reached a
maximum number. This limit was three for most experiments, but limits of five
and eight objects were also explored (Fig. S7).

### Test sets

In different experiments we allowed up to eight objects per image (Figs. 3,
S7) and thirty objects per image (Figs. 4, 5, 6) in order to assess whether
the network can generalize to tasks on scenes containing previously unseen
numbers of objects. The first test set (up to 8 objects) was generated
following the same recipe as the training set. The second test (up to 30
objects) set was generated to have random images with the specified number of
objects (without using actions), this test set is guaranteed to be balanced.

### Action classification performance

To visualize how well the model was able to perform the action classification
task, we predict actions between pairs of images in our first test set. The
error, calculated by comparing the ground truth actions to the predicted
actions, is plotted with respect to the number of objects in the visual scene
at $x_{t}$. 95% Bayesian confidence intervals with a uniform prior were
computed for each data point, and a lower bound on the number of samples is
provided in the figure captions (Figs. 3, S5, S7).

### Interpreting the embedding space

We first explored the structure of the embedding space by visualizing the
image embeddings in two dimensions. The points, each one of which corresponds
to one image, are not scattered across the embedding. Rather, they are
organized into a structure that exhibits five salient features: (a) the images
are arranged along a one-dimensional structure, (b) the ordering of the points
along the line is (almost) monotonic with respect to the number of objects in
the corresponding images, (c) images are separated into groups at one end of
the embedding, and these groups are discovered by unsupervised learning, (d)
these first few clusters are in one-to-one correspondence with the first few
natural numbers, (e) there is a limit to how many number-specific clusters are
discovered (Fig. 4).

To verify that the clusters can be recovered by unsupervised learning we
applied a standard clustering algorithm, and found almost perfect
correspondence between the clusters and the first few natural numbers (Fig.
4B). The clustering algorithm used was the default Python implementation of
HDBSCAN [49]. HDBSCAN is a hierarchical, density based clustering algorithm,
and we used the euclidean distance as an underlying metric [50]. HDBSCAN has
one main free parameter, the minimum cluster size, which was set to 70 in
Figure 4. All other free parameters were left at their default values. Varying
the minimum cluster size between 5 and 80 does not have an effect on the first
few clusters, although it does create variation in the number and size of the
later clusters. Beyond 80, the algorithm finds only three clusters
corresponding to 0, 1 and greater than 1.

One additional structure is not evident from the the embedding and may be
recovered from the action classifier: the connections between pairs of
clusters. For any pair of images that are related by a manipulation, two
computations will be simultaneously carried out; first, the supervised action
classifier in the model will classify the action as either P, T, or S (Fig. 3)
and, at the same time, the unsupervised subitization classifier (Fig. 8(b))
will assign each image in the pair to the corresponding number-specific
cluster. As a result, each pair of images that is related by a P action
provides a directed link between a pair of clusters (Fig. 8(b), red arrows),
and following such links one may traverse the sequence of numbers in an
ascending order. The T actions provide the same ordering in reverse (blue
arrows). Thus, the clusters corresponding to the first few natural numbers are
strung together like the beads in a necklace, providing an unambiguous
ordering that starts from zero and proceeds through one, two etc. (Fig. 8(b)).
The numbers may be visited both in ascending and descending order. As we
pointed out earlier, the same organization may be be obtained more simply by
recognizing that the clusters are spontaneously arranged along a line, which
also supports the natural ordering of the numbers [51, 52, 37]. However, the
connection between the order of the number concepts, and the actions of put
and take, will support counting, sum and subtraction.

To estimate whether the embedding structure is approximately one-dimensional
and linear in higher dimensions we computed the one-dimensional linear
approximation to the embedding line, and measured the average distortion of
using such approximation for representing the points. More in detail, we first
defined a mean-centered embedding matrix with M points and N dimensions, each
point corresponding to the embedding of an image. We then computed the best
rank 1 approximation to the data matrix by computing its singular value
decomposition (SVD) and zeroing all the singular values beyond the first one.
If the embedding is near linear, this rank 1 approximation should be quite
similar to the original matrix. To quantify the difference between the
original matrix and the approximation, we calculated the element-wise residual
(the Frobenius norm of the difference between the original matrix and the
approximation), then computed the ratio of the Frobenius norm of the residual
matrix and the Frobenius norm of the original matrix. The nearer the ratio is
to 0, the smaller the residual, and the better the rank 1 approximation. We
call this ratio the linear approximation error, we show this error compared to
some embeddings in Figure S8. We computed the embedding for dimensions 8, 16,
64, and 256, (one experiment each) and found ratios of 0.702%, 2.23%, 2.77%,
and 2.24%, suggesting that they are close to linear.

### Estimating relative quantity

We can use the perceived numerosity to reproduce a common task performed in
human psychophysics. Subjects are asked to compare a reference image to a test
image and respond in a two-alternative-forced choice paradigm with ‘more’ or
‘less’. We perform the same task using the magnitude of the embedding as the
fiducial signal. The model responds with more if the embedding of the test
image has a larger perceived numerosity than the reference image. The
psychometric curves generated by our model are presented in Figure 5 and match
qualitatively the available psychophysics [28, 31].

### Estimating absolute quantity

As noted above, the clusters are spaced regularly along a line and the points
in the embedding are ordered by the number of objects in the corresponding
images (Fig. S6). We postulate that the number of objects in an image is
proportional to the distance of that image’s embedding from the embedding of
the empty image. Given the linear structure, any one of the embedding
features, or their sum, may be used to estimate the position along the
embedding line. In order to produce an estimate we use the embedding of the
“zero” cluster as the origin. The zero cluster is special, and may be detected
as such without supervision, because all its images are identical and thus it
collapses to a point. The distance between “zero” and “one”, computed as the
pairwise distance between points belonging to the corresponding clusters,
provides a natural yardstick. This value, also learned without further
supervision, can be used as a unit distance to to interpret the signal between
0 and n. This estimate of numerosity is shown in Figure 6 against the actual
number of objects in the image. We draw two conclusions from this plot. First,
our unsupervised model allows an estimate of numerosity that is quite
accurate, within 10-15% of the actual number of objects. Second, the model
produces a systematic underestimate, similar to what is observed
psychophysically in human subjects [30].

### Controlling for spurious correlates of ‘number’

One may be concerned that image properties other than the abstraction of
‘object number’ may be driving the quantity estimate. Indeed, confound
variables , such as the count of pixels that are not black, are correlated
with object number and might play a role in the model’s ability to estimate
the number of objects in the scene.

We controlled for this hypothesis by exploiting the natural variability of our
test set images. We postulated that overall brightness, the area of the
envelope of the objects in the image, and total pixel area of the objects
would be good candidates as confound variables that might affect estimation.

To explore this possibility we analyzed close-call relative estimate tasks
(e.g. 16 vs 18 objects), where errors are frequent both for our model and for
human subjects, and, while holding the numbers of objects constant in the two
scenes, we studied the behavior of error rates as a function of fluctuations
in the confound variables. One would expect more errors when comparing image
pairs where quantities that typically correlate with the number of objects are
anticorrelated in the specific example (Fig. S4). Conversely, one would expect
lower error rates when the confound variables are positively correlated with
number.

In Fig. S3 we plotted error rates vs each one of the confound variables. As is
clear from the figure, we could not find large systematic biases even for
extreme variations in the confound variables. In conclusion, we are unable to
construct a convincing argument that any of the confound variables we studied
is strongly implicated in the estimate of quantity.

## Acknowledgements

The California Institute of Technology and the Simons Foundation (Global Brain
grant 543025 to PP) generously supported this work. We are very grateful to a
number of colleagues who provided references to the literature and insightful
suggestions: Alessandro Achille, Katie Bouman, David Burr, Eli Cole, Jay
MacClelland, Markus Meister, Mario Perona, Giovanni Paolini, Stefano Soatto,
Alberto Testolin, Kate Stevenson, Doris Tsao, and Yisong Yue.

## References

  * [1] Fei Xu, Elizabeth S Spelke, and Sydney Goddard.  Number sense in human infants.  Developmental science, 8(1):88–101, 2005.
  * [2] Stanislas Dehaene.  The number sense: How the mind creates mathematics.  OUP USA, 2011.
  * [3] Pooja Viswanathan and Andreas Nieder.  Neuronal correlates of a visual “sense of number” in primate parietal and prefrontal cortices.  Proceedings of the National Academy of Sciences, 110(27):11187–11192, 2013.
  * [4] Peter Gordon.  Numerical cognition without words: Evidence from Amazonia.  Science, 306(5695):496–499, 2004.
  * [5] Pierre Pica, Cathy Lemer, Véronique Izard, and Stanislas Dehaene.  Exact and approximate arithmetic in an Amazonian indigene group.  Science, 306(5695):499–503, 2004.
  * [6] Stanislas Dehaene, Elizabeth Spelke, Philippe Pinel, Ruxandra Stanescu, and Sanna Tsivkin.  Sources of mathematical thinking: Behavioral and brain-imaging evidence.  Science, 284(5416):970–974, 1999.
  * [7] Ben M Harvey, Barrie P Klein, Natalia Petridou, and Serge O Dumoulin.  Topographic representation of numerosity in the human parietal cortex.  Science, 341(6150):1123–1126, 2013.
  * [8] Mengting Fang, Zhenglong Zhou, Sharon Chen, and Jay McClelland.  Can a recurrent neural network learn to count things?  In CogSci, 2018.
  * [9] Silvester Sabathiel, James L McClelland, and Trygve Solstad.  Emerging representations for counting in a neural network agent interacting with a multimodal environment.  In Artificial Life Conference Proceedings, pages 736–743. MIT Press, 2020.
  * [10] Ivilin Stoianov and Marco Zorzi.  Emergence of a ‘visual number sense’ in hierarchical generative models.  Nature neuroscience, 15(2):194–196, 2012.
  * [11] Marco Zorzi and Alberto Testolin.  An emergentist perspective on the origin of number sense.  Philosophical Transactions of the Royal Society B: Biological Sciences, 373(1740):20170043, 2018.
  * [12] Khaled Nasr, Pooja Viswanathan, and Andreas Nieder.  Number detectors spontaneously emerge in a deep neural network designed for visual object recognition.  Science advances, 5(5):eaav7903, 2019.
  * [13] Gwangsu Kim, Jaeson Jang, Seungdae Baek, Min Song, and Se-Bum Paik.  Visual number sense in untrained deep neural networks.  Science Advances, 7(1):eabd6127, 2021.
  * [14] W Stanley Jevons.  The power of numerical discrimination.  Nature, 3:281–282, 1871.
  * [15] Manuela Piazza, Andrea Mechelli, Brian Butterworth, and Cathy J Price.  Are subitizing and counting implemented as separate or functionally overlapping processes?  Neuroimage, 15(2):435–446, 2002.
  * [16] Yosef Singer, Yayoi Teramoto, Ben DB Willmore, Jan WH Schnupp, Andrew J King, and Nicol S Harper.  Sensory cortex is optimized for prediction of future input.  Elife, 7:e31557, 2018.
  * [17] Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah.  Signature verification using a “siamese” time delay neural network.  In Advances in neural information processing systems, pages 737–744, 1994.
  * [18] David H Hubel and Torsten N Wiesel.  Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex.  The Journal of physiology, 160(1):106, 1962.
  * [19] Rüdiger Von der Heydt, Esther Peterhans, and Gunter Baumgartner.  Illusory contours and cortical neuron responses.  Science, 224(4654):1260–1262, 1984.
  * [20] Doris Y Tsao, Winrich A Freiwald, Roger BH Tootell, and Margaret S Livingstone.  A cortical region consisting entirely of face-selective cells.  Science, 311(5761):670–674, 2006.
  * [21] Doris Y Tsao, Winrich A Freiwald, Tamara A Knutsen, Joseph B Mandeville, and Roger BH Tootell.  Faces and objects in macaque cerebral cortex.  Nature neuroscience, 6(9):989–995, 2003.
  * [22] Chou P Hung, Gabriel Kreiman, Tomaso Poggio, and James J DiCarlo.  Fast readout of object identity from macaque inferior temporal cortex.  Science, 310(5749):863–866, 2005.
  * [23] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.  Gradient-based learning applied to document recognition.  Proceedings of the IEEE, 86(11):2278–2324, 1998.
  * [24] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  In Advances in neural information processing systems, pages 1097–1105, 2012.
  * [25] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.  Deep learning.  Nature, 521(7553):436–444, 2015.
  * [26] Jamie D Roitman, Elizabeth M Brannon, and Michael L Platt.  Monotonic coding of numerosity in macaque lateral intraparietal area.  PLoS Biol, 5(8):e208, 2007.
  * [27] Max Wertheimer.  Laws of organization in perceptual forms. Kegan Paul, Trench, Trubner & Company, 1938.
  * [28] David Burr and John Ross.  A visual sense of number.  Current biology, 18(6):425–428, 2008.
  * [29] Paula A Maldonado Moscoso, Guido M Cicchini, Roberto Arrighi, and David C Burr.  Adaptation to hand-tapping affects sensory processing of numerosity directly: evidence from reaction times and confidence.  Proceedings of the Royal Society B, 287(1927):20200801, 2020.
  * [30] Véronique Izard and Stanislas Dehaene.  Calibrating the mental number line.  Cognition, 106(3):1221–1247, 2008.
  * [31] Lester E Krueger.  Single judgments of numerosity.  Perception & Psychophysics, 31(2):175–182, 1982.
  * [32] Lisa Feigenson, Stanislas Dehaene, and Elizabeth Spelke.  Core systems of number.  Trends in cognitive sciences, 8(7):307–314, 2004.
  * [33] Stanislas Dehaene.  Origins of mathematical intuitions: The case of arithmetic.  Annals of the New York Academy of Sciences, 1156(1):232–259, 2009\.
  * [34] David Burr, Giovanni Anobile, and Marco Turi.  Adaptation affects both high and low (subitized) numbers under conditions of high attentional load.  Seeing and Perceiving, 24(2):141–150, 2011.
  * [35] Frank Restle.  Speed of adding and comparing numbers.  Journal of Experimental Psychology, 83(2p1):274, 1970.
  * [36] Stanislas Dehaene, Serge Bossini, and Pascal Giraux.  The mental representation of parity and number magnitude.  Journal of experimental psychology: General, 122(3):371, 1993.
  * [37] Stanislas Dehaene, Nicolas Molko, Laurent Cohen, and Anna J Wilson.  Arithmetic and the brain.  Current opinion in neurobiology, 14(2):218–224, 2004.
  * [38] Rosa Rugani, Giorgio Vallortigara, Konstantinos Priftis, and Lucia Regolin.  Number-space mapping in the newborn chick resembles humans’ mental number line.  Science, 347(6221):534–536, 2015.
  * [39] Andrew Trask, Felix Hill, Scott Reed, Jack Rae, Chris Dyer, and Phil Blunsom.  Neural arithmetic logic units.  arXiv preprint arXiv:1808.00508, 2018.
  * [40] Samuel J Cheyette and Steven T Piantadosi.  A unified account of numerosity perception.  Nature Human Behaviour, pages 1–8, 2020.
  * [41] Tom Verguts and Wim Fias.  Representation of number in animals and humans: A neural model.  Journal of cognitive neuroscience, 16(9):1493–1504, 2004.
  * [42] Andreas Nieder.  The neuronal code for number.  Nature Reviews Neuroscience, 17(6):366, 2016.
  * [43] Marie Amalric, Isabelle Denghien, and Stanislas Dehaene.  On the role of visual experience in mathematical development: Evidence from blind mathematicians.  Developmental cognitive neuroscience, 30:314–323, 2018.
  * [44] Virginie Crollen and Olivier Collignon.  How visual is the number sense? Insights from the blind.  Neuroscience & Biobehavioral Reviews, 2020.
  * [45] Giovanni Anobile, Roberto Arrighi, Elisa Castaldi, and David C Burr.  A sensorimotor numerosity system.  Trends in Cognitive Sciences, 2020.
  * [46] Raia Hadsell, Sumit Chopra, and Yann LeCun.  Dimensionality reduction by learning an invariant mapping.  In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pages 1735–1742. IEEE, 2006.
  * [47] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  * [48] Alberto Testolin, Serena Dolfi, Mathijs Rochus, and Marco Zorzi.  Visual sense of number vs. sense of magnitude in humans and machines.  Scientific reports, 10(1):1–13, 2020.
  * [49] Leland McInnes, John Healy, and Steve Astels.  hdbscan: Hierarchical density based clustering.  Journal of Open Source Software, 2(11):205, 2017.
  * [50] Ricardo JGB Campello, Davoud Moulavi, and Jörg Sander.  Density-based clustering based on hierarchical density estimates.  In Pacific-Asia conference on knowledge discovery and data mining, pages 160–172. Springer, 2013.
  * [51] Stanislas Dehaene and Laurent Cohen.  Towards an anatomical and functional model of number processing.  Mathematical cognition, 1(1):83–120, 1995.
  * [52] Marco Zorzi, Konstantinos Priftis, and Carlo Umiltà.  Neglect disrupts the mental number line.  Nature, 417(6885):138–139, 2002.

## Supplemental Figures

List of Supplementary Figures S1 Detailed diagram of the network structure.
(A) The feature extraction / embedding network. The gray layers are pre-
trained on ImageNet [47, 24] and remain fixed throughout the course of
training. The orange layers are randomly seeded and trained simultaneously
with the classifier in (B). The details of the layer are described within the
brackets. For example, [11x11 - s4, 64] is an 11x11 kernel with a stride of 4
and 64 filters. During a training step, the embedding network accepts an image
($x_{t}$) of the visual scene and generates a lower-dimensional feature
embedding ($z_{t}$) of the visual scene. An action: (P), (T), or (S) modifies
the visual scene and the “after” image ($x_{t+1}$) is passed through the
embedding network as well. The outputs of the embedding network, ($z_{t}$) and
($z_{t+1}$) are treated as inputs to the action classification network. The
shared embedding network is trained together with the classifier (B), in a
Siamese configuration. (B) The action classification network is a 2-layer
classifier network and is composed of two fully connected layers with a log-
softmax transformation on the output. The input is the representation of the
visual scene before and after an action is performed. The negative log-
likelihood (NLL) loss function is used to train both the action classification
network and the embedding network simultaneously. An overview of the entire
training paradigm is shown in Figure 2.

(a)

(b)

List of Supplementary Figures S2 Training set statistics. (A) In dataset 1
(Fig. 1A) objects have the same size and contrast. Thus, the number of objects
predicts the mean image intensity and vice-versa. (B) Objects in dataset 2
(Fig. 1B) have variable sizes and variable contrast, thus mean image intensity
is not sufficient to predict the number of objects. List of Supplementary
Figures S3 Effects of covariates of numerosity. Three covariates of the number
of objects in the scene are explored for possible influence on our model’s
estimate of numerosity. These are average image intensity (left column), the
sum of the areas of the objects (middle column), and the area of the objects’
convex hull (right column). Each plot shows the error rates in a relative
quantity discrimination task like the one in Figure 5. For each plot we chose
a reference image containing respectively 3, 9, 16 and 24 objects (rows of the
figure) and had our model judge relative numerosity w.r. to test images
containing a different but similar number of objects (indicated in the legend
and associated with colors). We used 4650 test images, 150 images per test
number per condition. Given the stochastic nature of the images, the
covariates vary over a wide range for each number of objects (see Fig.S4). For
each number of objects, we plot the model’s error rates (y axis) as a function
of the value of the covariate quantity (x axis) which is expressed as
fractional difference from the reference image (the values are binned).
Shadows display 95% Bayesian confidence intervals ($N>100$). Horizontal error
lines indicate no correlation of numerosity estimation with the covariate
quantity. A few lines have slopes that differ slightly from zero indicating a
possible correlation. However, some of the slopes indicate a negative
correlation (i.e. the better the signal, the higher the error rate). From this
evidence it is difficult to conclude that that the model is exploiting
anything but ‘number’ to estimate numerosity.  List of Supplementary Figures
S4 Sample images where covariates are anticorrelated with number. Examples of
test images used in the experiments shown in Figure S3. In these examples the
change in numerosity is anticorrelated with the three covariates we study (one
covariate per row). The number below each plot shows the fractional difference
from the value of the covariate in the reference image (center column). For
example, in the top left, there is a 37.1% increase in average image intensity
when compared to the intensity in the reference image (center column). In the
last row, the scene with 18 objects has a smaller convex hull than the
corresponding scenes with 14 and 16 objects. Our model correctly classifies
all these examples in relative numerosity judgements (left column compared to
center and right column compared to center). For each row, from the lowest
numerosity to the highest, the model predicts a perceived numerosity of 12.65,
14.91, and 15.61 (Intensity); 13.66, 13.87, 15.68 (Area); 12.65, 14.88, 15.92
(Convex Hull). As explained in Figure 5, perceived numerosity generally
slightly underestimates the true numerosity and is not necessarily an integer
value. List of Supplementary Figures S5 The embedding dimension effects on
error. Classification errors for Model 2, averaged over the number of items in
the scene (0 - 3) are plotted as a function of the dimension of the embedding
(a free parameter in our model). Since the effect is minimal we arbitrarily
picked a dimension of two for ease of visualization (Figs. 4B, S6). The
shadows show 95% Bayesian confidence intervals (282 $\leq$ N $\leq$ 355).

(a)

(b)

(c)

(d)

List of Supplementary Figures S6 Embeddings with topology for Model 1 and
Model 2. A close-up look at the embedding space within the training limit. The
left side are plots from Model 1 and the right side from Model 2. (A), (B)
Unsupervised clustering is performed on the embedding space. Each embedding is
colored by it’s cluster. Each cluster A0 - D0 correspond to images with
numerosites 0 - 3. The clusters are well-separated. The “zero” clusters, for
both Model 1 and Model 2, are immediately recognizable as they have no
variance (orange dot). As numerosity increases, Model 1 clusters remain well-
separated, whereas Model 2 clusters begin to come closer to each other. We
also overlay a topology from the training actions (P), (T), (S). Blue arrows
joining a pair of points represent take actions, red arrows represent put
actions. Arrows representing shake actions are under the point clouds and are
mostly not visible. (C), (D) Distances between pairs of points in the
embedding space are histogrammed by action. The histograms show the clearly
different distribution for shake actions in comparison to take and put
actions. Furthermore, the overlap between shake and non-shake actions is
smaller for Model 1 than Model 2, explaining the higher performance in action
classification for Model 1. (a) List of Supplementary Figures S7 Effect of
modifying the training limit. In order to explore the effect of the number of
objects during training, we trained the network to predict actions using a
maximum of 3, 5, or 8 objects with images like those in dataset 2 (1B). We
tested the network on 8 objects. Each panels shows errors on the training task
and are in the same style as Figure 3. The line-breaks and dashed lines mark
where the training limit ends and the testing region begins, and the legend
shows the training limit in parentheses. The shadows provide 95% confidence
intervals (267 $\leq$ N $\leq$ 355). As expected, the error is lower when the
training limit is higher.

(a)

(b)

(c)

List of Supplementary Figures S8 Miscellaneous embedding spaces (A) Embedding
space for the network trained on dataset 2, with up to five objects. (B)
Embedding space for the network trained on dataset 2, with up to eight
objects. (C) Embedding spaces for 30 different random initializations. We
repeated the training procedure 30 times on different random initializations
of dataset 2, with a training limit of 3 objects. Qualitatively, 21 embedding
spaces look like a straight line, three initializations present a slight kink
in the line, and six instances either present a large kink or two kinks. The
linear approximation error (Methods - Interpreting the Embedding Space) is
provided above each subplot and measures the approximate deviation from a
purely linear model. An error below 4% predicts an approximately linear
embedding line.


# Cyclic Frobenius algebras.

V. M. Buchstaber$, A. V. Mikhailov♮ $ Steklov Mathematical Institute, Moscow,
Department of Mechanics and Mathematics, Moscow State University, Russia
<EMAIL_ADDRESS>♮ University of Leeds, Leeds, UK<EMAIL_ADDRESS>
AVM acknowledges support from EPSRC Grant EP/V050451/1.

Let $\mathscr{A}$ be an associative $\mathbb{C}$-algebra with identity 1, and
let $\mathscr{M}$ be some $\mathbb{C}$-linear space, $\dim\mathscr{M}\geqslant
1$.

######  Definition.

A cyclic Frobenius algebra ($\rm{CF}$-algebra) $\mathscr{A}$ is an algebra
$\mathscr{A}$ with a $\mathbb{C}$-bilinear skew-symmetric form
$\Phi(\cdot,\cdot)\colon\mathscr{A}\otimes_{\mathbb{C}}\mathscr{A}\to\mathscr{M}$
such that $\Phi(A,BC)+\Phi(B,CA)+\Phi(C,AB)=0$ where $A,B,C\in\mathscr{A}$.

Examples: 1) $\mathscr{M}=\mathscr{A}$ and $\Phi(A,B)=AB-BA=[A,B]$; 2)
$\mathscr{A}$ is a commutative algebra with Poisson bracket
$\\{\cdot,\cdot\\}$, $\mathscr{M}=\mathscr{A}$ and $\Phi(A,B)=\\{A,B\\}$; 3)
$\rm{CF}$-algebras, where $\mathscr{M}$ is a field, obtained from the
constructions of algebras from [1], [5].

Let us introduce the $\mathbb{C}$-linear subspace
$\rm{Span}\subset\mathscr{A}$, spanned by all commutators
$[A,B]\in\mathscr{A}$, and the projection
$\pi\colon\mathscr{A}\to\mathscr{A}/\rm{Span}$. Set $A\approx B$ if
$A-B\in\rm{Span}$.

######  Lemma 1.

Let $\varphi(\cdot,\cdot)\colon\mathscr{A}\otimes\mathscr{A}\to\mathscr{A}$ be
a $\mathbb{C}$-bilinear skew-symmetric form such that
$\varphi(A,BC)\approx\varphi(A,B)C+B\varphi(A,C)$. Then $\mathscr{A}$ is a
$\rm{CF}$-algebra with $\mathscr{M}=\mathscr{A}/\rm{Span}$ and
$\Phi=\pi\varphi$.

Let $\mathscr{K}=\\{B\in\mathscr{A}\,:\,\Phi(A,B)=0,\text{ for
all}\,A\in\mathscr{A}\\}$. From Definition Definition it follows that
$\mathscr{K}$ is the subring in $\mathscr{A}$, $1\in\mathscr{K}$, and
$\Phi(A,bC)=\Phi(Ab,C)$ for all $A,C\in\mathscr{A}$ and $b\in\mathscr{K}$.

Let ${\mathfrak{A}}=\mathbb{C}\langle
u_{0},u_{1},\ldots\rangle=\oplus{\mathfrak{A}}_{k},\,k\geqslant 0$, be a free
associative graded algebra with derivation $D$, $|u_{k}|=k+2$,
$D(u_{k})=u_{k+1},\,k=0,1,\ldots$ . We introduce the algebra
${\mathfrak{A}}^{D}=\\{A=\sum_{i\leqslant m}a_{i}D^{i},\,a_{m}\neq
0,\,m\in\mathbb{Z},\,a_{i}\in{\mathfrak{A}}_{|a_{m}|+m-i}\\}$, where
$[D,u_{k}]=u_{k+1},\;[D^{-1},u_{k}]=\sum_{i\geqslant 1}(-1)^{i}\\\
u_{k+i}D^{-i-1}$. Let $A_{+}+A_{-}=A\in{\mathfrak{A}}^{D}$, where $A_{+}=\sum
a_{i}D^{i},\,0\leqslant i\leqslant m$, for $m\geqslant 0$, and $A_{+}=0$, for
$m<0$. We have ${\rm res}[D,A]=D({\rm res}A)$, where ${\rm res}A=a_{-1}$.

We introduce a homogeneous $\mathbb{C}$-bilinear skew-symmetric form
$\sigma(\cdot,\cdot)\colon{\mathfrak{A}}^{D}\otimes{\mathfrak{A}}^{D}\to{\mathfrak{A}}$,
$|\sigma(A,B)|=|A|+|B|$, by

$\sigma(aD^{n},bD^{m})\\!=\\!\begin{cases}\frac{1}{2}\binom{n}{n+m+1}\\!\sum\limits_{s=0}^{n+m}\\!(-1)^{s}\big{(}a^{(s)}b^{(n+m-s)}\\!+b^{(n+m-s)}a^{(s)}\big{)},\text{for
}n+m\geqslant 0,\,nm<0,\\\\[1.0pt] 0,\qquad\mbox{otherwise}.\end{cases}$

######  Lemma 2.

For any $A,B\in{\mathfrak{A}}^{D}$, we have $D\big{(}\sigma(A,B)\big{)}={\rm
res}\,[A,B]-\text{$\Delta$}(A,B)$, where

$\Delta(aD^{n},bD^{m})=\begin{cases}\frac{1}{2}\binom{n}{n+m+1}\Big{(}[a,b^{(n+m+1)}]+(-1)^{n+m}[b,a^{(n+m+1)}]\Big{)},\text{
for }n+m\geqslant-1,\\\\[1.0pt] 0,\qquad\mbox{otherwise}.\end{cases}$

######  Lemma 3.

Under the canonical projection
$\pi\colon{\mathfrak{A}}\to{\mathfrak{A}}/\rm{Span}=\mathscr{M}=\oplus\mathscr{M}_{k},\;k\geqslant
0$, the operator $D$ gives monomorphisms
$\overline{D}\colon\mathscr{M}_{k}\to\mathscr{M}_{k+1},\;k>0$.

######  Theorem 1.

The algebra ${\mathfrak{A}}^{D}$ is a $\rm{CF}$-algebra with the form
$\Phi=\pi\sigma\colon{\mathfrak{A}}^{D}\otimes{\mathfrak{A}}^{D}\to\mathscr{M}$
such that $\mathscr{K}={\mathfrak{A}}$ and
$\Phi(D^{n},D^{-n})=n,\;n\in\mathbb{Z}$.

######  Corollary 1.

On the set of generators $\\{D^{n},\,n\in\mathbb{Z}\\}$ of the free left
${\mathfrak{A}}$-module ${\mathfrak{A}}^{D}$, the form $\Phi$ is non-
degenerate.

Let $L=D^{2}-u$ and $\mathscr{L}=D+\sum I_{i}D^{-i},\;i\geqslant
1,\,I_{i}\in{\mathfrak{A}}_{i+1}$, where $\mathscr{L}^{2}=L$. We obtain a
sequence of series $\mathscr{L}^{2k-1},\,k\in\mathbb{N}$. Set

$\sigma_{2k-1,2n-1}=\sigma(\mathscr{L}_{+}^{2k-1},\mathscr{L}^{2n-1})\in{\mathfrak{A}}_{2n+2k-2},\qquad\rho_{2n}=\sigma_{1,2n-1}={\rm
res}\,\mathscr{L}^{2n-1},\,n>0.$ (1)

From the properties of the form $\sigma(\cdot,\cdot)$, we obtain
$\sigma_{2k-1,2n-1}=\sigma_{2n-1,2k-1},\;k,n\in\mathbb{N}$.

Let us introduce derivations $D_{2k-1},\,k\in\mathbb{N}$, of the algebra
${\mathfrak{A}}$ such that

$D_{1}=D;\quad[D,D_{2k-1}]=0;\quad
D_{2k-1}(u)=-2D(\rho_{2k}),\;k\in\mathbb{N}.$ (2)

######  Corollary 2.

$[D_{2k-1},D_{2n-1}]=0$.

Let $u=u(t_{1},t_{3},\ldots)$. Set $\partial_{t_{2k-1}}(u)=D_{2k-1}(u)$.

######  Theorem 2.

The system of equations
$\partial_{t_{2k-1}}(u)=-2D\big{(}\rho_{2k}(u)\big{)},\,k\in\mathbb{N}$, where
$\rho_{2}(u)=-\frac{1}{2}u$, coincides with the KdV hierarchy on
${\mathfrak{A}}$

$4\partial_{t_{3}}(u)=D(u_{2}-3u^{2});\quad
16\partial_{t_{5}}(u)=D(u_{4}-5u_{2}u-5uu_{2}-5u_{1}^{2}+10u^{3});\,\ldots\,.$

The proof follows from the above constructions (cf. [2]–[6]).

Following [3], for $N\in\mathbb{N}$, set
$F_{2N+2}(u)=\rho_{2N+2}+\sum_{k=0}^{N-1}\alpha_{2(N-k+1)}\rho_{2k}$, where
$\rho_{0}=1$ and $\alpha_{4},\ldots,\alpha_{2N+2}$ are free parameters.
Equation $F_{2N+2}(u)=0$ is called the $N$-th Novikov equation. Let $J(F)$ be
the two-sided $D$-differential ideal in ${\mathfrak{A}}$ generated by the
polynomial $F_{2N+2}(u)$. Since
$2^{2k+1}\rho_{2k+2}=u_{2k}+f(u_{2k-2},\ldots,u)$ and $u_{k+1}=D^{k}(u)$, on
the factor-algebra ${\mathfrak{A}}\diagup J(F)=\mathbb{C}\langle
u,\ldots,u_{2N-1}\rangle$, the KdV hierarchy (see Theorem 2) reduces to the
$N$-th Novikov hierarchy, where the first system represents the $N$-th Novikov
equation in the form $D(u_{k})=u_{k+1}$ for $0\leqslant k<2N-1,\
D(u_{2N-1})=-f(u_{2k-2},\ldots,u)$. It was shown in [3] that, in terms of the
form $\Phi$, (see Theorem 1), the polynomials

$H_{2n+1,2N+1}=\sigma_{2n+1,2N+1}+\sum_{k=1}^{N-1}\alpha_{2(N-k+1)}\sigma_{2n+1,2k-1},\;n=1,\ldots,N,$
(3)

define first integrals $\widehat{H}_{2n+1,2N+1}=\pi(H_{2n+1,2N+1})$ of the
$N$-th Novikov hierarchy:

$\pi(\partial_{t_{2k-1}}(\widehat{H}_{2n+1,2N+1}))=0,\qquad
k,n\in\\{1,\ldots,N\\}.$

######  Theorem 3.

In the quantum case, the $N$-th Novikov hierarchy (see [3]) is written in the
form of compatible systems of Heisenberg equations. The polynomials (3) are
quantum commuting Hamiltonians of the hierarchy and are self-adjoint if the
variables $u_{k}$ are self-adjoint and the values of the parameters
$\alpha_{4},\ldots,\alpha_{2N+2}$ are real.

We would like to thank S.P. Novikov and V.N. Rubtsov for stimulative
discussions of the results of this paper.

## References

  * [1] M. Aguiar, _On the associative analog of Lie bialgebras._ , J. Algebra 244:2 (2001), 492–532.
  * [2] V. M. Buchstaber, A. V. Mikhailov, _Integrable polynomial Hamiltonian systems and symmetric powers of plane algebraic curves._ , Russian Math. Surv., 76:4(460)(2021), 587–652.
  * [3] V. M. Buchstaber, A. V. Mikhailov, _KdV hierarchies and quantum Novikov’s equations._ , arXiv:2109.06357 v2 [nlin.SI].
  * [4] I. M. Gelfand, L. A. Dikii, _Asymptotic benaviour of the resolvent of Sturm-Liouville equations and the algebra of the Korteweg-de Vries equations._ , Russian Math. Surveys, 30:5 (1975), 77–113.
  * [5] A. V. Odesskii, V. N. Rubtsov, V. V. Sokolov, _Bi-Hamiltonian ordinary differential equations with matrix variables ._ , Theor. Math. Phys., 171:1 (2012), 26–32.
  * [6] V. V. Sokolov, _Algebraic Structures in Integrability._ , World Sci. Publ., Hackensack, NJ, 2020.