# The Jones polynomial of collections of open curves in $3-$space

Kasturi Barkataki 1 Eleni Panagiotou2

(1Department of Mathematics and SimCenter, University of Tennessee at
Chattanooga, $613$ McCallie Ave, Chattanooga, $37403$, TN, USA.
2Department of Mathematics and SimCenter, University of Tennessee at
Chattanooga, $613$ McCallie Ave, Chattanooga, $37403$, TN, USA.
)

###### Abstract

Measuring the entanglement complexity of collections of open curves in 3-space
has been an intractable, yet pressing mathematical problem, relevant to a
plethora of physical systems, such as in polymers and biopolymers. In this
manuscript, we give a novel definition of the Jones polynomial that
generalizes the classic Jones polynomial to collections of open curves in
3-space. More precisely, first we provide a novel definition of the Jones
polynomial of linkoids (open link diagrams) and show that this is a well-
defined single variable polynomial that is a topological invariant, which, for
link-type linkoids, it coincides with that of the corresponding link. Using
the framework introduced in [14], this enables us to define the Jones
polynomial of collections of open and closed curves in 3-space. For
collections of open curves in 3-space, the Jones polynomial has real
coefficients and it is a continuous function of the curves’ coordinates. As
the endpoints of the curves tend to coincide, the Jones polynomial tends to
that of the resultant link. We demonstrate with numerical examples that the
novel Jones polynomial enables us to characterize the topological/geometrical
complexity of collections of open curves in 3-space for the first time.

##  1 Introduction

Many physical systems, such as polymers and biopolymers, textiles, and
chemical compounds are composed by filamentous structures, that can be modeled
by mathematical curves in 3-space, whose entanglement complexity determines
their mechanical properties and function [1, 21, 24, 23, 20, 18, 11, 3].
Measuring multi-chain entanglement in such systems has remained an open
problem for many decades [3, 4, 2]. In this paper we introduce the first
rigorous measure of complexity of collections of open curves in 3-space, via a
traditional invariant of knots and links, the Jones polynomial. More
precisely, the novel Jones polynomial that we introduce here generalizes the
traditional Jones polynomial, so that it is applicable to collections of open
curves in 3-space and gives a continuous measure of linking complexity which
reduces to a topological invariant for closed curves.

Collections of simple closed curves in 3-space (links) can be classified upon
deformations without allowing cutting and pasting (topological equivalence).
Topological invariants are functions defined on links that are invariant under
Reidemeister moves and can be used to characterize the complexity of simple
closed curves in 3-space. The notion of topological equivalence however is not
useful for systems of open curves in 3-space, since any collection of open
curves is topologically equivalent to any other. Instead of topological
invariants, to characterize the topological complexity of open curves in
3-space, we seek measures of topological complexity that are continuous
functions in the space of configurations. Until recently, the only measure of
topological entanglement that could be applied to one or two open curves in
3-space to give a continuous measure of single or pairwise topological
complexity, was the Gauss linking integral [6, 19, 17, 16, 13]. It was not
until [14], where the necessary framework to define the Jones polynomial of a
single open curve in 3-space was introduced, based on the notion of knotoids
(open ended knot diagrams) and their Jones polynomial. The Jones polynomial of
an open curve is a polynomial with real coefficients that are continuous
functions of the curve coordinates. This new framework also allowed to define
Vassiliev measures of open curves in 3-space and to derive closed formulas for
the second Vassiliev measure of single open curves in 3-space [15]. These
advances led to immediate applications in materials and biology to obtain
novel understanding of such physical systems, rigorously and without any
closure scheme for the first time [10, 22, 26]. However, extending the Jones
polynomial to collections of open curves in 3-space, has not been possible,
even though one would think it would be straightforward, as the definition of
the classical Jones polynomial applies to both knots and links. The reason for
this is that an appropriate definition of the Jones polynomial of linkoids
(open ended link diagrams) is missing. This will be fully addressed in this
manuscript.

The theory of knotoids was introduced by V.Turaev [25] as a means to study
parts of knot diagrams with the aim to characterize knot complexity. Knotoids
on a surface are open-ended knot diagrams and many analogous ideas of
classical knot theory are applicable to the study of knotoids [7, 9, 8, 12].
The Jones polynomial of knotoids follows from that of knots with a simple
modification. The modification relies on assigning a value to states
containing an open arc in the bracket expansion. The notion of knotoids can be
naturally extended to multi-component cases, which we call linkoids (these can
be seen as open link diagrams). Even though the definition of the Jones
polynomial of knots extends naturally to that of links, the definition of the
Jones polynomial of linkoids does not follow directly from that of knotoids
and remains elusive. The difficulty consists on the fact that the states
resulting from linkoids may contain several non-intersecting open arcs, which
may connect end-points of different components, and it has been unclear how to
deal with those in the polynomial. Very recently, in [5] two definitions of
the Jones polynomial were introduced to classify linkoids. Those definitions
however, do not satisfy an important property, which prevents them to be used
in giving an appropriate definition of the Jones polynomial of open curves in
3-space, as we will see later. In this manuscript, we will provide a novel
definition of the bracket polynomial that can properly account for those
states and which satisfies an important desired property, which enables the
definition of a single variable Jones polynomial of linkoids and the Jones
polynomial of collections of open curves in 3-space.

More precisely, in this paper we provide the first rigorous definition of the
Jones polynomial of linkoids which has the property that the Jones polynomial
of a link-type linkoid is equal to the Jones polynomial of the corresponding
link. We use the framework introduced in [14] to define the Jones polynomial
of a collection of open curves in 3-space [14]. Namely, any projection of a
collection of open curves in 3-space to a plane corresponds to a linkoid
diagram, whose Jones polynomial is now well defined. We define the Jones
polynomial of the collection of open curves in 3-space as the average of the
Jones polynomials in a projection over all possible projection directions. We
prove that for a collection of open curves in 3-space, the newly introduced
Jones polynomial is a polynomial with real coefficients which are continuous
functions of the curve coordinates. We also prove that, as the endpoints of
the curves move to coincide and form a link in 3-space, the Jones polynomial
of the collection of open curves tends to that of the resulting link. We
stress that the latter is possible only via the Jones polynomial of linkoids
definition we introduce in this manuscript. The Jones polynomial introduced
here provides a holistic definition of a Jones polynomial which applies to
open and closed curves of single or multi-component systems. Our theoretical
results are accompanied by an illustrative example of an open Borromean ring
in 3-space.

The contents of this paper are summarised as follows: In Section 2, we give
the definitions for linkoid diagrams and linkoids and also make precise the
notion of link-type linkoids. In Section 3, we provide all the necessary
framework and then define the Jones polynomial of linkoids and we study its
properties. In Section 4, we give the definition of the Jones polynomial of a
collection of open curves in 3-space and we study its properties. Finally, in
Section 5, we present the conclusions of our results.

##  2 Linkoids

As mentioned in the previous section, the Jones polynomial of collections of
open curves in 3-space will be defined via projections of the curves, which
can be seen as linkoids. Even though we think of linkoids as projections of
open curves, linkoids have been typically studied as diagrammatic objects. In
this section, we present the definitions of linkoids and linkoid diagrams,
originally defined in [7, 9, 8, 12], as well as the definition of link-type
linkoids.

######  Definition 2.1.

(Linkoid Diagram) Let $\Sigma$ be a surface. In this manuscript, we will
consider $\Sigma=S^{2}=\mathbb{R}^{2}\cup\infty$. A linkoid diagram $L$ with
$n\in\mathbb{N}$ components in $\Sigma$ is a generic immersion of
$[0,1]\times[0,1]\times\cdots\times[0,1]$ (upto $n$ times) in the interior of
$\Sigma$ whose only singularities are transversal double points endowed with
over/undercrossing data. These double points are called the crossings of $L$.
The immersion of each $[0,1]$ is referred to as a component of the linkoid
diagram and the images of $0$ and $1$ under this immersion are called the leg
and the head of the component, respectively. These two points are distinct
from each other and from the double points; they are called the endpoints of
the component. The diagram $L$ has a total of $2n$ endpoints. A natural
orientation is assigned for each $l_{i}$ from the leg to the head.

For a linkoid diagram $L$ with $n$ components, we may introduce a convention
to index all the legs by odd numbers $i\in\\{1,3,\cdots,2n-1\\}$, and the
corresponding heads by even numbers $i+1\in\\{2,4,\cdots,2n\\}$. Thus, a
component of $L$ is denoted by $l_{\left(2j-1,2j\right)}\quad$ such that
$j\in\\{1,2,\cdots,n\\}\quad$ where the index $\quad 2j-1\quad$ indicates the
leg and the index $\quad 2j\quad$ indicates the head of the component. See
Figure 1 for examples of labeled linkoids.

Figure 1: Examples of Knotoid and Linkoid Diagrams.

######  Definition 2.2.

(Linkoid) A linkoid is an equivalence class of the linkoid diagrams up to the
equivalence relation induced by the three Reidemeister moves and isotopy. The
three Reidemeister moves, denoted by $\Omega_{1},\Omega_{2},\Omega_{3}$, see
Figure 2(a), are defined on linkoid diagrams and referred to as
$\Omega$-moves. It is forbidden to pull the strand adjacent to an endpoint
over/under a transversal strand as shown in Figure 2(b). These moves are
called forbidden linkoid moves, and denoted by $\Phi_{+}$ and $\Phi_{-}$ ,
respectively.

Figure 2: The $\Omega$-moves and Forbidden moves on a Linkoid Diagram. (See
[7]). Note that these arcs are considered as parts of a larger diagram.

######  Definition 2.3.

(Link-type Linkoid) A linkoid is said to be of link-type if there exists a
diagram in its equivalence class in which it is possible to draw a closure arc
connecting the pair of endpoints (from the head to the leg) per component
without introducing additional double points (crossings) to the diagram or
between the closure arcs. A linkoid that is not of link-type, is called a
proper linkoid.

######  Remark 2.1.

For knotoids (linkoids with 1 component), another way to distinguish between
proper knotoids and knot-type knotoids is by checking whether there exists a
diagram in which the two endpoints lie in the same region. This definition is
consistent with Definition 3.3. Notice that, if for any diagram of the knotoid
it is impossible to draw a closure arc without introducing new crossings, it
follows that the endpoints must lie in different regions of the diagram. The
converse is also true.

######  Remark 2.2.

Note that any pure braid (In a pure braid, the beginning and the end of each
strand are in the same position) can be thought of as a link-type linkoid.
Indeed, the closure arcs of a pure braid do not intersect any other arc, as do
the closure arcs of a linkoid.

##  3 The Jones polynomial of Linkoids

In this Section we will define the Jones polynomial of linkoids. As we
mentioned in the Introduction, this will be later seen as the Jones polynomial
of a projection of a collection of open curves in 3-space. However, it is also
of interest in the study of linkoids in general. We will define the Jones
polynomial of linkoids as the normalized Kauffman bracket polynomial of
linkoids. Let us start with the classic smoothing skein relations, shown in
Equation 1.

$\langle\raisebox{-10.0pt}{\includegraphics[width=21.68231pt]{fig/cross10.png}}\rangle=A\langle\raisebox{-10.0pt}{\includegraphics[width=21.68231pt]{fig/cross30.png}}\rangle+A^{-1}\langle\raisebox{-10.0pt}{\includegraphics[width=21.68231pt]{fig/cross20.png}}\rangle,\hskip
14.22636pt\langle L\cup\bigcirc\rangle=(-A^{2}-A^{-2})\langle L\rangle.$ (1)

Upon recursively smoothing a linkoid diagram using these relations, we obtain
states with no crossings (see Figure 3).

######  Definition 3.1.

(State of a Linkoid Diagram) A state $S$ of a linkoid diagram $L$ with $n$
components is an assignment of a choice of smoothing at each crossing. This
results in a crossingless diagram with disjoint circles and $n$ long segments.
Each long segment is labelled by the two endpoints that it connects. See
Figure 3 for an illustrative example of a linkoid diagram and a corresponding
state.

Figure 3: (Left) A linkoid diagram with 3 components, $l_{(1,2)},l_{(3,4)}$
and $l_{(5,6)}$. (Right) One of the 64 possible states in the state sum
expansion of the diagram. Clearly, this state is crossingless and contains 3
disjoint segments, namely $(1,3),(2,6)$ and $(4,5)$. Note that even though the
number of long segments is the same as the number of components of the
original linkoid, there has been a rearrangement in the pairing of endpoints
per segment. Figure 4: An oriented link-type linkoid with 2 components,
corresponding to the Hopf link.

Example 1 : To motivate the relations that will enable us to evaluate the
bracket polynomial on these states, we will focus on a particular example,
that of the linkoid diagram in Figure 4. Note that this linkoid diagram
represents a link-type linkoid, corresponding to the Hopf link. It is natural
to require that the bracket (and consequently the Jones) polynomial of the
linkoid in this example reflects the entanglement present in the Hopf link.
Notice that the open Hopf-type linkoid, is equivalent to a Hopf link with two
$\epsilon>0$ infinitesimal segments removed. Thus, it is natural to require
that the bracket polynomial of the Hopf-type linkoid is in fact equal to that
of the Hopf link, which is equal to
$\left\langle\raisebox{-5.0pt}{\includegraphics[width=21.68231pt]{fig/hc.png}}\right\rangle=-A^{4}-A^{-4}$.
This would in fact generalize the corresponding property of knot-type knotoids
for the bracket (or Jones). Moreover, as we will see later, this not only a
natural, but a necessary property to satisfy so that the Jones polynomial of
collections of open curves in 3-space tends to that of the corresponding link,
when the curves close, an essential desired property for measuring multi-curve
complexity of curves in 3-space.

Consider the oriented linkoid consisting of 2 components as shown in Figure 4.
By smoothing the crossings in the linkoid diagram using the bracket polynomial
definition for knotoids (See Equation 1), we get the following expansion :

$\begin{split}\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h1.png}}\right\rangle&=A^{2}\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h2.png}}\right\rangle+\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h3.png}}\right\rangle+\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h4.png}}\right\rangle+A^{-2}\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h5.png}}\right\rangle\\\
&=A^{2}d\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h6.png}}\right\rangle+2\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h6.png}}\right\rangle+A^{-2}\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h5.png}}\right\rangle\\\
&=(A^{2}d+2)\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h6.png}}\right\rangle+A^{-2}\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h7.png}}\right\rangle\\\
\end{split}$ (2)

where $d=-A^{2}-A^{-2}$.

The final expression in Equation 2 is a summation of bracket polynomials of
states of the linkoid diagram with two long segments each. Notice that the
Kauffman bracket polynomial of knotoids can evaluate the bracket of an open
arc, simply by setting
$\langle\includegraphics[width=21.68231pt]{fig/sk30.png}\rangle=1$, but in
linkoids we will have more than one long arc in a state, as shown in Equation
2. Moreover, even though the states will consist of two long segments, the
segments connect different endpoints of the original components, as shown in
Equation 2. For this reason, even though the Jones polynomial of links only
accounts for the number of components in a state, this will not work for
linkoids. For example, if we assign the same value to both of the cases in
Equation 2, say
$\left\langle\raisebox{-6.0pt}{\includegraphics[width=17.34189pt]{fig/h6.png}}\right\rangle=\left\langle\raisebox{-6.0pt}{\includegraphics[width=17.34189pt]{fig/h7.png}}\right\rangle=t,\quad$
then
$\left\langle\raisebox{-6.0pt}{\includegraphics[width=17.34189pt]{fig/h1.png}}\right\rangle=(A^{2}d+2+A^{-2})t=(-A^{4}+A^{-2}+1)t$,
which is different than the bracket polynomial of the Hopf link, even if we
set $t=d$. A very recent preprint (at the time of writing this manuscript)
provides a definition of the Jones polynomial of linkoids, where the authors
propose to simply introduce new variables, $\lambda_{ij}$ where $i<j$ refer to
endpoints of the linkoid, for the different connecting arcs [5]. Using that
definition however, we get the expression of the bracket polynomial of the
linkoid in Figure 4 to be equal to
$\quad({-A}^{4}+1)\lambda_{13}\lambda_{24}+A^{-2}\lambda_{12}\lambda_{34}$,
which is different from that of the Hopf link. Thus, none of the, up to date,
proposed definitions satisfy the basic requirement that we set with this
example.

In the following we will show that the definition we introduce in this
manuscript will be a single variable, well-defined, topological invariant of
linkoids, that satisfies the condition that the Jones polynomial of a link-
type linkoid is that of the corresponding link (Theorem 3.1). To do this, we
will first set the framework for analyzing states of linkoids in Section 3.1.
This enables a novel definition of the Kauffman bracket and the Jones
polynomials, which is given in Section 3.2. In Section 3.3 we prove Theorem
3.1.

###  3.1 Segment Cycles of a linkoid state

In this section we introduce an essential concept that will be necessary for
classifying the states of a linkoid diagram, the segment cycles of a state. In
the following, let $L$ be a linkoid diagram with $n\in\mathbb{N}$ components
and let $G$ be the set containing all the endpoints (heads and legs) of $L$.
Recall that, a component of a linkoid with $n$ components is denoted by
$l_{(2j-1,2j)}$ where $j\in\\{1,2,\cdots,n\\}$. Thus
$G=\\{1,2,3,\cdots,2n\\}$.

######  Definition 3.2.

(Pairing Combination and Head-Leg pairing) We will call an element $J\in
S_{2n}$, which can be expressed as the product of $n$ disjoint $2-cycles$, a
pairing combination. In particular, we will call the element $HL=(1\quad
2)(3\quad 4)\cdots(2n-1\quad 2n)=\prod_{i=1}^{n}(2i-1\quad 2i)\in S_{2n}$, the
head-leg pairing.

Note that any linkoid defines, by the connectivity of its components, a head-
leg pairing (see Definition 2.1), while a state, $S$, of a linkoid diagram,
defines a pairing combination ,$J_{S}$ (which may, or may not, be a head-leg
pairing).

######  Definition 3.3.

(Orbit of an endpoint) Given an endpoint $a\in G$, and an arbitrary pairing
combination, $J$ on $G$, the set $Orb_{J}(a)$ of $a$ is defined to be the
orbit of the composition function $HL\circ J$. Symbolically, $Orb_{J}(a)$ is
given as,

$Orb_{J}(a)=\left\\{x\in G\quad\mid\quad x=(HL\circ J)^{m}(a)\quad,\quad
m\in\mathbb{Z}\right\\}$

######  Definition 3.4.

(Segment Cycle) Given a pairing combination, $J$ on $G$, the Segment Cycle of
an endpoint $a\in G$ is defined as follows:

$Seg(a)=Orb_{J}(a)\sqcup Orb_{J}(HL(a))$

where, $\quad Orb_{J}(a)\quad$ (resp. $Orb_{J}(HL(a))$) is the orbit of the
point $a$ (resp. $HL(a)$) under the action of the pairing combination $J$.
Note that for any point $a\in G$, $HL(a)\in G$ always belongs to the same
segment cycle. Thus, a segment cycle always contains an even number of
elements.

######  Remark 3.1.

The orbit of an endpoint can be also defined as the equivalence class of the
endpoint (as an element of $G$) under the equivalence relation
$\mathcal{R}_{\langle HL\circ J\rangle}$, induced by $\langle HL\circ
J\rangle$, which is the action of the cyclic group generated by $HL\circ J\in
S_{2n}$ on the set $G$. Let $\mathcal{G}=\langle HL\circ J\rangle\times\langle
HL\rangle$. The segment cycle of an element $a\in G$ can also be defined as
the orbit of $a$ under the action of $\mathcal{G}$ as shown below :

$Sec(a)=Orb_{\mathcal{G}}(a):=\\{y\in
G:\exists\quad(g,h)\in\mathcal{G}:y=(g,h)\star a\\}$

where, $\star$ means the group action such that $(g,h)\star a=g(h(a))$, the
image of $a$ with respect to the composition function $g\circ h$. The quotient
set $G/\mathcal{R}_{\mathcal{G}}$, where $\mathcal{R}_{\mathcal{G}}$ is the
equivalence relation induced by $\mathcal{G}$, gives the set of all segment
cycles. The cardinality of this set gives the total number of distinct segment
cycles due to the pairing combination $J$.

######  Proposition 3.1.

The number of segment cycles in a state $S$, $|S|_{cyc}$, is bounded by:
$\quad 1\leq|S|_{cyc}\leq n.$.

###### Proof.

If $J_{S}=HL$ (the trivial pairing combination), it follows from Remark 3.1
that $\mathcal{G}=\langle HL\circ HL\rangle\times\langle
HL\rangle=Id\times\langle HL\rangle$. Thus, $|G/\mathcal{R}_{\mathcal{G}}|=n$.
If $J_{S}$ is single cycle, then $\mathcal{G}$ partitions the set $G$ into a
single equivalence class. Therefore, $|G/\mathcal{R}_{\mathcal{G}}|=1$.

∎

######  Remark 3.2.

Let $S$ denote a state of $L$ with associated pairing $J_{S}$, and let
$Seg(a)$ be a segment cycle in $L$, such that $|Seg(a)|=2k$. Then we can
represent $Seg(a)$ by a circle decorated with the $2k$ endpoints of $L$ (See
Figure 5), following the order in which they appear in the cycle (See Figure
5). Note that the arcs connecting 2 adjacent points in this circle alternate
between the functions, $J$ and $HL$. Any two points connected by $J$ belong to
the same component of the state $S$ and any two points connected by $HL$
belong to the same component in $L$.

Figure 5: Representation of the segment cycle of $a\in G$ in terms of a
decorated circle. Let us consider a circle and let $a\in G$ be the initial
point on the circle. The remaining $2k-1$ endpoints can be uniquely added into
the circle in the order $\quad J(a),\quad HL(J(a)),\quad
J(HL(J(a))),\quad\cdots,\quad\text{upto }HL(a)$. Note that the arcs connecting
2 adjacent points in this circle alternate between the functions, $J$ (solid)
and $HL$ (dashed). Any two points connected by $J$ belong to the same
component of the state $S$ and any two points connected by $HL$ belong to the
same component in $L$. (The dotted line in the figure indicates the continued
process of adding points.)

Example 1 (cont.) :

Consider the linkoid diagram and the particular state (say $S$) as shown in
Figure 4. In this example, the set $G$ of all endpoints is $\\{1,2,3,4\\}$.
Note that the open arc components of a linkoid diagram $L$ define a head-leg
pairing, while the states of $L$ can define other pairing combinations. For
example, let us denote by $s_{1},s_{2}$ the final states of Equation 2. Then
$s_{1}$ defines the pairing combination $J_{s_{1}}$ , which can be represented
by the permutation $(1\quad 3)(2\quad 4)\in S_{4}$ and $s_{2}$ defines the
pairing combination $J_{s_{2}}$ , which can be represented by the permutation
$(1\quad 2)(3\quad 4)\in S_{4}$. The states $s_{1}$ and $s_{2}$ are explicitly
shown in Figure 6.

For each of the permutations, $J_{s_{1}}$ and $J_{s_{2}}$, we can construct
the corresponding set of segment cycles. Note that, a segment cycle can be
represented as a decorated circle. Corresponding $J_{s_{1}}$ and $J_{s_{2}}$,
the possible segment cycles are shown in Figure 6.

Figure 6: (Top) The connections among endpoints due to state $s_{1}$,
corresponding to the pairing combination $J_{s_{1}}=(1\quad 3)(2\quad 4)\in
S_{4}$, and the resultant segment cycle. (Bottom) The connections among
endpoints due to state $s_{2}$, corresponding to the pairing combination
$J_{s_{2}}=(1\quad 2)(3\quad 4)\in S_{4}$, and the resultant distinct segment
cycles.

###  3.2 The Jones polynomial of Linkoids

In the previous section we showed that open segments in a smoothed state of a
linkoid can be grouped into segment cycles. We also saw the analogy between
segment cycles and decorated circles. In this section, we use these concepts
to define the bracket polynomial of linkoids and the Jones polynomial of
linkoids (as the normalized bracket polynomial).

######  Definition 3.5.

(Bracket Polynomial of a Linkoid) Let $L$ be a linkoid diagram with $n$
components. The bracket polynomial of the linkoid is completely characterised
by the following Skein relation and initial conditions:

$\begin{split}\left\langle\raisebox{-10.0pt}{\includegraphics[width=26.01613pt]{fig/cross10.png}}\right\rangle=A\left\langle\raisebox{-10.0pt}{\includegraphics[width=26.01613pt]{fig/cross30.png}}\right\rangle+A^{-1}\left\langle\raisebox{-10.0pt}{\includegraphics[width=26.01613pt]{fig/cross20.png}}\right\rangle,&\hskip
14.22636pt\left\langle L\cup\bigcirc\right\rangle=(-A^{2}-A^{-2})\left\langle
L\right\rangle,\\\ \hskip
14.22636pt\left\langle\raisebox{-12.0pt}{\includegraphics[width=56.37271pt]{fig/multi.png}}\right\rangle=(-A^{2}-A^{-2})^{|cyc|}.&\\\
\end{split}$ (3)

where $|cyc|$ denotes the number of distinct segment cycles. In other words,
$|cyc|=|G/\mathcal{R}_{\mathcal{G}}|$, where $\mathcal{R}_{\mathcal{G}}$ is
the equivalence relation induced on $G$ by $\mathcal{G}=\langle HL\circ
J\rangle\times\langle HL\rangle$, where $J$ is the pairing defined by the
indices $a,b,c,d,\cdots p,q$.

The bracket polynomial of $L$ can be formulated as the following state sum
expression :

$\langle L\rangle:=\sum_{S}A^{\sigma(S)}d^{|S|_{circ}-1}d^{|S|_{cyc}}\\\ $ (4)

where, $S$ is a state corresponding to a choice of smoothing over all double
points in $L$; $\sigma(S)$ is the algebraic sum of the smoothing labels of
$S$; $|S|_{circ}$ is the number of disjoint circles in $S$ and $|S|_{cyc}$ is
the number of distinct segment cycles in $S$.

The bracket polynomial of linkoids has the following properties:

  1. 1.

It preserves the underlying Skein relation used in the computation of bracket
polynomial of knots and knotoids.

  2. 2.

If $L$ is a link diagram, the novel definition of the bracket polynomial
(Definition 4) coincides with the traditional Kauffman bracket polynomial of
$L$. Indeed, the states of $L$ have no long segments. Therefore $|S|_{cyc}=0$
and

$\langle L\rangle=\sum_{S}A^{\sigma(S)}d^{|S|_{circ}-1}.$

  3. 3.

If $L$ is a knotoid diagram, then the novel definition of the bracket
polynomial (Definition 4) coincides with the Kauffman bracket polynomial of
knotoids. Indeed, for a knotoid, $|S|_{cyc}=1$ for all states and

$\langle
L\rangle=\sum_{S}A^{\sigma(S)}d^{|S|_{circ}-1}d^{1}=\sum_{S}A^{\sigma(S)}d^{|S|_{circ}}.$

  4. 4.

For a trivial linkoid with $n$ components,

$\langle L\rangle=d^{n-1}.$

Indeed, for a trivial linkoid with $n$ components, there are no rearrangements
in the pairing of endpoints since there are no crossings to be resolved.
Therefore there is only 1 state in the state sum which has the original $n$
segments and their endpoints intact and

$\langle L\rangle=A^{0}\times d^{0-1}\times d^{n}=d^{n-1}.$

  5. 5.

For link-type linkoids the bracket polynomial coincides with that of the
corresponding link upon the closure of endpoints (see Theorem 3.1).

In the following, the bracket polynomial turns into an invariant for linkoids
with a normalization by the writhe giving rise to a definition for the Jones
polynomial of linkoids with the substitution of $A=t^{-1/4}$. The writhe,
$Wr(L)$, of an oriented linkoid diagram $L$ is the algebraic sum of signs
(positive or negative) of crossings of $L$.

######  Definition 3.6.

(Jones Polynomial of a Linkoid) The normalized bracket polynomial of a linkoid
diagram $L$ is defined as

$f_{L}=(-A^{3})^{-Wr(L)}\langle L\rangle$

where $Wr(L)$ is the writhe of the linkoid diagram and $\langle L\rangle$ is
as seen in Definition 4. This gives the Jones polynomial of a linkoid with the
substitution $A=t^{-1/4}$.

The Jones polynomial of linkoids (See Definition 3.6) has the following
properties:

  1. 1.

The Jones polynomial of a linkoid (See Definition 3.6) is a topological
invariant of linkoids and it satisfies the Jones polynomial Skein relations :

$t^{-1}f_{L_{+}}-tf_{L_{-}}=(t^{1/2}-t^{-1/2})f_{L_{0}}.$

where, the linkoids $L_{+},\quad L_{-}$ and $L_{0}$ are identical almost
everywhere except at one crossing as shown below :

  2. 2.

If $L$ is a link diagram, then the new definition of the Jones polynomial
(Definition 3.6) gives the traditional Jones polynomial of the link. Indeed,
for a link $L$, there is no long segment. Therefore $|S|_{cyc}=0$ and

$f_{L}=(-A^{3})^{-Wr(L)}\sum_{S}A^{\sigma(S)}d^{|S|_{circ}-1}d^{0}.$

  3. 3.

If $L$ is a knotoid diagram, the Jones polynomial of $L$ defined here
coincides with the Jones polynomial of knotoids. Indeed, if $L$ has only one
component, then $|S|_{cyc}=1$ (only 1 long segment) for all states and

$f_{L}=(-A^{3})^{-Wr(L)}\sum_{S}A^{\sigma(S)}d^{|S|_{circ}}.$

  4. 4.

For a trivial linkoid with $n$ components,

$f_{L}=d^{n-1}.$

Indeed, in that case there are no rearrangements in the pairing of endpoints
since there are no crossings to be resolved. Therefore there is only 1 state
in the state sum which has the original $n$ segments and their endpoints
intact. Since there are no crossings, the writhe is zero and this allows us to
express the Jones polynomial as

$f_{L}=(-A^{3})^{0}A^{0}\times d^{0-1}\times d^{n}=d^{n-1}.$

  5. 5.

For link-type linkoids the Jones polynomial coincides with that of the
corresponding link upon the closure of endpoints (see Theorem 3.1).

Example 1 (cont.) :

Let us return to the example of a linkoid diagram with 2 components as given
in Figure 4. Here, the set $G$ of endpoints is equal to $\\{1,2,3,4\\}$. The
final step of the state sum expansion of its bracket polynomial (See Equation
2) involves the bracket polynomials of the two states  and . Let us call them
$s_{1}$ and $s_{2}$ respectively and their respective pairing combinations as
$J_{s_{1}}$ and $J_{s_{2}}$. Note that $J_{s_{2}}=HL=(1\quad 2)(3\quad 4)$
implies that there are 2 distinct segment cycles corresponding to $s_{2}$,
namely $Seg(1)=\\{1,2\\}$ and $Seg(3)=\\{3,4\\}$. Whereas, the action of
$J_{s_{1}}=(1\quad 3)(2\quad 4)$ on $G$ gives $Seg(1)=\\{1,2,3,4\\}=G$.
Therefore, there is only 1 distinct segment cycle corresponding to $s_{1}$.

Using Definition 4, the final step of Equation 2 can be simplified as follows
:

$\begin{split}\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h1.png}}\right\rangle&=(A^{2}d+2)\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h6.png}}\right\rangle+A^{-2}\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h7.png}}\right\rangle\\\
&=(A^{2}d+2)d^{0-1}d^{1}+A^{-2}d^{0-1}d^{2}\\\ &=-A^{4}-A^{-4}\\\ \end{split}$
(5)

Notice that the above expression matches the bracket polynomial for the Hopf
link, i.e.
$\left\langle\raisebox{-5.0pt}{\includegraphics[width=17.34189pt]{fig/hc.png}}\right\rangle=-A^{4}-A^{-4}$.
The writhe of the diagram in Figure 4 is
$Wr(\raisebox{-3.0pt}{\includegraphics[width=8.67424pt]{fig/h0_dark.png}})=-2$.
Therefore, the Jones polynomial of the linkoid is evaluated as follows :

$\begin{split}f_{\raisebox{-6.0pt}{\includegraphics[width=13.00806pt]{fig/h0_dark.png}}}&=(-A^{3})^{-Wr(\raisebox{-3.0pt}{\includegraphics[width=8.67424pt]{fig/h0_dark.png}})}\left\langle\raisebox{-12.0pt}{\includegraphics[width=26.01613pt]{fig/h1.png}}\right\rangle\\\
&=(-A^{3})^{2}(-A^{4}-A^{-4})\\\ &=-A^{10}-A^{2}\\\ \end{split}$ (6)

which is equal to that of the Hopf link. In fact, we will prove that this is
the case for all link-type linkoids.

(a)  (b)

Figure 7: Pure linkoids i.e. non-link-type linkoids. Clearly Fig (a) has
Writhe = +1 and Fig (b) has Writhe = 0.

The Jones polynomial (Definition 3.6) distinguishes non-equivalent linkoids in
$S^{2}$ and for proper linkoids, this measure is not that of any link. For
example the Jones polynomial of the linkoid given in Figure 7 (a) is equal to
$f_{\raisebox{-6.0pt}{\includegraphics[width=17.34189pt]{fig/properK2.png}}}=A+A^{-1}$
whereas, the Jones polynomial of the linkoid given in Figure 7 (b) is equal to
$f_{\raisebox{-6.0pt}{\includegraphics[width=17.34189pt]{fig/properK1.png}}}=-A^{-2}+A^{-4}$.

###  3.3 The Jones polynomial of Link-type Linkoids

In this section we will prove that the Jones polynomial of link-type linkoids
is equal to the Jones polynomial of the corresponding link.

######  Theorem 3.1.

Let $L$ be a link-type linkoid with $n$ components and $L_{c}$ be the
corresponding link (the link that results from connecting the head to the leg
of each component in a way that no new crossing is created). Then the Jones
polynomials of $L$ and $L_{c}$ are equal, that is $f_{L}=f_{L_{c}}$.

###### Proof.

By the definition of $L$ and $L_{c}$, we notice that $L$ can be created from
$L_{c}$ by omitting $n$ arcs (closure arcs). These arcs connect the endpoints
$2j-1$ and $2j$ for each component $l_{(2j-1,2j)}$ of $L$, where $j\in
G=\\{1,2,\cdots,n\\}$. Let us construct a decoration on $L_{c}$ by $L$, by
keeping track of the endpoints of $L$ with labels. Thus $L_{c}$ is a union of
$L$ with closure arcs.

The closure arcs in $L$ do not create any new crossings, hence the total
number of double points in $L_{c}$ is the same as that in $L$. Therefore,
$Wr(L)=Wr(L_{c})$. In the following, we show that the bracket polynomials,
$\langle L\rangle$ and $\langle L_{c}\rangle$ are also equal.

We know that the states of a link or a linkoid diagram are completely
determined by the choice of smoothing at the crossings of the diagram. Since
$L$ and $L_{c}$ have identical crossings, for each state $S$ of the linkoid
diagram $L$, there exists a state $S_{c}$ in the link diagram $L_{c}$, such
that their smoothing labels are equal, that is,
$\quad\sigma(S)=\sigma(S_{c})$. By definition, the contribution of $S$ in the
state sum expression of $\quad\langle L\rangle\quad$ is $\quad
A^{\sigma(S)}d^{|S|_{circ}+|S|_{cyc}-1}\quad$, where $|S|_{circ}$ is the
number of disjoint circles and $|S|_{cyc}$ is the number of segment cycles
formed by the disjoint long segments in the state $S$. Similarly, the
contribution of $S_{c}$ in the state sum expression of $\quad\langle
L_{c}\rangle\quad$ is $\quad A^{\sigma(S)}d^{|S_{c}|_{circ}-1}.\quad$ We will
prove that these two terms are equal by showing that
$\quad|S|_{circ}+|S|_{cyc}=|S_{c}|_{circ}$.

Since the closure arcs of $L$ in $L_{c}$ do not introduce crossings, the state
$S$ can be obtained from $S_{c}$, by omitting the closure arcs. Thus, for a
decorated $L_{c}$, $S_{c}$ will also be decorated with an even number of
labels on each circle (since any closure arc has two endpoints and closure
arcs are disjoint). Notice that if a circle in $S_{c}$ is not decorated, then
it corresponds to a circle in $S$. Let us express the total number of circles
in $S_{c}$ as $|S_{c}|_{circ}=|S_{c}|_{(circ,u)}+|S_{c}|_{(circ,d)}\quad,$
where $\quad|S_{c}|_{(circ,u)}$ and $|S_{c}|_{(circ,d)}\quad$ denote the
number of undecorated and decorated circles, respectively. Thus,
$\quad|S|_{circ}=|S_{c}|_{(circ,u)}$. Therefore, we need to prove that
$|S|_{cyc}=|S_{c}|_{(circ,d)}$.

Once all the circles that do not involve endpoints are taken care of in both
$S$ and $S_{c}$, we are left with $n$ long segments in $S$ and
$|S_{c}|_{(circ,d)}$ number of decorated circles in $S_{c}$. Since $S$ is
formed by $S_{c}$ by removing the closure arcs, a decorated circle in $S_{c}$
with $2k$ labels gives rise to $k$ long segments in $S$. We will show that
these $k$ long segments form a segment cycle.

Notice that for a decorated circle in $S_{c}$, two adjacent labels are either
endpoints of a closure arc, or a new connection formed by the smoothings. So,
each consecutive pair of labels in a decorated circle in $S_{c}$ that does not
belong to a closure arc, it defines a pairing $J_{S}$ in $S$ and each
consecutive pair of labels in the decorated circle in $S_{c}$ that belong to
the same closure arc, they are related by $HL$. Then the corresponding $k$
long segments in $S$ define a segment cycle.

These $k$ long segments cannot overlap with any other decorated circle of
$S_{c}$ because it will violate the fact that all the decorated circles of
$S_{c}$ are disjoint. Therefore, for every decorated circle in $S_{c}$ there
is a unique collection of long segments in $S$ such that these long segments
form a segment cycle. This implies, $|S_{c}|_{(circ,d)}=|S|_{cyc}$. Therefore,
$\langle L\rangle=\langle L_{c}\rangle$. ∎

##  4 The Jones polynomial of Open Curves in 3-space

Consider a collection of $m\in\mathbb{N}$ open or closed curves in 3-space in
general position. A (regular) projection of these collection of $n$ curves can
give rise to a different linkoid diagram based on the choice of the direction
of projection. Notice that with probability one, a projection will be generic.
We use the framework introduced in [14] and Definition 4, to rigorously define
the bracket and Jones polynomials of a collection of $n\in\mathbb{N}$ open
curves in 3-space. We will define the Jones polynomial as the normalized
bracket polynomial.

######  Definition 4.1.

Let $\mathcal{L}$ denote a collection of $m\in\mathbb{N}$ open curves in
3-space. Let $\mathcal{L}_{\vec{\xi}}$ denote the projection of $\mathcal{L}$
on a plane with normal vector $\vec{\xi}$. The normalised bracket polynomial
of $\mathcal{L}$ is defined as:

$f_{\mathcal{L}}=\frac{1}{4\pi}\int_{\vec{\xi}\in
S^{2}}(-A^{3})^{-Wr(\mathcal{L}_{\vec{\xi}})}\langle\mathcal{L}_{\vec{\xi}}\rangle
dS.$ (7)

where, each $\mathcal{L}_{\vec{\xi}}$ is a linkoid diagram and its bracket
polynomial can be calculated by using Definition 4. Note that the integral is
taken over all vectors $\vec{\xi}\in S^{2}$ except a set of measure zero
(corresponding to the irregular projections). This gives the Jones polynomial
of a collection of open curves in 3-space with the substitution $A=t^{-1/4}$.

This new definition of the Jones polynomial of collections of open or closed
curves in 3-space generalizes all the previous definitions of the Jones
polynomial, so that it satisfies the following properties:

  1. 1.

The Jones polynomial defined by Equation 7 does not depend on any particular
projection of the collection of open or closed curves.

  2. 2.

For a collection of open curves this polynomial is not the polynomial of a
corresponding/approximating link, nor that of a corresponding/approximating
linkoid.

  3. 3.

The Jones polynomial of a collection of open curves in $3-$space has real
coefficients. It is not a topological invariant, but it is a continuous
function of the curve coordinates (see Proposition 4.1).

  4. 4.

For a collection of closed curves in $3-$space (a link), the Jones polynomial
defined in Equation 7 gives the traditional Jones polynomial and it can be
computed from a single projection, i.e.
$f_{\mathcal{L}}=f_{\mathcal{L}_{\vec{\xi}}}$ where, $\vec{\xi}\in S^{2}$ is
any projection vector.

  5. 5.

As the endpoints of a collection of open curves in 3-space tend to coincide,
the Jones polynomial tends to that of the corresponding link.

  6. 6.

For a linkoid of 1 component (a knotoid), the Jones polynomial of Definition
4.1 gives the Jones polynomial defined in [14].

In the case of polygonal curves in 3-space, the Jones polynomial attains a
simpler expression. Without loss of generality, suppose that all the curves
have $n$ edges each. Then there exists a finite number (Say $k\in\mathbb{N}$)
of distinct linkoid types ($L_{i}$) which may occur in any projection of the
collection of open curves, $\mathcal{L}$. Therefore, Equation 10 can also be
expressed as the following finite sum:

$f_{\mathcal{L}}=\sum_{i=1}^{k}p_{i}f_{L_{i}}$ (8)

where $p_{i}$ denotes the geometric probability that a projection of
$\mathcal{L}$ gives the linkoid $L_{i}$.

######  Proposition 4.1.

Let $\mathcal{L}$ denote a collection of simple open curves in 3-space. Then
$f_{\mathcal{L}}$ is a continuous function of the co-ordinates of
$\mathcal{L}$.

###### Proof.

Let us approximate $\mathcal{L}$ by a set of polygonal curves of $n$ edges
each, we denote $\mathcal{L}^{(n)}$. Then

$f_{\mathcal{L}^{(n)}}=\sum_{i=1}^{k}p_{i}f_{L^{(n)}_{i}}$ (9)

where $L^{(n)}_{i},i=1,\dotsc,k$ are the possible linkoids that can occur in
all projections of $\mathcal{L}^{(n)}$ and $p_{i}$ the corresponding geometric
probabilities. The geometric probability $p_{i}$ can be expressed as

$p_{i}=\frac{2A_{0}}{4\pi}$

where $A_{0}$ is the Area on the sphere corresponding to vectors $\vec{\xi}\in
S^{2}$ such that the projection of $\mathcal{L}^{(n)}$ along such vectors
results in the linkoid $L_{i}^{(n)}$. $A_{0}$ is a quadrangle bounded by great
circles defined by the edges and vertices of the polygonal curves in
$\mathcal{L}^{(n)}$. Thus it is a continuous function of the coordinates of
$L^{(n)}$ (see proof of Lemma 3.1 in [14]). The result follows as $n$ goes to
infinity. ∎

######  Corollary 4.1.

Let $L$ denote a collection of open curves in 3-space. As the endpoints of the
curves tend to coincide to form a link $L_{c}$, $f_{\mathcal{L}}$ tends to
$f_{\mathcal{L}_{c}}$

###### Proof.

The result follows by Proposition 4.1, Definition 4.1 and Theorem 3.1. ∎

The statement below follows as a corollary of the properties of open and
closed curves in 3-space that we have established so far.

######  Corollary 4.2.

The Jones polynomial is a continuous function in the space of all simple
curves (open or closed) in 3-space.

In a similar way, we can define the Kauffman bracket polynomial of a
collection of open curves in 3-space, as follows:

######  Definition 4.2.

Let $\mathcal{L}$ denote a collection of $n\in\mathbb{N}$ open curves in
3-space. Let $\mathcal{L}_{\vec{\xi}}$ denote the projection of $\mathcal{L}$
on a plane with normal vector $\vec{\xi}$. The bracket polynomial of
$\mathcal{L}$ is defined as:

$\langle\mathcal{L}\rangle=\frac{1}{4\pi}\int_{\vec{\xi}\in
S^{2}}\langle\mathcal{L}_{\vec{\xi}}\rangle dS$ (10)

where, each $\mathcal{L}_{\vec{\xi}}$ is a linkoid diagram and its bracket
polynomial can be calculated by using Definition 4. Note that the integral is
taken over all vectors $\vec{\xi}\in S^{2}$ except a set of measure zero
(corresponding to the irregular projections).

Properties of the bracket polynomial of collections of open curves in 3-space
:

  1. 1.

The bracket polynomial defined in Equation 10 does not depend on any
particular projection of the collection of curves.

  2. 2.

For an open curve this polynomial is not the polynomial of a
corresponding/approximating link, nor that of a corresponding/approximating
linkoid.

  3. 3.

The bracket polynomial defined in Equation 10 is not a topological invariant,
but it is a continuous function of the curve coordinates for both open and
closed curves in 3-space.

  4. 4.

As the endpoints of a collection of open curves in 3-space tend to coincide,
the bracket polynomial tends to that of the corresponding link.

  5. 5.

For a linkoid of one component, the bracket polynomial of Definition 4.2 gives
the bracket polynomial defined in [14].

Example 2 :

Consider a set of open borromean rings realised in 3-space by the following
three lists of co-ordinates :

       R = [[0,0,0],[1,1,0],[2,2,0.5],[3,3,0.5],[4,4,0],[5,5,0],[6,6,0.5],
       [7,7,0.5],[8,7,0.5], [9,5,0.2],[9,3,0.2],[8,0,0.2],[8,-1,0.2],
       [6,-1.5,0],[4,-2,0],[2,-1.5,0]]

       B = [[1,0,0.5],[4,0,0],[5,1,0],[5,4,0.5],[4,5,0.5],[3,6,0],
       [2,7,0],[-1,6,0],[-1,3,0.5]]

       K = [[6,0,0.5],[7,6,0],[6,7,0],[3,7,0.5],[2,6,0.5],[2,3,0],
       [3,2,0],[4,1,0.5]]

where R, B and K denote the red, blue and black curves, respectively. The list
R can be updated by an additional element (say $\vec{r}$) by using the
following parametrization :

$\vec{r}=\vec{r_{1}}+s(\vec{r_{2}}-\vec{r_{1}})$

where, $0\leq s\leq 1$ and $\vec{r_{1}}$ and $\vec{r_{2}}$ are respectively
the the last and the first points in R i.e. $\vec{r_{1}}=\text{{\tt R}}[-1]$
and $\vec{r_{2}}=\text{{\tt R}}[0]$. Using the same parameter $s$, the lists B
and K can each be updated by an additional element.

Let us denote a configuration of the system of open borromean rings by $w(s)$,
where $s$ is the value of the parameter in concern. Clearly, the initial
configuration of the system of open borromean rings can thus be denoted as
$w(0)$. As we start updating the lists R, B and K by the above
parametrization, it means that the endpoints of each component of $w(0)$ move
closer and closer in time, eventually attaining the configuration of the
closed borromean rings, namely $w(1)$. During this journey from $w(0)$ to
$w(1)$, we encounter infinitely many intermediate configurations, some of
which are : $w(0.22)$, $w(0.44)$, $w(0.67)$, $w(0.68)$, $w(0.70)$, and
$w(0.89)$. The Jones polyomials of the collection of curves from the initial
to the final stage, along with the aforementioned intermediate steps are
presented in Figure 8 and their explicit expressions are given as follows:

Figure 8: The Jones polynomial of a system of open borromean rings in 3-space
as the end points move closer in time to ultimately give rise to the closed
borromean rings in 3-space. The coefficients of the Jones polynomial are a
continuous function of the chain coordinates.

$f_{w(0)}$ | = | $-0.26t^{-3}+1.49t^{-2}+1.84t^{-3/2}+0.16t^{-1}-0.38t^{-1/2}$
---|---|---
|  | \+ $0.72+0.67t^{1/2}-0.18t-0.22t^{3/2}+0.22t^{2}-0.07t^{3}$
$f_{w(0.22)}$ | = | $-0.59t^{-3}+2.15t^{-2}+1.83t^{-3/2}-0.81t^{-1}-1.02t^{-1/2}$
|  | \+ $1.39+1.59t^{1/2}-0.21t-0.53t^{3/2}-0.27t^{2}-0.08t^{3}$
$f_{w(0.44)}$ | = | $-0.92t^{-3}+2.83t^{-2}+1.67t^{-3/2}-1.86t^{-1}-1.53t^{-1/2}$
|  | \+ $2.27+2.39t^{1/2}-0.45t-0.85t^{3/2}+0.53t^{2}-0.16t^{3}$
$f_{w(0.67)}$ | = | $-0.98t^{-3}+2.95t^{-2}+1.45t^{-3/2}-2.04t^{-1}-1.4t^{-1/2}$
|  | \+ $2.60+2.22t^{1/2}-0.68t-0.78t^{3/2}+0.86t^{2}-0.27t^{3}$
$f_{w(0.68)}$ | = | $-0.98t^{-3}+2.94t^{-2}+1.43t^{-3/2}-2.02t^{-1}-1.39t^{-1/2}$
|  | \+ $2.60+2.19t^{1/2}-0.68t-0.77t^{3/2}+0.88t^{2}-0.28t^{3}$
$f_{w(0.70)}$ | = | $-0.98t^{-3}+2.96t^{-2}+1.38t^{-3/2}-2.04t^{-1}-1.34t^{-1/2}$
|  | \+ $2.66+2.11t^{1/2}-0.74t-0.75t^{3/2}+0.96t^{2}-0.31t^{3}$
$f_{w(0.89)}$ | = | $-0.99t^{-3}+2.98t^{-2}+0.18t^{-3/2}-2.06t^{-1}-0.17t^{-1/2}$
|  | \+ $3.86+0.35t^{1/2}-1.9t-0.15t^{3/2}+2.74t^{2}-0.9t^{3}$
$f_{w(1)}$ | = | $-t^{-3}+3t^{-2}-2t^{-1}+4-2t+3t^{2}-t^{3}$

Notice that $f_{w(0)}$ is a new polynomial representing the particular
configuration of the open Borromean ring in 3-space. This is a polynomial with
real coefficients, while $f_{w(1)}$ is the integer polynomial invariant of the
Borromean ring. We notice that, as the endpoints of the open link tend to
coincide, the coefficients of the powers of $t$ that compose the Borromean
ring tend to their corresponding integer values, while the coefficients of the
powers of $t$ that are not part of the Borromean ring, tend to zero.

##  5 Conclusions

In this work we introduced the first measure of topological complexity of
collections of open curves in 3-space, based on a novel Jones polynomial. The
classical Jones polynomial is a special case of this novel Jones polynomial.
For collections of open curves in 3-space, the novel Jones polynomial is a
polynomial with real coefficients, which are continuous functions of the curve
coordinates and as the endpoints of the curves tend to coincide, it tends to
the integer coefficient, Jones polynomial invariant of the resulting link.

The definition of the Jones polynomial of open curves in 3-space is based on a
novel definition of the Jones polynomial of linkoids that we introduced in
this manuscript as well. This novel Jones polynomial of linkoids is the only
such definition that satisfies the basic property that the polynomial of a
link-type linkoid is that of corresponding link. This polynomial thus
generalizes the Jones polynomial of knotoids, while maintaining its
properties. This new definition of the Jones polynomial of linkoids, will
enable to properly define other invariants of linkoids as well in the future.

We demonstrated with numerical examples how the novel Jones polynomial of open
curves in 3-space can be useful in practice to characterize multi-chain
complexity for the first time. This enables the rigorous characterization of
multi-chain entanglement in many physical systems obtained either from
experiments or simulations, such as polymers and biopolymers, where
entanglement is arguably an important factor of mechanics and function, which
has been elusive.

##  6 Acknowledgements

Kasturi Barkataki and Eleni Panagiotou were supported by NSF (Grant No.
DMS-1913180 and NSF CAREER 2047587).

## References

  * [1] J. Arsuaga, M. Vazquez, P. McGuirk, S. Trigueros, D. W. Sumners, and J. Roca.  DNA knots reveal a chiral organization of DNA in phage capsids.  Proc. Natl. Acad. Sci. (USA), 102:9165–9169, 2005.
  * [2] P. G. De Gennes.  The physics of liquid crystals.  Clarendon Press, Oxford, 1974.
  * [3] S. F. Edwards.  Statistical mechanics with topological constraints: I.  Proc. Phys. Soc., 91:513–9, 1967.
  * [4] S. F. Edwards.  Statistical mechanics with topological constraints: Ii.  J. Phys. A: Gen. Phys., 1:15–28, 1968.
  * [5] Boštjan Gabrovšek and Neslihan Gügümcü.  Invariants of multi-linkoids, 2022.
  * [6] K. F. Gauss.  Werke.  Kgl. Gesellsch. Wiss. Göttingen, 1877.
  * [7] N. Gügümcu and L. H. Kauffman.  New invariants of knotoids.  Eur. J. Comb., 65:186–229, 2017.
  * [8] N. Gügümcu and L. H. Kauffman.  Parity in knotoids.  arXiv:1905.04089, pages 1–19, 2019.
  * [9] N. Gügümcu and S. Lambropoulou.  Knotoids, braidoids and applications.  Symmetry, 9:315, 2017.
  * [10] Tom Herschberg, Jan-Michael Y. Carrillo, Bobby G. Sumpter, Eleni Panagiotou, and Rajeev Kumar.  Topological effects near order–disorder transitions in symmetric diblock copolymer melts.  Macromolecules, 54(16):7492–7499, 2021.
  * [11] Y. Liu, M. O’Keeffe, M. Treacy, and O. Yaghi.  The geometry of periodic knots, polycatenanes and weaving from a chemical perspective: a library for reticular chemistry.  Chem. Soc. Rev., 47:4642–4664, 2018.
  * [12] Manousos Manouras, Sofia Lambropoulou, and Louis H. Kauffman.  Finite type invariants for knotoids.  European Journal of Combinatorics, 98:103402, dec 2021.
  * [13] E. Panagiotou.  The linking number in systems with periodic boundary conditions.  J. Comput. Phys., 300:533–573, 2015.
  * [14] E. Panagiotou and L. Kauffman.  Knot polynomials of open and closed curves.  Proc. R. Soc. A, 476:20200124, 2020.
  * [15] E. Panagiotou and L. Kauffman.  Vassiliev measures of complexity for open and closed curves in 3-space.  Proc. R. Soc. A, 477:20210440, 2021.
  * [16] E. Panagiotou and M. Kröger.  Pulling-force-induced elongation and alignment effects on entanglement and knotting characteristics of linear polymers in a melt.  Phys. Rev. E, 90:042602, 2014.
  * [17] E. Panagiotou, M. Kröger, and K. C. Millett.  Writhe and mutual entanglement combine to give the entanglement length.  Phys. Rev. E, 88:062604, 2013.
  * [18] E. Panagiotou, K. C. Millett, and P. J. Atzberger.  Topological methods for polymeric materials: characterizing the relationship between polymer entanglement and viscoelasticity.  Polymers, 11:11030437, 2019.
  * [19] E. Panagiotou, C. Tzoumanekas, S. Lambropoulou, K. C. Millett, and D. N. Theodorou.  A study of the entanglement in systems with periodic boundary conditions.  Progr. Theor. Phys. Suppl., 191:172–181, 2011.
  * [20] J. Qin and S. T. Milner.  Counting polymer knots to find the entanglement length.  Soft Matter, 7:10676–93, 2011.
  * [21] R. Ricca.  Topology bounds energy of knots and links.  Proc. Royal Society A, 464:293–300, 2008.
  * [22] Philip Smith and Eleni Panagiotou.  The second vassiliev measure of uniform random walks and polygons in confined space.  Journal of Physics A: Mathematical and Theoretical, 55(9):095601, feb 2022.
  * [23] J. I. Sulkowska, E. J. Rawdon, K. C. Millett, J. N. Onuchic, and A. Stasiak.  Conservation of complex knotting and slipknotting in patterns in proteins.  Proc. Natl. Acad. Sci., 109:E1715, 2012.
  * [24] J. B. Taylor.  Relaxation of toroidal plasma and generation of reverse magnetic fields.  Phys. Rev. Lett., 33:1139, 1974.
  * [25] V. Turaev.  Knotoids.  Osaka, J. Math., 49:195–223, 2012.
  * [26] Jason Wang and Eleni Panagiotou.  The protein folding rate and the geometry and topology of the native state.  bioRxiv, 2021.


# Learning to solve arithmetic problems with a virtual abacus

Flavio Petruzzellis  Ling Xuan Chen  Alberto Testolin Corresponding Author:
<EMAIL_ADDRESS>University of Padova, Italy

###### Abstract

Acquiring mathematical skills is considered a key challenge for modern
Artificial Intelligence systems. Inspired by the way humans discover numerical
knowledge, here we introduce a deep reinforcement learning framework that
allows to simulate how cognitive agents could gradually learn to solve
arithmetic problems by interacting with a virtual abacus. The proposed model
successfully learn to perform multi-digit additions and subtractions,
achieving an error rate below 1% even when operands are much longer than those
observed during training. We also compare the performance of learning agents
receiving a different amount of explicit supervision, and we analyze the most
common error patterns to better understand the limitations and biases
resulting from our design choices.

##  1 Introduction

Deep learning systems excel in a variety of domains, but struggle to learn
cognitive tasks that require the manipulation of symbolic knowledge [1]. This
limitation is particularly evident in the field of mathematical cognition [2],
which requires to grasp abstract relationships and deploy sophisticated
reasoning procedures. Indeed, even state-of-the-art language models fall short
in mathematical tasks that require strong generalization capabilities [3]
(though very recent work has shown that performance significantly improves
following fine-tuning on large-scale mathematical datasets [4]).

One possibility to tackle this challenge is to endow deep architectures with
ad-hoc primitives specifically designed to manipulate arithmetic concepts [5,
6]. Nevertheless, alternative approaches suggest that symbolic numerical
competence could emerge from domain-general learning mechanisms [7, 8]. Recent
work has also tried to deploy deep reinforcement learning (RL) to solve math
word problems [9]. Notably, deep RL architectures that incorporate copy and
alignment mechanisms seem to discover more sophisticated problem solving
procedures [10], and could even learn automatic theorem proving when combined
with Monte-Carlo tree search algorithms [11].

In this work we explore whether model-free deep RL agents could learn to solve
simple arithmetic tasks (expressions involving sum and subtraction) by
exploiting an external representational tool, which can be functionally
conceived as a virtual abacus. Differently from the above-mentioned
approaches, our goal is to take advantage of the way humans solve the task by
simulating the interaction of the RL agent with an abacus, which can be
partially guided through supervised learning mechanisms. This allows teaching
the agent existing algorithms for the solution of arithmetic problems, rather
than forcing it to discover possible solution strategies only by trial-and-
error. Our work is similar in spirit to recent scientific endeavors directed
towards the design of learning systems that are capable of the kind of
systematic generalization that is required to execute algorithmic procedures
[12, 13]. The main challenge of the problem is to discover an algorithm that
can be used to solve arithmetic problems, and thus being able to generalize to
never-seen instances of the same class of tasks. In a deep RL framework, this
even more difficult since rewards could become very sparse due to the length
of the solution procedures.

In particular, we are interested in addressing these two key questions: Is it
possible to learn to solve mathematical problems that require long-term
planning by only relying on model-free RL? If not, what is the minimal amount
of guidance (in the form of learning biases and/or explicit supervision) that
is necessary to successfully solve such problems?

We find that our agent is able to solve the sum and subtraction problems with
a considerable capacity of out-of-distribution (OOD) generalization over the
length of the operands. At the same time, our simulations shed light on
difficulties in solving mathematical problems with model-free RL, especially
when learning requires to plan in the very distant future or to discover
solution strategies in which simple steps should be combined to solve
arbitrarily complex problems. By systematically analysing the errors made by
the agent in the OOD generalization regime, we also provide some intuitions
about the functioning and limitations of the proposed learning framework.

##  2 Methods

In this section we describe the task to be solved, the design of the
environment with which the agent interacts, the agent architecture, and the
training procedure.

###  2.1 Task

The task of the agent is to compute arithmetic operations between integers,
which are provided as a sequence of input symbols that can be either an
operation (plus or minus) or an operand. Operands are represented as sequences
of digits: during training, the length of the operand is sampled uniformly in
$\\{1,2,3,4,5,6\\}$, and each digit is sampled uniformly in $\\{0,1,2,3,4\\}$,
except for the first digit which cannot be 0. The first operand of any
operation is always the one represented in the current state of the abacus,
whose initial configuration represents the value 0.

Figure 1: The environment consists of a simulated abacus with 10 columns, a
padding on the left and a periodic positional encoding on top. The agent
observes the abacus through a sliding window composed of two columns,
interacts with it using an ‘operating finger’ and uses a ‘signpost’ to mark
the column where the current operation is going on.

###  2.2 Environment

The learning environment is conceived as a simulated abacus, featuring 10
columns with 5 positions for each column: that is, we represent numbers in
base 5 (see Fig. 1). On the top edge of the abacus there is an additional row
representing a periodic positional encoding, which allows to associate each
column with a value in the sequence [0.25, 0.5, 0.75, 0.25 …]. Such additional
input allows to effectively encode the position of the fingers in the abacus
(as explained below), thus improving learning speed and generalization.

The agent can manipulate the abacus using an ‘operating finger’, which serves
as a pointer positioned over the abacus at a given location at every time
step. Furthermore, the agent can use an indicator (dubbed the ‘signpost’) that
can be used to signal the column where the current operation is occurring, and
where the operation must resume once a carry operation is over.

The agent partially observes the abacus through a sliding window composed by
two columns: the one where the operating finger is, and the one to its left.
If the signpost is in the sliding window, the agent can see it in
superposition to the positional encoding. In case the operating finger is on
the first column, the agent observes a padding instead of the column to the
left of the operating finger.

The agent can interact with the environment performing one of the following
actions: a movement of the operating finger in the four directions, a movement
of the signpost along two directions (left, right), a slide of the beads in
the column where the operating finger is currently positioned (dubbed the
‘move and slide’ action) and an action to signal that it has finished
processing the current digit (dubbed the ‘submit’ action).

When the agent acts, the environment changes its state and gives as output the
reward and a flag indicating whether the current episode is over. At each time
step, the agent receives as input either an operation symbol or the next digit
that should be processed, both represented as one-hot vectors. The operation
is also signalled using a flag throughout the duration of the operation. The
episode ends if the abacus reaches its maximum representational capacity, or
if terminated early by the environment (see section 2.4).

###  2.3 Agent’s architecture

We simultaneously train an actor and a critic network. Both are memory-less
feed-forward networks and thus receive as input a stack of the last 3
observations, processed by a feature extractor implemented as a 3D
convolutional network with three layers of 128, 255 and 512 channels,
respectively. We use kernels of size 3 in the first layer and size 2 in the
last two, all with padding of 1. The output of the feature extractor is then
concatenated with the one-hot encoded symbol received from the environment;
the critic also receives as input the previous action. Both networks then
process this input via 5 feed-forward layers with 2048, 1024, 512, 256 and 128
neurons, respectively111We have chosen all hyper-parameters for the feature
extractor and the feed-forward networks starting from small architectures and
increasing their size until we achieved a satisfactory performance.. Finally,
the actor network returns a probability distribution over the actions, while
the critic returns an estimate of the value of each action.

###  2.4 Reward function

We define specific algorithms to solve the sum and subtraction problems using
the virtual abacus, and use such ideal solutions to provide supervision to the
learning agent (pseudo-code is provided in Algorithm 1)222We do not report the
subtraction algorithm since it only differs in lines 4, 7, 13 and 14 where we
implement the actual operation and check if a carry is necessary.. In order to
encourage the agent to learn such algorithms, we designed a modular reward
function that makes it possible to provide an increasing amount of supervision
through the following feedbacks:

  * •

A penalty of -0.05 for each action performed, to discourage long sequences of
actions.

  * •

A reward of 0.10 whenever a movement action (left, right, down, up) moves the
operating finger into a position that is closer to the target algorithmic
solution. A penalty of -0.10 is given if the opposite happens.

  * •

A reward of 1 if the signpost correctly moves to the next position when a
partial sum is done, or when the agent correctly resets the signpost.

  * •

A reward of 1 for correct move and slide.

  * •

A reward of 1 when the agent chooses the submit action, and the abacus
configuration represents the correct partial result.

  * •

If the agent does any of the previous three actions in the wrong way, the
episode is terminated and the agent is given a penalty of -1.

Algorithm 1 Addition algorithm. $c_{r}$ and $c_{l}$ are the two columns (left
and right) visible to the agent in the sliding window. $S(c)$ is a function
that returns the symbol encoded in a column.

1:while not done do

2: Read symbol $s$ from environment

3: if $s$ is digit then

4: Write $(S(c_{r})+s)\%5$ on $c_{r}$

5: if signpost in $c_{l}$ then

6: Move signpost right

7: else if $S(c_{r})+s\geq 5$ then

8: carry $\leftarrow True$

9: else

10: carry $\leftarrow False$

11: while carry $=True$ do

12: Move operating finger right

13: Write $(S(c_{r})+s)\%5$ on $c_{r}$

14: if $S(c_{r})+1\geq 5$ then

15: carry $\leftarrow True$

16: else

17: carry $\leftarrow False$

18: while signpost not in $c_{l}$ do

19: Move operating finger left

20: else$\triangleright$ Reset the abacus

21: while $c_{l}$ is not padding do

22: Move operating finger left

23: while signpost not in $c_{r}$ do

24: Move signpost left

We determine the maximum length of an episode dynamically, in order to avoid
long loops of meaningless actions. The agent is given 32 timesteps for each
correct move and slide action - i.e., the maximal theoretical distance between
any two possible move and slide actions in a solution trajectory according to
the reference algorithm. This mechanism grants the agent enough timesteps to
complete the operations that have long trajectories (i.e. resetting the abacus
at the end of an operation or computing a long carry), while also allowing the
agent to explore the functioning of the virtual abacus during training.

###  2.5 Model training

We use the Proximal Policy Optimization (PPO) learning algorithm with a
linearly-decaying sinusoidal learning rate, clipped surrogate objective
funcion [14], frame stacking [15] and masking [16, 17], as implemented in the
Python framework Stable Baselines3 [18]. We have also tried to apply the DQN
learning algorithm [15] with frame stacking and the same learning rate decay
scheme we used with PPO. However, we observed an extremely slow speed of
convergence in all reward settings, and especially with the ones having sparse
rewards.

Past information is provided by feeding the last 3 environment states stacked
into a 3D tensor. We mask illegal actions, e.g. moving left on the left edge
of the board, or actions that do not produce any effect on the environment,
such as using move and slide when the abacus is already in the target
configuration. We implemented an early stopping criterion, whereby learning is
interrupted if the KL divergence between the old policy and the new one is
greater than a threshold $\alpha=0.2$ that was empirically chosen.

##  3 Results

In this section we describe the simulation results and we provide some
insights in the way the agent works by analysing its failures when probed to
solve problems involving integers longer than the ones seen during training.
333We make the code that was used to run the experiments publicly available at
this GitHub repository: https://github.com/ChenEmmaL/imitation_abacus.

###  3.1 Solving arithmetic tasks

We designed the simulations with the aim of studying the level of supervision
that is necessary to successfully learn the arithmetic task. To this aim, we
trained the agent with varying amounts of reward: in the simplest case, we
included all components of the modular reward function. We then removed the
components of the reward related to the movement of the operating finger (OF)
and those related to the movement of the signpost (SP). The first component
provides very frequent but not strictly necessary supervision, as the agent
can discover the correct movement of the operating finger by exploration. The
second component provides important information to the agent, in that the
correct movement of the signpost is necessary to solve operations that require
(possibly very long) carries.

Figure 2: Learning performance of models trained with varying amounts of
supervision (oscillations are due to the sinusoidal learning rate). We measure
accuracy as the fraction of operations correctly computed in one epoch.

As shown in Fig. 2, the agent is able to solve the task in all cases, reaching
an almost perfect accuracy. Surprisingly, reducing the amount of supervision
and letting the agent discover the most effective way to use the operating
finger leads to a faster training and also higher performance in terms of
number of consecutive operations successfully computed (see Table 1). Further
reducing the level of supervision causes an initial learning slowdown but
still allows to reach a very high accuracy later on, although the final
performance in terms of number of consecutive operations is lower compared to
the intermediate level of supervision.

Next, we have removed the reward for correct move and slide actions, observing
that learning becomes so slow that the performance at every epoch is barely
improving 444We did not remove the penalty for long trajectories as it shapes
the behavior of the agent only indirectly. We also did not remove the
component of the reward that signals if a partial operation was successfully
completed, as it provides the agent with the most important signal about the
completion of the task.. Therefore, we find that in order to learn the
algorithm in a reasonable time, the agent needs to receive the following
essential feedback: the reward for correct move and slides, the reward for
correct partial answers given using the ‘submit’ action, the penalty for long
trajectories, and the penalty (including episode termination) in case of wrong
move and slides or partial answers.

We also trained the agent on each arithmetical task separately, using the
highest level of supervision. As expected, learning a single task is simpler,
both in terms of learning speed (Fig. 3) and number of successful consecutive
operations (Table 1).

Figure 3: Learning performance of the models trained on sum only or
subtraction only with the dense reward function.

###  3.2 Generalizing to longer operands

Learning to solve arithmetic operations with operands ranging beyond the
intervals encountered during training is one of the main challenges for deep
learning models [8]. We thus investigated the capability of our best
performing agent, namely the one trained on both tasks with an intermediate
level of supervision555Note that in this case we extend the abacus to 20
columns in order to represent the extended range of operands. by sampling the
two test operands in the intervals $[5^{x-1},5^{x})$, where
$x\in\\{1,2,4,8,16\\}$.

For each interval, we sampled 100000 operands and counted the number of
mistakes made by the agent. Although the trained agent does not exhibit
perfect OOD generalization capabilities and its error rate grows with the
number of digits involved in the operations (see Fig. 4), it can still solve
sums and differences involving operands in intervals unseen during training
with an arguably low error rate (e.g., when operands involved more than two
times the amount of digits observed during training, the error rate was still
below 1%).

Model | N. op.
---|---
Dense reward | 515
No OF | 1145
No OF + SP | 773
Addition | 907
Subtraction | 675
Table 1: Maximum number of operations successfully completed during training
in the different training scenarios. Figure 4: Error rate of the model on
operations involving longer operands than those seen during training, which
contained at most 6 digits.

###  3.3 Analysis of errors

We analyzed the pattern of errors made in the most extreme OOD regime, that
is, when operands were sampled in the interval $[5^{15},5^{16})$, by
collecting statistics over 100000 simulations. The agent commits 601 errors,
which is coherent with the error rate reported in Fig. 4. We recorded the
errors and manually compiled a list of 4 different error classes: errors in
the movement of the signpost, errors during a carry operation, and errors
occurring during ‘simple’ addition or subtraction operations. The latter kind
of errors might in fact include operations that require a carry; we only
selected errors that did not happen when performing the carry operation (e.g.
removing one unit from the column to the right and fill the current column)
but instead occurred when the carry was completed and the agent needed to
compute a sum or subtraction.

Error class | %
---|---
Simple operation | 32.9
Signpost right | 27.7
Carry | 27.3
Signpost left | 12.1
Table 2: Relative frequency of error types. A simple operation is a sum or
subtraction between two digits. The Carry and Signpost right classes occur
when the agent must compute a carry operation.

As shown in Table 2 the most frequent kind of error is the one involving a
simple sum or subtraction operation. The second and third most common classes
of errors are wrong carries or movements of the signpost rightwards: notice
that such a movement is required precisely when the agent must perform a carry
operation. These results reflect the fact that the carry operation is the most
complicated step in multi-digit sum or subtraction. Lastly, the least frequent
kind of error is the one involving the movement of the signpost to the left,
indicating that the agent has learned to reset the abacus to the initial
position almost perfectly.

Since our system includes a sliding window mechanism that limits the capacity
of the agent to observe the abacus, we investigated whether errors are equally
distributed on all columns. The histogram in Fig. 5 representing the frequency
of errors by column shows that most errors occur on the second column, which
is the first one observed outside the initial position of the sliding window.
This is consistent with our previous observation that many errors occur during
a carry operation, which requires to move the sliding window to the right. We
can also observe a periodic pattern of errors from the third column onward,
most likely due to the specific choice used in the positional encoding. This
reveals that, although this element of the environment contributes to the
capacity of the agent to generalize to unseen ranges of operands, it also
introduces a regularity in the errors which depends on the column where an
operation must be computed.

Finally, in Table 3 we report a few significant examples of errors made by our
system in the OOD generalization regimen. Consistently with the previous
analysis, we can see that for both sums and subtractions the agent can make a
mistake in the early positions (first columns) as well as in the last ones.
Also, it is evident that an error in a carry or regrouping can propagate to
the following columns. However, it can also happen that the system is
resilient to such mistakes, and thus still compute the rest of the operation
correctly: this is a desirable property, since it avoids propagating errors
and thus keeps the absolute value of the error relatively low.

Figure 5: Relative frequency of errors by column of the abacus for the best model. Most errors happen on the second column, and a periodic pattern emerges from the third column onward. S | 2204010440402424 | 3014100322010344
---|---|---
I | +3422111404102300 | +1342122200324413
O | 11131122400010024 |  3014100322340312
T | 11131122400010224 | 4411223022340312
S | 4110012024223441 | 3342443241400324
I | -1300014214322224 | -3231303122242113
O |  4041442304401212 | 111040114103211
T | 2304442304401212 | 111140114103211
Table 3: Examples of errors. We report the state of the abacus (S), operation
input to the system to be executed (I), output produced (O) and true output
(T). Operands are in base 5 to facilitate the interpretation of mistakes in
carries and regroupings.

##  4 Discussion

In this work we introduced a framework that can be used to study how an agent
can learn to solve arithmetic operations by exploiting deep reinforcement
learning and by interacting with external representations that incorporate the
working principles of an abacus. Our framework is inspired by the way humans
interact with external representational tools to solve mathematical problems,
connecting to the more general trend of exploring tool use in deep
reinforcement learning environments [19, 20, 21]. Differently from similar
problems in the deep RL literature, such as learning to play combinatorial
games, the problem we propose is characterized by an algorithmic nature, in
that the goal of the agent is learning an exact solution algorithm to
arithmetic problems, rather than a strategy to win a game.

Our simulations suggest that in order to learn to solve arithmetic problems
with model-free reinforcement learning, a memory-less agent needs to receive a
certain amount of explicit supervision to overcome its inability to plan in
the long-term. Notably, the agent we present is able to solve problems
involving operands that are well outside the training range, which can be
considered the main challenge of the class of arithmetic problems we propose.
The agent is able to do so thanks to specific learning biases: a sliding
window, a positional encoding, and the representation format of the operands
on the abacus.

It might be possible let the agent fully observe the virtual abacus and learn
which part is relevant in any given moment; however, by adopting a relative
view on the learning environment through the sliding window the agent can
effortlessly generalize the strategy learned on a limited interval of operands
to ones that are more than two times longer. At the same time, it turned out
that including elements contributing to generalization capability, such as the
periodic positional encoding, also introduced unwanted regularities in the
errors committed by the system.

An exciting venue for future research would be to explore the possibility to
endow the agent with some form of memory (e.g., by exploiting recurrent
architectures), which would allow to plan in the distant future and thus learn
to solve the problem with even less supervision. Furthermore, it would be
interesting to exploit model-based and hierarchical reinforcement learning
approaches to endow the agent with native capability to internally simulate
the functioning of the external tool and learn to compose simple solution
steps into more complex strategies.

## References

  * Lake et al. [2017] Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J. Gershman.  Building machines that learn and think like people.  _Behavioral and Brain Sciences_ , 40:e253, 2017.  10.1017/S0140525X16001837.
  * Testolin [2020] Alberto Testolin.  The challenge of modeling the acquisition of mathematical concepts.  _Frontiers in Human Neuroscience_ , 14, 2020.  ISSN 1662-5161.  10.3389/fnhum.2020.00100.  URL https://www.frontiersin.org/articles/10.3389/fnhum.2020.00100.
  * Saxton et al. [2019] David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli.  Analysing mathematical reasoning abilities of neural models.  In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net, 2019.  URL https://openreview.net/forum?id=H1gR5iR5FX.
  * Lewkowycz et al. [2022] Aitor Lewkowycz, Anders Johan Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Venkatesh Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra.  Solving quantitative reasoning problems with language models.  In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, _Advances in Neural Information Processing Systems_ , 2022.  URL https://openreview.net/forum?id=IFXTZERXdM7.
  * Trask et al. [2018] Andrew Trask, Felix Hill, Scott E Reed, Jack Rae, Chris Dyer, and Phil Blunsom.  Neural arithmetic logic units.  In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, _Advances in Neural Information Processing Systems_ , volume 31. Curran Associates, Inc., 2018.  URL https://proceedings.neurips.cc/paper/2018/file/0e64a7b00c83e3d22ce6b3acf2c582b6-Paper.pdf.
  * Madsen and Johansen [2020] Andreas Madsen and Alexander Rosenberg Johansen.  Neural arithmetic units.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net, 2020.  URL https://openreview.net/forum?id=H1gNOeHKPS.
  * Kaiser and Sutskever [2016] Lukasz Kaiser and Ilya Sutskever.  Neural gpus learn algorithms.  In Yoshua Bengio and Yann LeCun, editors, _4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings_ , 2016.  URL http://arxiv.org/abs/1511.08228.
  * Cognolato and Testolin [2022] Samuel Cognolato and Alberto Testolin.  Transformers discover an elementary calculation system exploiting local attention and grid-like problem representation.  In _International Joint Conference on Neural Networks_ , 2022.  10.1109/IJCNN55064.2022.9892619.
  * Wang et al. [2018] Lei Wang, Dongxiang Zhang, Lianli Gao, Jingkuan Song, Long Guo, and Heng Tao Shen.  Mathdqn: Solving arithmetic word problems via deep reinforcement learning.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 32(1), Apr. 2018.  10.1609/aaai.v32i1.11981.  URL https://ojs.aaai.org/index.php/AAAI/article/view/11981.
  * Huang et al. [2018] Danqing Huang, Jing Liu, Chin-Yew Lin, and Jian Yin.  Neural math word problem solver with reinforcement learning.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 213–223, Santa Fe, New Mexico, USA, August 2018\. Association for Computational Linguistics.  URL https://aclanthology.org/C18-1018.
  * Kaliszyk et al. [2018] Cezary Kaliszyk, Josef Urban, Henryk Michalewski, and Miroslav Olšák.  Reinforcement learning of theorem proving.  In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, _Advances in Neural Information Processing Systems_ , volume 31. Curran Associates, Inc., 2018.  URL https://proceedings.neurips.cc/paper/2018/file/55acf8539596d25624059980986aaa78-Paper.pdf.
  * Velickovic et al. [2020] Petar Velickovic, Rex Ying, Matilde Padovano, Raia Hadsell, and Charles Blundell.  Neural execution of graph algorithms.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net, 2020.  URL https://openreview.net/forum?id=SkgKO0EtvS.
  * Schwarzschild et al. [2021] Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, and Tom Goldstein.  Can you learn an algorithm? generalizing from easy to hard problems with recurrent networks.  In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, _Advances in Neural Information Processing Systems_ , volume 34, pages 6695–6706. Curran Associates, Inc., 2021.  URL https://proceedings.neurips.cc/paper/2021/file/3501672ebc68a5524629080e3ef60aef-Paper.pdf.
  * Schulman et al. [2017] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.  Proximal policy optimization algorithms, 2017.  URL https://arxiv.org/abs/1707.06347.
  * Mnih et al. [2013] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller.  Playing atari with deep reinforcement learning, 2013.  URL https://arxiv.org/abs/1312.5602.
  * Vinyals et al. [2017] Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev, Alexander Sasha Vezhnevets, Michelle Yeo, Alireza Makhzani, Heinrich Küttler, John Agapiou, Julian Schrittwieser, John Quan, Stephen Gaffney, Stig Petersen, Karen Simonyan, Tom Schaul, Hado van Hasselt, David Silver, Timothy Lillicrap, Kevin Calderone, Paul Keet, Anthony Brunasso, David Lawrence, Anders Ekermo, Jacob Repp, and Rodney Tsing.  Starcraft ii: A new challenge for reinforcement learning, 2017.  URL https://arxiv.org/abs/1708.04782.
  * Huang and Ontañón [2022] Shengyi Huang and Santiago Ontañón.  A closer look at invalid action masking in policy gradient algorithms.  _The International FLAIRS Conference Proceedings_ , 35, May 2022.  10.32473/flairs.v35i.130584.  URL https://journals.flvc.org/FLAIRS/article/view/130584.
  * Raffin et al. [2021] Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann.  Stable-baselines3: Reliable reinforcement learning implementations.  _Journal of Machine Learning Research_ , 22(268):1–8, 2021.  URL http://jmlr.org/papers/v22/20-1364.html.
  * Baker et al. [2020] Bowen Baker, Ingmar Kanitscheider, Todor M. Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch.  Emergent tool use from multi-agent autocurricula.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net, 2020.  URL https://openreview.net/forum?id=SkxpxJBKwS.
  * Fleer and Ritter [2020] Sascha Fleer and Helge Ritter.  Solving a tool-based interaction task using deep reinforcement learning with visual attention.  In Alfredo Vellido, Karina Gibert, Cecilio Angulo, and José David Martín Guerrero, editors, _Advances in Self-Organizing Maps, Learning Vector Quantization, Clustering and Data Visualization_ , pages 231–240, Cham, 2020. Springer International Publishing.  ISBN 978-3-030-19642-4.  10.1007/978-3-030-19642-4_23.
  * Sabathiel et al. [2022] Silvester Sabathiel, Flavio Petruzzellis, Alberto Testolin, and Trygve Solstad.  Self-communicating deep reinforcement learning agents develop external number representations.  In _Proceedings of the Northern Lights Deep Learning Workshop_ , volume 3, 2022.  10.7557/18.6291.