Thermodynamic analysis of magnetic phase transitions in the ferromagnetic
superconductor UGe2 at ambient pressure

Diana V. Shopova

Institute of Solid State Physics, Bulgarian Academy of Sciences,

1784 Sofia, Bulgaria<EMAIL_ADDRESS>
###### Abstract

We study the possibility to apply phenomenological approach to the description
of magnetic transitions in UGe2 at ambient pressure with the help of Landau
free energy expanded to $8^{th}$ order in magnetisation. The analysis shows
that for certain values of parameters in front of $M^{4}$, and $M^{6}$ terms
in the free energy there is possibility for the appearance of two successive
phase transitions between the low-magnetisation and high-magnetisation phases
with the same magnetic structure. We establish the relation of the parameters
in Landau energy, for which the phase transition from the disordered to low-
magnetisation phase is of second order as the experimental data shows. We have
also found the relations between the Landau coefficients which result in
transition of crossover type between the low- and high-magnetisation phases
when temperature is lowered.

Keywords: magnetic phase transition, ferromagnetic superconductor, Landau free
energy

PACS: 74.20.De, 74.20.Rp, 74.40-n, 74.78-w

##  1  Introduction

It is established that in Uranium-based ferromagnetic superconductors UGe2,
URhGe, and UCoGe [1], [2], [3], the same $5f$-electrons are responsible for
the occurrence of both ferromagnetism and superconductivity. The first
discovered ferromagnetic superconductor is UGe2, where the superconductivity
appears only under pressure deep inside the ferromagnetic phase at low
temperature. There exist two different ferromagnetic phases with same magnetic
structure and different magnetisations. The crystal structure of UGe2 is
orthorhombic where the U-atoms form coupled zigzag chains along the a-axis and
the magnetic moments are aligned along the direction of the chain. In (1) the
schematic pressure-temperature phase diagram for UGe2 is shown, and recent
summarized experimental diagram may be found in [4]. The transition at ambient
pressure from paramagnetic to weakly polarized ferromagnetic phase (FM1)occurs
at Tc = 52.6 K through second order phase transition with
M${}_{0}=0.9\mu_{B}$/U. With the increase of pressure, the transition changes
from second to first order at tricritical point T${}_{CP}\simeq 22$ K,and
$P_{CP}\approx 1.42$ GPa and ferromagnetism and superconductivity disappear at
pressure $\sim 1.5$ GPa and T=0.

Figure 1: The schematic temperature-pressure phase diagram of UGe2. FM1 and
FM2 describe the low- and high-magnetisation phases, respectively. The blue
curve and the area below it describe the superconducting phase.

When temperature is lowered a second ferromagnetic phase (FM2) appears with
the same magnetic structure and stronger magnetic polarization
-1.41$\mu_{B}/$U [9] at T${}_{x}\approx$ 30 K. (see (1)). It is experimentally
found that at ambient pressure the transition between FM1 and FM2
ferromagnetic phases is of crossover type and only with increasing pressure
there is a true first order phase transition close to the appearance of
superconductivity with a maximum transition temperature of $\approx 0.7$ K at
px. The superconductivity exists in a limited pressure range between 1.0 and
1.5 GPa. In this pressure range, the magnetic moment is $1\mu_{B}/U$ and the
transition between two ferromagnetic phases is of first order at the critical
end point (CEP: T${}_{CEP}\sim 7$ K, P${}_{CEP}\sim 1.16$ GPa).
Experimentally, it is established that UGe2 has very strong uniaxial magnetic
anisotropy [6], and at some earlier stages of study, it has been considered
that 3D Ising model can well describe its magnetic properties - see, for
example, [7], and for recent review [8]. It is experimentally found and
theoretically justified [9] that the critical exponents for UGe2 do not belong
to any known universality class including 3D Ising model and anisotropic next
nearest neighbor 3D Ising model. One of the reasons for this lays in the
dualism of 5f-electrons in UGe2 where two subsets of itinerant and localised
ones exist. This fact is experimentally established by positive muon
relaxation measurements where the itinerant magnetic moment at ambient
pressure is found to be $\approx$ 0.02$\mu_{B}$ [10]. Later this dualism of
5f-electrons is confirmed by magnetic, transport and specific heat
experiments, see, for example, [11].
Understanding of magnetic phase transitions from microscopic point of view is
important for understanding the appearance of superconductivity - for review,
see for example [5], but there is now up to now consensus on this problem.
Recently, an estimate is made on the basis of Takahashi’s spin fluctuation
theory about the itinerancy of 5f-electron systems in actinides [12]. The
comparison of this phenomenological approach with the experimental data shows
that UGe2 within the criterion used in the paper is an intermediate case
between strongly localised and itinerant 5f-electron system. In the paper [4],
measurements of magnetization are performed at high pressure for UGe2 with the
aim to make clear the correlation between the superconductivity and pressure-
enhanced ferromagnetic fluctuations. From the experimental data a conclusion
is made that at lower pressure the ferromagnetic state is more itinerant in
FM1 compared to FM2.
Usually microscopic models are focused on the phase transition between FM1 and
FM2 around the critical end point, where superconducting phase appears under
pressure [13] although at ambient pressure this problem is not yet resolved,
especially of the nature of crossover transition between FM1 and FM2. In
earlier studies, it is proposed that the electronic density of states has two
very closely placed peaks near the low-dimensional anisotropic Fermi surface,
whose topology changes by the appearance of magnetisation [14]. This
calculation is performed at T=0 with the help of Stoner theory for itinerant
magnets. There the authors claim than within mean-field approximation their
theory will be in analogy with Shimizu paper [15] for the metamagnetic
transitions under external magnetic field by including the $M^{8}$ term in the
free energy. The experimentally established first order FM1$\rightarrow$FM2
around $p_{x}$ is also considered as metamagnetic, but in UGe2 this
metamagnetism exhibits some peculiar features, for example Tx $\ll$Tc and no
reorientation or reversal of spin occurs, but only change in the magnitude of
magnetic moment.

In this paper we will consider the ferromagnetic phase transitions in UGe2 at
ambient pressure in the absence of external magnetic field. There are few
experimental data in this regime, see for example the experiments on linear
thermal expansions of single-crystalline UGe2, [16] at ambient pressure.
Usually the transition between FM1 and FM2 at pressures lower than 1 GPa down
to $p=0$ is considered as crossover, see, for example [4] and [8] and not a
real thermodynamic phase transition. At ambient pressure there are
experimentally observed singularities at $T_{x}=30$ K in different physical
quantities, for example broad hump in the specific heat, see [1] and the
papers cited therein. But systematic measurement at ambient pressure which may
shed light on the nature of FM1$\rightarrow$FM2 crossover at $p=0$ are
missing. Special attention can be paid to positive muon spin rotation
measurements of paper [17] where the authors claim that at pressures 1.00 GPa
and below the FM1$\rightarrow$FM2 corresponds to a real thermodynamic phase
transition, but there the explanation of obtained experimental data at low and
especially at ambient pressure is not quite definite.
In view of the above arguments we will adopt here the hypothesis that at
ambient pressure and zero magnetic field the transition between two
ferromagnetic phases in UGe2 is of isostructural nature as defined in [18] and
later on generalised see, for example [19], [20]. This means that the phase
transition may occur between two ordered phases with the jump of order
parameter without changing the structure (for example, the change of volume
and/or valence without change of crystal symmetry). From the general theory
for isostructural phase transitions, they can be described by including in the
expansion of Landau free energy, a term to the 8th order in magnetization,
namely, M8. For this free energy expansion the phase diagram is rich and
proposes different solutions depending on the values of coefficients in the
Landau expansion. Here we find out those relations between the coefficients
which can lead to the experimentally observed crossover between FM1 and FM2 in
UGe2, and the phase transition from paramagnetic to FM1 phase is of second
order. Such an approach gives further possibility to include in calculations
the influence of external pressure by supposing the model dependence of Landau
coefficients on it.

##  2  Theoretical approach

As pointed in the previous section the magnetization in UGe2 is highly
anisotropic due to localized 5f-electrons. The a-axis is the easy
magnetization axis along which $M_{a}$, coming from both itinerant and
localised 5f-electrons has the greatest value. The magnetic moment coming from
itinerant 5f-electrons is isotropic and, in principle, also the perpendicular
part of magnetisation ($M_{b}$, $M_{c}$), where b and c are the crystal axes,
should be taken into account.
As a first step in considering the phase transitions between the paramagnetic
phase and FM1 and FM2 at ambient pressure we will not take into account the
transverse magnetisation as small compared to the magnitude of magnetisation
along $a$-axis - $\sim 0.02$ as experiment shows, and will expand the Landau
free energy $F(M)$ in $\overrightarrow{M}=(0,0,M_{a})$, the subscript $a$ will
be omitted hereunder. Then $F(M)=f(M)V$ where $V$ is the volume and the free
energy density $g(M)$ is given by the expression:

$g=aM^{2}+\frac{b}{2}M^{4}+\frac{c}{3}M^{6}+\frac{v}{4}M^{8}.$ (1)

In the above equation, $a=\alpha(T-T_{c})$, where $T_{c}$ is the Curie
temperature and $\alpha$ is a material parameter. The other coefficients in
the Landau expansion, namely, $b,\;c,\;v$ are considered at this stage not
dependant on temperature and external pressure, namely
$b=b(T_{c},P_{0}),\;c=c(T_{c},P_{0}),\;v=v(T_{c},P_{0})$; here by $P_{0}$ we
denote the ambient pressure. The sign of $b,\;c$ may be either positive or
negative, but $v>0$ in order to ensure convergence of $f$.
In the expression (1) there are too many unknown parameters, which in
principle may be derived from experiment in combination with respective
microscopic calculations at ambient pressure. In order to reduce this number,
as well as to make free energy density $f$ dimensionless we will redefine the
order parameter $M$ by introducing:

$m=v^{1/8}M.$ (2)

Then the free energy density - Eq. (1) in dimensionless form becomes:

$f=tm^{2}+\frac{u}{4}m^{4}+\frac{w}{3}m^{6}+\frac{1}{4}m^{8};$ (3)

and the coefficients in Eq.(3) are related to the initial ones in the
following way:

$\displaystyle u$ $\displaystyle=$ $\displaystyle\frac{b}{v^{1/2}}$
$\displaystyle w$ $\displaystyle=$ $\displaystyle\frac{c}{v^{3/4}}$

The reduced temperature is $t=\beta(T/T_{c}-1)$ with $T_{c}$ \- the Curie
temperature, $\beta=\alpha T_{c}/v^{1/4}$.
The equation of state $(df/dm)$ :

$2m(t+um^{2}+wm^{4}+m^{6})=0$ (4)

has an obvious solution for disordered (paramagnetic) phase $m=0$.
The stability condition is given by inequality:

$\frac{d^{2}f}{dm^{2}}=2(t+3um^{2}+5wm^{4}+7m^{6})\geq 0$ (5)

and there are several methods to resolve this condition in analytical form:
see for example [19]. This may be also done numerically by direct substitution
of solutions of Eq. (4) in the above equation and analysing its positiveness.
If more than one solution is stable also a comparison between free energies of
respective solutions of Eq. (4) should be made in order to find out which one
in what domain of reduced temperature is an absolute minimum.
If we substitute $x=m^{2}\geq 0$, the equation (4) will become standard 3-rd
order algebraic equation, its solutions can be given in analytical form, see,
for example [21].
Eq. (4) expressed by new variable $x$ reads:

$t+ux+wx^{2}+x^{3}=0.$ (6)

The number of real solutions for $x=m^{2}$ is determined by the sign the
quantity

$Q=\frac{t^{2}}{4}+\frac{2w}{3}(\frac{2w^{2}}{9}-u)t+\frac{u^{2}}{27}(u-\frac{w^{2}}{4})$
(7)

The quantity $Q(t)$ which is a quadratic equation with respect to the reduced
temperature, $t$ as function of parameters $u,w$ with solutions $t_{1,2}$
given by:

$t_{1,2}=\frac{w}{3}(u-\frac{2}{9}w^{2})\pm\frac{2}{27}(w^{2}-3u)^{3/2}$ (8)

also determines the region of existence of non-negative solutions for
magnetisation as function of parameters $u$ and $w$ in Landau energy. It is
obvious that $t_{1,2}$ are real for $w^{2}\geq 3u$ and $t_{0}=w^{2}/(3u)$ is a
special point, for which $t_{1}=t_{2}=w^{3}/27$. Q may also be written as

$Q=(t-t_{1})(t-t_{2}).$ (9)

Independent of the sign of $w$, $t_{1}>t_{2}$; then for $t$ between t1,2,
Q$<0$ and Eq. (6) will have three real solutions; for $t>t_{1}$, and
$t<t_{2}$, Q$>0$ and there are one real solution and two complex conjugate
solutions; for Q$=0$ \- three real solutions, two of them are equal. In
addition, the real solutions should be non-negative as $x=m^{2}$ and this
strongly depends both on sign and magnitude of parameters $w$ and $u$.
We denote the solutions of Eq. (6) by $x_{1},\;x_{2},\;x_{3}$.
The description of all solutions of equation of state, Eq. (4), is very well
illustrated in the parameter space $(u,t)$ for fixed value of coefficient $w$.
We show graphically in Fig. (2) the dependence of $t_{1,2}$ on parameter $u$,
for fixed parameter $w<0$. In this case $t_{2}<0$; $t_{1}$ is positive and
intersects the $u$-axis at $u=w^{2}/4$.

Figure 2: The solutions $t_{1,2}$ of quantity Q as function of the parameter
$u$. The dashed vertical curve is drawn at $u=\frac{w^{2}}{4}$ marked by point
C in the figure. Point B is at $u=\frac{w^{2}}{3}$.

The $(u,t)$ diagram for $w>0$ is mirror image of Fig. (2) as there
$t_{1}(w<0)=-t_{2}(w>0)$, and $t_{2}(w<0)=-t_{1}(w>0)$; the point B is above
$u$-axis, and $t>0$.
We will not consider positive values of parameter $w$ as the analysis shows
that for $w>0$, and $u>0$, no isostructural transition appears and only one
ferromagnetic phase exists through second order transition from the para
phase. In this case to expand Landau free energy up to $M^{8}$ is redundant.
For $w>0$ and $u<0$ the ferromagnetic phase occurs through first order phase
transition from para phase and again no isostructural transition may occur.
The experiment shows that in UGe2 the phase transition para$\rightarrow$ FM1
is of second order at ambient and low pressures. The order of phase transition
to FM1 changes from second to first only under increasing pressure, passing
through tricritical point (see Fig. (1), point (TTCP)).
Here we will make some remarks on the appearance of tricritical point TTCP on
the transition line between disordered phase and FM1 with the increase of
pressure from zero to $P_{c}$, at which the ferromagnetism disappears. Within
the Landau approach we use this means change of parameter $u$ with pressure
from positive to negative value. At the tricritical point this parameter turns
zero, see [20]. In order to describe the change of order of transition
para$\rightarrow$FM1 with pressure, a supposition should be made on the
dependence of $u$ on pressure based on microscopic picture of behavior of
f-electrons with pressure.
Another theory shows that the effects of gapless particle-hole excitations at
the Fermi surface for itinerant ferromagnets induce a nonanalytic term of
logarithmic type in the Landau expansion of the free energy as a function of
the magnetic moment M [22]. When applied to magnetic transitions in UGe2 [23]
this theory explains the appearance of tricritical point and the first order
transition at low temperature, as well as the second order transition at
higher temperatures. In a recent review on the U-based ferromagnetic
superconductors [8] different theoretical approaches to the UGe2 phase diagram
under pressure are presented. The discussion of dependence of magnetic phase
transitions in UGe2 on pressure is outside the scope of our paper.
Further we will consider only $w<0$, for which two ferromagnetic phases may
appear with the decrease of temperature. The calculations show that the
solution $x_{1}$ of equation of state (6) is always positive and bigger than
$x_{2}$; $x_{2}$ may be negative for $t>0$ and positive for $t<0$. This is
why, for $w<0$ we consider that $x_{1}$ describes the high-magnetization phase
and $x_{2}$ \- the low-magnetization phase. The solution $x_{3}$ is positive
and unstable. When $u\leq 0$ and $w<0$, $x_{1}$ exists and is positive even
for $t>0$, $x_{2}$ is negative, $x_{3}$ is positive but metastable in the
region between $t_{1}$ and $t_{2}$ according to the stability condition, Eq.
(5) and no isostructural transition can take place there. The phase transition
from paramagnetic to ferromagnetic phase is of first order.
The calculations show that region of interest in the $(u,t)$ phase diagram,
see Fig. (2) is where $Q\leq 0$, for negative values of coefficient $w<0$
before $m^{6}$, see Eq.(3) and positive values of coefficient $u$ before
$m^{4}$ as far as there is possibility for the occurrence of transition
between two magnetic phases with same structure.
We will describe in some detail different points and lines in Fig. (2). The
violet line in the figure denotes the equilibrium line $t_{eq}$ of first order
isostructural phase transition and is determined by making equal the free
energies $f$, Eq. (3), of low- and high-magnetisation phases
$f(m_{1})=f(m_{2})$. The equilibrium line
$t_{eq}=\frac{w}{3}\left(u-\frac{2}{9}w^{2}\right)$ crosses the $u$-axis at
$u_{E}=\frac{2}{9}w^{2}$, denoted by point E in Fig (4). For
$0<u<\frac{2}{9}w^{2}$, the high-magnetization phase ($x_{1}$) is more stable
than the low-magnetization phase ($x_{2}$) in the whole interval of its
existence and occurs through first order phase transition from para phase;
this transition is shifted to positive values of $t$. In this interval of $u$
and within our approximation the low-magnetization phase always has higher
free energy than the high-magnetization phase and is metastable.
For $u=u_{E}$ the high-magnetization phase is stable only for $t<0$ but
$f(x_{2})$ is higher than $f(x_{1})$ and no isostructural transition can occur
within our approximation. When $u>u_{E}$, the region of stability of $x_{1}$
is shifted to more negative values of $t$ and in the interval of $0<t\leq
t_{eq}$, $x_{2}$ has lower energy than $x_{1}$ and this interval grows with
the increase of $u$.
Another special point in Fig. (4) is at $u=w^{2}/4$, point C where $t_{1}$
crosses $u$-axis. There $x_{1}$ is non-negative only for $t\leq 0$, and its
stability is shifted to $t<0$. As far as the FM1 in UGe2 exists in a
relatively large temperature interval $\sim 22$ K, we are interested in those
values of $u$ where the low magnetization phase, described by $x_{2}$ exists
and is stable in relatively large interval of reduced temperature $t$. The
calculations show that this condition is fulfilled for $w^{2}/4<u<w^{2}/3$. In
this range of parameter $u$ the phase transition from the disordered to
ferromagnetic phase is of second order and when the temperature is lowered
isostructural transition of first order occurs.
The region of parameters $u$ where second order phase transition from
disordered phase followed by first order isostructural transition to high-
magnetization phase can occur is illustrated in Fig. (4) by green dashed line.

##  3  Results and discussion

The possibility for second order phase transition to low-magnetisation phase
followed by isostructural transition to high magnetisation phase includes the
values of $u$ at fixed $w$ in the interval $w^{2}/4<u\leq w^{2}/3$; points $C$
and $D$ in Fig.(2), respectively. The calculations show that within this
interval with exception of point $D$ there is real first order transition
between FM1 and FM2 and this contradicts to the experiments in UGe2 at ambient
pressure, which claim that there is not a true thermodynamic transition
FM1-FM2 at ambient pressure, but a crossover, see for example the review [8].
So we suppose that at ambient pressure the Landau expansion to $M^{8}$ can be
applied to description of FM1-FM2 transition if we adopt that within our
approach that the following conditions are fulfilled $u>0$, $w<0$ and
$u=w^{2}/3$ for the Landau coefficients in the free energy expansion Eq.(2).
This is illustrated in Fig.(2) by the line $SDB$. At $t=t_{D}=0$, the
transition of second order from para phase to low-magnetisation phase occurs.
At $t=t_{B}=w^{3}/27$ there is a crossover between FM1 and high-magnetisation
phase. FM1 exists and is stable in the interval $BD$ and for $t<t_{B}$, the
high-magnetisation phase is stable.
To be more explicit we will write down the expression of free energy for
$u=w^{2}/3$, denoted by $f_{c}$ and the respective equation of state:

$f_{c}=tM^{2}+\frac{1}{6}w^{2}M^{4}+\frac{1}{3}wM^{6}+\frac{1}{4}M^{8},$ (10)

and

$\frac{\partial f_{c}}{\partial
M}=\frac{2}{3}M(3t+M^{2}w^{2}+3M^{4}w+3M^{6})=0$ (11)

For $0<t<w^{3}/27$, there is one real positive solution $M_{1}$ for
magnetisation of Eq. (11) and in the temperature interval $t<w^{3}/27$ , the
real positive solution for magnetisation is $M_{2}$ ; the calculations show
that $M_{1}<M_{2}$. At the crossover reduced temperature, denoted by
$t_{cr}=w^{3}/27$, the equation of state, Eq. (11), becomes:

$\frac{1}{9}(3M^{2}+w)^{3}$

with the obvious solution for magnetisation $M^{2}=-w/3$ and all real positive
solutions for $M$ are equal. The dependence of magnetisation on the reduced
temperature is illustrated in Fig. (3).

Figure 3: The dependence of magnetisation $M$ on the reduced temperature $t$
for two values of the parameter $w$. The points marked by diamonds on the
curves denote the crossover transition between FM1 and FM2.

It is seen from the figure that around $t_{cr}=w^{3}/27$ there is a jump in
magnetisation of FM2 phase. This is reflected on the behaviour of free energy
as function of magnetisation for different values of reduced temperature $t$,
as shown in Fig. (4).

Figure 4: The dependence of free energy $f_{c}$ , Eq.(10) on magnetisation $M$
at different values of reduced temperature $t$. The curves are drawn for
$w=-1.2$.The orange curve is for $t_{cr}=\frac{w^{3}}{27}$; the point denoted
by diamond marks the value of $f$ for $M_{cr}=\sqrt{-3w}/3$. The green curve
is drawn for $0>t>t_{cr}$, and the violet line is for $t<t_{cr}$.

There is a very broad and flat minimum at $t_{cr}$, see the orange curve in
Fig.(4), where the diamond denotes the equilibrium magnetisation as a solution
of equation of state Eq. (11) at crossover transition. The minimum of $f_{c}$
for $0>t>t_{cr}$ gives the equilibrium value of $M$ for low-magnetisation
phase, and the respective minimum for $t<t_{cr}$ the equilibrium value of $M$
for high-magnetisation phase
We should mention here that the second derivative of free energy Eq. (10)
defines the stability of ferromagnetic phases. Taking into account the
equation of state, Eq. (10), the stability condition may be written in the
form

$-4w(M^{2}-M_{1}^{2})(M^{2}-M_{2}^{2})\geq 0$

with

$M^{2}_{1,2}=\frac{-w}{3}(1\pm\sqrt{1-\frac{27}{w^{3}}t})$ (12)

and it is satisfied when $M^{2}>M_{1}^{2}$ or $M^{2}<M_{2}^{2}$.
The calculations show that the high-magnetisation phase satisfies the first
condition and the low-magnetisation phase - the second one within the
respective temperature intervals. For $M_{cr}$ the stability condition is
equal to zero. The stability conditions are illustrated graphically in Fig.
(5) for $w=-1.2$. The area encircled by blue lines is the region of
instability. At temperature of crossover transition, the stability is
marginal, as both second and third derivatives of free energy Eq.(10) are zero
there.

Figure 5: The stability conditions for FM1 and FM2. On the vertical axes
$x=M^{2}$ and the blue lines denote $M^{2}_{1,2}$ as given by Eq.(12). The
solid orange line stands for $M^{2}$ of low-magnetisation phase, and the
orange curve marked by crosses is for $M^{2}$ of high-magnetisation phase. The
diamond marks $M^{2}$ at $t_{cr}$.

This is seen very well in the dependence of equilibrium free energy Eq. (10)
on the reduced temperature $t$ shown in Fig. (6)for two values of $w$.

Figure 6: The dependence of equilibrium free energy $f_{c}$ , Eq. (10) on
reduced temperature $t$. The points marked by diamonds show the crossover
transformation between the low- and high-magnetisation phases.

There is a small cusp at $t_{cr}$ on the curves, which marks the crossover
between FM1 and FM2.
We suppose that taking into account also the transverse components of
magnetisation the above established relation between the coefficients $u$ and
$w$ in Landau expansion, namely $u=w^{2}/3$ will be modified. From
experimental data the ratio between transverse and longitudinal magnetisations
is $\sim 0.02$, so the transverse components may be included as perturbation
and such approach is justified. Although small, their role in description of
ferromagnetism and especially of superconductivity is important as pointed,
for example in [1].

##  4  Concluding remarks

In this paper we apply the phenomenological Landau approach to UGe2 for the
description of magnetic phase transitions at ambient pressure far from
superconducting transition. The aim is to find a model, by which the
occurrence of two different magnetic phases with same magnetic structure but
different magnetisations can be described. We propose expansion of Landau free
energy in magnetisation up to $M^{8}$, for which by general theory it has been
proven that for particular values of Landau parameters, there is a possibility
for isostructural transition to occur, [18]. [19], [20]. We found the
relations between the parameters of Landau energy for which there is
possibility of second order phase transition to low-magnetization phase with
subsequent transition of crossover type at temperature lowering to high-
magnetization phase with the same magnetic structure. As shown there is strong
dependence between the magnitudes of parameters $u$ and $w$, see Eq. (6), and
obviously for the appearance of isostructural transition from low- to high-
magnetisation phase $w<0,u>0$ and $u=w^{2}/3$ . In order to make reliable
estimate of the magnitude of these parameters an adequate microscopic
calculations should be made for magnetic phase transitions at ambient
pressure.
Also the fact of working in quite rough approximation, dropping the
temperature dependence of coefficients $u$ and $w$ before $m^{4}$ and $m^{6}$
terms in Landau energy, Eq. (3) limits the possible variations of both $w$ and
$u$.
The nature of magnetic phase transitions from experimental point of view at
$p=0$ is still an open question. Here we propose a general thermodynamic
approach which does not depend on the undelying microscopic theories but we
consider that at this first stage of our study to make direct comparison with
experiment is preliminary. Our model needs some generalisation, especially the
coefficient in the Landau free energy, Eq. (1), namely, $b=b(T_{c},P_{0})$;
should be considered as dependent on temperature and not fixed at Curie
temperature.
As far as it is generally accepted that the spin fluctuations are responsible
for occurrence of superconductivity of p-type, it will be of great interest
also to consider their role on magnetic transitions within this
phenomenological approach. Recent experiments at ambient pressure [24] show
that the spin fluctuations in UGe2 are highly anisotropic, and the authors
claim that in PM state longitudinal spin fluctuations prevail, while in FM
state the transverse spin fluctuations are predominant. The nature of spin
fluctuations at ambient pressure remains unclear, having in mind also the dual
nature of f-electrons. The above issues will be the subject of future studies.

## Acknowledgements

This work is supported by Grant $KP-06-N38/6$ of the Bulgarian National
Science Fund.

## References

  * [1] C. Pfleider, Rev. Mod. Phys. 81, 1551 (2009)
  * [2] S. S. Saxena, P. Agarwal, K. Ahilan, F. M. Grosche, R. K. W. Haselwimmer, M. J. Steiner, E. Pugh, I. R. Walker, S. R. Julian, P. Monthoux, G. G. Lonzarich, A. Huxley, I. Sheikin, D. Braithwaite, and J. Flouquet, Nature (London) 406, 587 (2000).
  * [3] N. T. Huy, A. Gasparini, D. E. de Nijs, Y. Huang, J. C. P. Klaasse, T. Gortenmulder, A. de Visser, A. Hamann, T. Görlach, and H. v. Löhneysen, Phys. Rev. Lett. 99, 067006 (2007).
  * [4] Naoyuki Tateiwa, Yoshinori Haga, and Etsuji Yamamoto, Phys. Rev. Lett. 121, 237001 (2018)
  * [5] A. D. Huxley, Physica C 514, 368 (2015).
  * [6] Y. Onuki et al, J. Phys. Soc. Jpn. 61, 293(1992).
  * [7] Diana V. Shopova and Dimo I. Uzunov, Phys. Rev. B 79, 064501 (2009)
  * [8] Dai Aoki, Kenji Ishida, and Jacques Flouquet, J. Phys. Soc. Jpn. 88, 022001 (2019)
  * [9] N. Tateiwa, , Y. Haga, T. D. Matsuda, E. Yamamoto and Z. Fisk, Phys . Rev B 89, 064420 (2014)
  * [10] A. Yaouanc, P. Dalmas de Réotier, P. C. M. Gubbens, C. T. Kaiser, A. A. Menovsky, M. Mihalik, and S. P. Cottrell Phys. Rev. Lett. 89, 147001 (2002)
  * [11] R. Tro´c, Z. Gajek, and A. Pikul, Phys. Rev. B 86, 224403 (2012).
  * [12] Naoyuki Tateiwa, Jiří Pospíšil, Yoshinori Haga, Hironori Sakai, Tatsuma D. Matsuda, and Etsuji Yamamoto Phys. Rev. B 96, 035125 (2017)
  * [13] M. Fidrysiak, D. Goc-Jagło, E. Kadzielawa-Major, P. Kubiczek, and J. Spałek Phys. Rev. B 99, 205106 (2019)
  * [14] K. G. Sandeman, G. G. Lonzarich, and A. J. Schofield, Phys Rev. Lett, 90, 167005 (2002)
  * [15] M. Shimizu, Journal de Physique, 43 (1), 155 (1982)
  * [16] Gendo Oomi, Masashi.Ohashi, Kazutaka Nishimura, and Yoshichika Onuki, Journal of NUCLEAR SCIENCE and TECHNOLOGY, Supplement 3, 90 (2002)
  * [17] S. Sakarya, P. C. M. Gubbens, A. Yaouanc, P. Dalmas de Réotier, D. Andreica, A. Amato, U. Zimmermann, N. H. van Dijk, E. Brück, Y. Huang, and T.Gortenmulder, Phys. Rev. B 81, 024429 (2010).
  * [18] Yu. M. Gufan, E. S. Larin, Dokl. Akad. Nauk SSSR, 242, 1311–1313 (1978)
  * [19] Izyumov, Yu. A. and Syromiatnikov, N. V. , Phase transitions and symmetry of crystals, Nauka, Moscow - in Russian (1984)
  * [20] Toledano J. C. and Toledano P., The Landau theory of phase transitions, World Scientific, Singapore, (1987)
  * [21] M. Abrmowitz, and I. Stegun, Handbook of mathematical functions, National Bureau of Standards, Applied mathematics series (1964)
  * [22] D. Belitz, T. R. Kirkpatrick, and Thomas Vojta, Phys. Rev. Lett. 82, 4707 (1999)
  * [23] D. Belitz, T. R. Kirkpatrick, and Jörg Rollbühler Phys. Rev. Lett. 94, 247205 (2005)
  * [24] Yuichiro Noma, Hisashi Kotegawa, Tetsuro Kubo, Hideki Tou, Hisatomo Harima, Yoshinori Haga, Etsuji Yamamoto, Yoshichika Onuki, Kohei M. Itoh, Eugene E. Haller, Ai Nakamura, Yoshiya Homma, Fuminori Honda, and Dai Aoki, Journal of the Physical Society of Japan 87, 033704 (2018)


# Recognizing Handwritten Mathematical Expressions of Vertical Addition and
Subtraction

Daniel Rosa1, Filipe R. Cordeiro1, Ruan Carvalho1, Everton Souza1, Sergio
Chevtchenko2,
Luiz Rodrigues3, Marcelo Marinho1, Thales Vieira3 and Valmir Macario1 Email:
{daniel.carneiro, filipe.rolim, ruan.carvalho, marcelo.marinho,
<EMAIL_ADDRESS><EMAIL_ADDRESS>1 Visual Computing Lab,
Department of Computing, Universidade Federal Rural de Pernambuco (UFRPE),
Brazil  2Centro de Informática - CIn, Universidade Federal de Pernambuco
(UFPE), Brazil  3Center for Excellence in Social Technologies (NEES), Federal
University of Alagoas, Brazil

###### Abstract

Handwritten Mathematical Expression Recognition (HMER) is a challenging task
with many educational applications. Recent methods for HMER have been
developed for complex mathematical expressions in standard horizontal format.
However, solutions for elementary mathematical expression, such as vertical
addition and subtraction, have not been explored in the literature. This work
proposes a new handwritten elementary mathematical expression dataset composed
of addition and subtraction expressions in a vertical format. We also extended
the MNIST dataset to generate artificial images with this structure.
Furthermore, we proposed a solution for offline HMER, able to recognize
vertical addition and subtraction expressions. Our analysis evaluated the
object detection algorithms YOLO_v7, YOLO_v8, YOLO-NAS, NanoDet and FCOS for
identifying the mathematical symbols. We also proposed a transcription method
to map the bounding boxes from the object detection stage to a mathematical
expression in the LaTeX markup sequence. Results show that our approach is
efficient, achieving a high expression recognition rate. The code and dataset
are available at https://github.com/Danielgol/HME-VAS

††
979-8-3503-3872-0/23/$31.00 ©2023 IEEE

##  I Introduction

Computer-Assisted Learning (CAL) tools can contribute positively to teaching
and learning mathematics, helping with automatic examining scores and feedback
[1]. A common task of these tools is to use an image of a handwritten problem,
in which the system recognises the handwritten characters and solves the
problem or gives feedback [2]. One of the main stages of CAL systems is the
handwritten mathematical expression recognition (HMER), which is an image-to-
text task to generate the corresponding mathematical symbols, usually in a
LaTeX markup sequence, from an input image.

Although HMER can be compared to traditional handwriting text recognition
tasks, it is considered more challenging [3][4]. One of the characteristics of
HMER is the presence of spatial relations classification. In mathematical
notation, spatial relationships are mostly used as implicit operators. For
example, one number can be a superscript, subscript or regular number based on
the expression’s position, size and context. Besides, the incorrect
classification of a single number or mathematical symbol changes the result of
the operation and evaluation of the expression. Another challenge is the
presence of complex structures and formulas and large amounts of symbols,
often similar [5].

Recent advancements in HMER have been proposed using deep learning models [6].
Most of the recent research in HMER uses popular datasets such as CROHME 2019
[7] and HME100k [8], which consist of thousands of images of complex equations
in a horizontal format. However, none of these datasets contains equations in
vertical format, which is used elementary mathematical expressions. Vertical
addition and subtraction use columns to align each number’s place values,
solving each one before combining for the solution. In this process, the
numbers are lined up in columns according to their place values. The numbers
are added or subtracted in each place value separately to calculate the
solution. To the best of our knowledge, there is no work evaluating this type
of equation format for HMER using deep learning models. Moreover, there is no
public dataset of mathematical expressions in vertical formats, such as column
addition and subtraction. Fig. 1 shows images from CROHME, compared to our
dataset. Although the images from our dataset are related to mathematical
expressions, they are in a different domain because the equations are column-
wise, there is a carry symbol in the expression, and they require a different
training from the model.

(a) CROHME 2019

(b) Our dataset

Figure 1: Image samples from (a) CROHME 2019 [7] and (b) our built dataset.

In this work, we built an image dataset of basic addition and subtraction
equations written in a vertical format. We also extended MNIST [9] dataset to
generate artificial column addition and subtractions. We evaluated the object
detection deep learning models YOLO_v7 [10], YOLO_v8 [11], YOLO-NAS [12],
NanoDet [13] and FCOS [14] for mathematical symbol detection and proposed a
solution to HMER. The main contributions of this paper are as follows:

  * •

We propose a dataset of handwritten column addition and subtraction format to
simulate elementary school mathematical equations, which have not been
explored in the literature. We also extended MNIST to generate images for the
problem artificially;

  * •

We evaluated state-of-the-art (SOTA) methods of object detection for vertical
addition and subtraction;

  * •

We propose a transcription stage to map the bounding boxes to a LaTeX
expression, evaluating the ability of our solution to deal with the column-
wise format.

##  II Related Work

Traditional HMER solutions divide the recognition task into three stages: (1)
symbol recognition, (2) symbol classification and (3) structural analysis
[15]. Several strategies have been proposed to solve these problems
sequentially, using Support Vector Machines [16], Elastic Matching [17] and
tree transformation [18]. Hu and Zanibbi [19] proposed a symbol segmentation
method using AdaBoost algorithm and geometric multi-scale shape context
features. Le and Nakagawa [20] used a SVM-based classifier for symbol
segmentation, using 12 geometric features and 9 additional features. Hu and
Zanibbi [21] proposed a symbol segmentation algorithm based on graphs and
Parzen window-modified Shape Context features. Fang and Zhang [22] introduced
a new approach to isolated symbol recognition called squeeze-extracted multi-
feature convolution neural network. In the structure analysis step, formal
grammars have been proposed to recognize mathematical expression [23][24].

In recent years, encoder-decoder architectures have shown increased
performance in various image-to-text tasks, using deep learning models [6].
Zhang et al. [25] were one of the first to use an encoder-decoder neural
architecture to solve HMER tasks with their model WAP, outperforming
traditional grammar-based methods in the CROHME 2014 competition [26]. Later,
they proposed DenseWAP [27], which uses a multi-scale DenseNet encoder to
improve the ability to handle multi-scale symbols. Further, DenseWAP-TD [28]
improves the ability to handle complex formulas by substituting a string
decoder with a tree decoder. Although these end-to-end architectures perform
well for horizontal handwritten mathematical expressions, they require large
datasets to train the models and have millions of parameters.

Although SOTA methods in HMER have been extensively evaluated in large public
datasets, such as CROHME 2019 [7] and HME100k [8], these datasets are composed
of equations only in horizontal format. To the best of our knowledge, there is
no public dataset from equations in vertical format, using column addition or
subtraction, as shown in Fig 1(b). Our work differs from previous
contributions by proposing an HMER solution for column-wise equation structure
used in elementary school mathematics. We also simplify the structural
analysis by using an algorithm based on the spatial relation of detected
objects. Furthermore, we evaluated different SOTA backbones for our approach
to this new task domain.

##  III Method

###  III-A Problem Definition

Consider $D=\\{(x_{1},y_{1}),...(x_{n},y_{n})\\}$ as the training set of the
handwritten mathematical expression recognition problem, where
$\mathbf{x}_{i}\in\mathcal{X}$ is the $i^{th}$ image containing a mathematical
expression in a vertical format and $\mathbf{y}_{i}\in\mathcal{Y}$ is the
corresponding LaTeX expression.

A supervised classification learns a function $f:\mathcal{X}\to\mathcal{Y}$
that maps the input space $\mathcal{X}$ to the observed label space
$\mathcal{Y}$. The function $f$ can be decomposed as $f=g(h(\mathbf{x}_{i}))$,
where $h:\mathcal{X}\to\mathcal{W}$ is an object detector that maps the input
image $\mathbf{x}_{i}\in\mathcal{X}$ to a set $\mathbf{w}_{i}\in\mathbf{W}$ of
classification and localization of mathematical symbols of $\mathbf{x}_{i}$,
and $g:\mathcal{W}\to\mathcal{Y}$ is a transcriptor that maps the bounding
boxes’ positions and classes to a LaTeX expression. Therefore, $D$ can be
extended to
$\tilde{D}=\\{(\mathbf{x}_{n},\mathbf{w}_{n},\mathbf{y}_{n})\\}_{i=1}^{n}$,
where
$\mathbf{w}_{i}=\\{(\mathbf{l}_{i,j},\mathbf{b}_{i,j})\\}_{j=1}^{|\mathbf{w}_{i}|}$
denotes the classification and localization of the $|w_{i}|$ mathematical
symbols, with $\mathbf{l}_{i,j}$ denoting the label $l$ of the $j^{th}$ symbol
and $\mathbf{b}_{i,j}\in\mathbb{R}^{4}$ representing the top-left and bottom-
right coordinates of the bounding box of the $j^{th}$ symbol on
$\mathbf{x}_{i}$.

We restrict the mathematical expressions in $\mathbf{x}_{i}$ as vertical sums
and subtractions, using the operands “$+$” and “$-$”. As in common addition
and subtraction operations, it might contain carrying values, defined as the
label $c$. Therefore, each bounding box has an associated class
$l\in\mathcal{L}$, where $\mathcal{L}=\\{0,1,2,3,4,5,6,7,8,9,+,-,=,c\\}$.
Although the carry symbol $c$ also is represented as the number 1 in the
images, it depends on the size and positional features to be identified.

The transcription labelling in $\mathcal{Y}$ is built through a direct mapping
of classes in $\mathcal{L}$ to the LaTeX expression, except for the presence
of carrying symbols, which we use “$\backslash$overset{}{}” expression. For
example, the number 2 with a carry is denoted as “$\backslash$overset{1}{2}”
in the LaTeX markup sequence.

###  III-B Proposed Solution

The proposed solution divides the expression recognition problem into two main
stages: (1) Object Detection and (2) Transcription. In the object detection
stage, we aim to identify the localization of the mathematical symbols and
obtain the corresponding bounding boxes. Next, we perform a post-processing
step to improve the detection results. Once we have the position of each
mathematical symbol in the image, we use a transcriptor that receives as input
the bounding box and classes and produces a resulting expression in LaTeX.
Fig. 2 shows the proposed HMER solution.

Figure 2: Flowchart of the proposed solution for column addition and
subtraction recognition.

####  III-B1 Object Detection

In the object detection stage, we apply different state-of-the-art methods
based on convolutional neural networks. For this stage, we evaluated the
models YOLO_v7 [10], YOLO_v8 [11], YOLO_NAS [12], FCOS [14] and NanoDet [13].
All these approaches receive as input an RGB image and produce as output the
bounding boxes and corresponding classes.

We perform a post-processing stage to improve the obtained detection. In this
stage, we first filter the bounding boxes with a confidence level $\zeta$
lower than a threshold $\theta$, removing them. In the second step, when two
or more bounding boxes have an Intersection Over Union (IoU) value
$IoU>\alpha$, we keep the bounding box of the most confident class and discard
the remaining ones. The values of $\theta$ and $\alpha$ are selected using an
optimizer on the validation set. This strategy showed to improve the detection
and removed duplicated bounding boxes.

####  III-B2 Transcription

The transcription stage receives the bounding boxes and corresponding classes
as input and produces a LaTeX expression related to the mathematical equation
in the image. Some traditional approaches for HMER solve this problem by
either building a grammar [29] or using a graph strategy [30]. In this work,
we simplify this stage by proposing a transcription based on the positional
structure of the bounding boxes. Fig. 3 shows each step of the proposed
transcription stage, described below.

At the transcriptor stage, the detections are divided into four categories:
number, operation (op), equal sign, and carry, where the number refers to the
digits 0 to 9, operation $\in\\{+,-\\}$, equal sign refers to the equal line
in the expression, and carry refers to the small “1” digits related to the
carry symbol. The expression is expected to be in the format “A op B = R”,
where A and B are the operands and R is the resulting number. The operation
and equal symbols are both expected to have only one bounding box associated
with each image while the numbers and carry categories are expected to have
varying amounts of detections.

Based on the bounding box position of the equals sign, it is possible to
define the regions of the resulting term and the operand terms of the
equation. The bounding boxes below the equal sign are related to R, while the
others are associated with the operands A or B. From the resulting term R, we
define its transcription based on the predicted class since it is composed
only of numbers and has no carry symbol. The numbers of R are built based on
the $x$ coordinates of the bounding boxes in R, from left to right. The next
step is identifying the bounding boxes related to the expression’s operands,
including the associated carry symbols. To do so, we group the remaining
bounding boxes (i.e. not include operator, equal sign and resulting term) in
two groups, based on the $y$ value of the top-left $(x,y)$ coordinate of each
bounding box, defining the elements of operands A and B. For each term, we
identify the numbers and carry symbols.

In order to associate each carry symbol with its respective number, another
sequence is followed: First, the digits from each term are taken and checked
if there is any carry symbol nearby. This check occurs from the rightmost to
the leftmost digit from the actual term, starting from A term to B. We
associate a carry symbol with a related digit based on the intersection of the
carry and digit candidate.

Once the carry is associated with a specific digit, it can no longer be
related to any other digit. If any carry has not been linked to any digit, it
will be considered isolated and unrelated to any number. All information
needed to build the equation LaTeX expression is extracted at the end of the
mentioned processes. To generate it, it is necessary to translate each digit
of the following data from left to right. All the numbers associated with the
carry values are represented in LaTeX as “$\backslash$overset”, as described
in Sec. III-A.

Figure 3: Transcription stage. (a) bounding boxes and predictions of the
object detection; (b) Identification of operator Op, equal symbol and result
numbers. The results are obtained from the number below equal sign’s bounding
box; (c) identification of carry symbols and number associated, and (d)
construction of the expression based on the identified bounding boxes.

##  IV Methodology

###  IV-A Dataset

We built a custom dataset of elemental addition and subtraction arithmetic
equations written in a vertical format. Fig. 4 shows image samples from our
built dataset. The dataset comprises 300 images, with mathematical expressions
written by four human annotators and acquired using a camera. The images from
the dataset vary in dimension, but all are resized to $320\times 320$ pixels
to train the model. Besides the different writing styles, image writing also
varies using a pencil or pen. The ground truth of each image is saved in a
LaTeX format.

We considered 14 classes for the proposed dataset, composed of 10 classes
related to the digits from 0 to 9, 2 classes corresponding to the symbols of
sum and subtraction, 1 class related to the equal symbol, and 1 class
corresponding to the carry symbol. Although the carry has the same shape
representation as the digit $1$, the carry is also identified by its size
related to the other numbers and its position.

To define the training and test sets, we use a 3-fold cross-validation
approach, where at each evaluation, we leave one annotator out as the test set
and use the remaining ones as the training set. We refer to the annotators as
H1, H2, H3 and H4. We define the following splits in the format “training set/
test set”: (1) training set: H1, H2, and H3/ test set: H4 ; (2) training set:
H1, H3 and H4/ test set: H2, (3) training set: H1, H2 and H4/ test set: H3.
Table I shows the dataset distribution. For each training split, we separate
50 images from the training set as the validation set.

TABLE I: Our dataset distribution. | Number of Images | Split 1 | Split 2 | Split 3
---|---|---|---|---
Annotator 1 (H1) | 151 | Train | Train | Train
Annotator 2 (H2) | 75 | Train | Train | Test
Annotator 3 (H3) | 50 | Train | Test | Train
Annotator 4 (H4) | 25 | Test | Train | Train
Total | 300 |  |  |
Figure 4: Examples of images from our proposed column addition and subtraction
dataset.

We also proposed a dataset augmentation based on MNIST [9] dataset. MNIST
consists of handwritten digit images of size $28\times 28$ pixels with a
training set of 60,000 samples and a test set of 10,000 samples. As the MNIST
dataset contains only one digit by image, with no mathematical expression, we
built new images using the position of the mathematical expressions of our
dataset but replacing the number for the MNIST digits. We also used symbols
“+”, “-”, and “=” from our dataset but used the images from MNIST to fill the
position of the digits randomly. For this augmented dataset, we did not care
about the correctness of the equation because our goal is to recognize the
mathematical expression independent of the correctness. Fig. 5 shows examples
of the extended MNIST dataset. This artificially extended dataset is composed
of 2,600 images using MNIST digits.

Figure 5: Extended MNIST dataset based on mathematical expressions using MNIST
[9] digits.

In our work, we combined our 300 handwritten images dataset with the 2,600
augmented MNIST for the training set. The test and validation sets are
composed only of the built handwritten images with human annotators.

###  IV-B Metrics

We evaluated the proposed solution and different object detection methods by
using metrics related to the quality of the detection and transcription.

Average Precision (AP) is a commonly used metric for object detection, derived
from precision and recall. Average precision computes the average precision
value for recall value over 0 to 1 and is usually evaluated separately for
each object category. Precision is derived from Intersection over Union (IoU),
which is the ratio of the area of overlap and the area of union between the
ground truth and the predicted bounding box. A threshold is set to determine
if the detection is correct. Precision measures the percentage of correct
predictions, while recall measures the correct predictions to the ground
truth. To compare performance overall object categories, the mean AP (mAP)
averaged over all object categories is adopted as the final measure of
performance.

Expression recognition rate (ER), defined as the percentage of correctly
recognized expressions, is used to evaluate the performance of different
methods on mathematical expression recognition [3][31]. Moreover, $\leq 1$ and
$\leq 2$ are also used, indicating that the expression recognition rate is
tolerable at most one or two symbol-level errors. ER can be defined as:

$ER=\frac{\text{Number of correctly recognized expressions}}{\text{Total
number of expressions}}$ (1)

###  IV-C Implementation

For all models, we initialized the models using the pre-trained weights, pre-
trained at COCO dataset [32]. For all models, we used their implementation in
the provided repositories with the default parameters. We resized all the
images from our dataset to 320$\times$320 pixels and trained all the models
for 300 epochs. All the images are in RGB colour space. We used Optuna [33] to
optimize the parameters $\alpha$ and $\theta$ in the post-processing stage,
using the validation set. The models were trained and evaluated using the
splits defined in Tab. I, but we also added the 2600 artificial images from
the extended MNIST in each training split. The validation and test sets are
composed only of images from human annotators.

##  V Results and Discussion

We evaluated our solution using different object detectors for the proposed
dataset: YOLO_v7 [10], YOLO_v8 [11], YOLO-NAS [12], NanoDet [13] and FCOS
[14]. We selected these models because they are SOTA for object detection
[34]. We divided our analysis into three subsections: Object Detection,
Ablation Study and Expression Recognition.

###  V-A Object Detection

We evaluated the best object detector for the task of detecting the
handwritten mathematical symbols, as defined in Sec. IV-A. Tab. II shows the
mAP results, considering the different annotators as test set (H2, H3, H4) and
the average results.

TABLE II: Results of mAP for the object detection stage, reported @IoU=0.5. H2, H3 and H3 denote different annotators for the test set. The top methods are in bold. Method | H2 | H3 | H4 | Avg.
---|---|---|---|---
YOLO_v7 [10] | 87.29 | 98.07 | 99.55 | 94.97
YOLO_v8 [11] | 87.64 | 99.68 | 98.20 | 95.17
YOLO-NAS [12] | 66.81 | 74.63 | 99.67 | 80.37
NanoDet [13] | 82.89 | 98.62 | 99.32 | 93.61
FCOS [14] | 81.34 | 98.51 | 95.46 | 91.77

Results show that YOLO_v8 presented the best results for object recognition
compared with the other state-of-the-art methods. The annotator H2 has the
most challenging symbols for the models to detect because many symbols are
written similarly. Although YOLO_v7 also had high accuracy, on average,
YOLO_v8 had the best results. Fig. 6 shows examples of the detection of this
method, which indicates the high quality of the detection, despite the small-
size training set.

Figure 6: Results of object detection of the mathematical symbols for vertical
addition and subtraction.

###  V-B Ablation Study

We also evaluated the detection quality based on the built training set and
the extended MNIST. For this purpose, we evaluated the different training set
scenarios: (1) using only our proposed small-training dataset; (2) using only
the artificial equations based on augmented MNIST, and (3) using a combination
of augmented MNIST and the proposed dataset. For all scenarios, the validation
and test sets are composed only of our handwritten images. Table III shows
results considering these different setups, using YOLO_v8 object detector.

TABLE III: Ablation study using our handwritten dataset and the augmented MNIST, using YOLO_v8 object detector. The top methods are in bold. Training Data | H2 | H3 | H4 | Avg.
---|---|---|---|---
Our | 80.39 | 96.05 | 98.32 | 91.58
Augmented MNIST | 50.75 | 84.29 | 76.59 | 70.54
Our + Augmented MNIST | 87.64 | 99.68 | 98.20 | 95.17

Results show that the combination “Our + Augmented MNIST ” produces the best
results than only using MNIST-based expressions or our handwritten dataset.
This shows that using artificial images generated from MNIST digits, combined
with real handwritten images, improves the results, and both datasets are
important to solve the problem.

###  V-C Expression Recognition

We analyzed the expression recognition of the proposed method using different
methods for object detection. We used the metric ER for this task, defined in
Sec. IV-B. Table IV shows the results for ER, where $\leq 1$ and $\leq 2$
denote an error tolerance of 1 and 2 symbols, respectively.

TABLE IV: Results of Expression Recognition Rate (ER). ER, “$\leq 1$ error”
and “$\leq 2$ error” columns mean expression recognition rate when zero to two
structural or symbol errors can be tolerated. All results are reported as a
percentage (%). H2, H3 and H4 denote different annotators for the test set.
The top methods are in bold.

Method | H2 | H3 | H4 | Avg
---|---|---|---|---
ER |  $\leq 1$ error |  $\leq 2$ error | ER |  $\leq 1$ error |  $\leq 2$ errors | ER |  $\leq 1$ error |  $\leq 2$ error | ER |  $\leq 1$ error |  $\leq 2$ error
YOLO_v7 [10] | 44.28 | 71.42 | 88.57 | 72.00 | 96.00 | 100 | 100 | 100 | 100 | 72.09 | 89.14 | 96.19
YOLO_v8 [11] | 48.57 | 75.71 | 91.42 | 96.00 | 100 | 100 | 84.00 | 96.00 | 100 | 76.19 | 90.57 | 97.14
YOLO-NAS [12] | 12.85 | 28.57 | 47.14 | 8.00 | 26.00 | 54.00 | 92.00 | 100 | 100 | 37.61 | 51.52 | 67.04
NanoDet [13] | 22.85 | 47.14 | 71.42 | 84.00 | 98.00 | 100 | 92.00 | 100 | 100 | 66.28 | 81.71 | 90.47
FCOS [14] | 27.14 | 52.85 | 81.42 | 90.00 | 100 | 100 | 68.00 | 88.00 | 96.00 | 61.71 | 80.28 | 92.47

Results show that method YOLO_v8, combined with our proposed transcription
strategy, obtained the best results with an ER of 76.19% on average. As the H2
annotator had the hardest digits to detect, as shown in Tab II, it impacted
the ER metric, as each misclassified symbol counts as an error for the
expression. This shows that the area and the built dataset are challenging,
and the proposed solution is a promising direction to deal with this type of
equation.

##  VI Conclusion

In this work, we proposed a solution for automatically detecting and
recognising handwritten mathematical expressions in vertical format, which is
used for basic mathematical teaching. As there is no public dataset for this
task, we built a dataset and developed a method for identifying the
mathematical expression. We also extended MNIST to generate vertical addition
and subtractions, improving the results.

We also evaluated SOTA object detection methods for identifying mathematical
symbols, and YOLO_v8 obtained the best results, with 95.17% mAP on average. We
also proposed a transcription stage to transform the bounding boxes in a LaTeX
expression and could obtain high-quality results in expression recognition
metric.

This work provided a simple and effective solution to identify HME in a
vertical format, which was not explored in the literature. In future works, we
aim to expand the number of symbols and complexity of the expressions.

## References

  * [1] M. Meeter, “Primary school mathematics during the covid-19 pandemic: No evidence of learning gaps in adaptive practicing results,” _Trends in neuroscience and education_ , vol. 25, p. 100163, 2021.
  * [2] M. C. G. Neri, O. O. V. Villegas, V. G. C. Sánchez, H. d. J. O. Domínguez, M. Nandayapa, and J. H. S. Azuela, “A methodology for character recognition and revision of the linear equations solving procedure,” _Information Processing & Management_, vol. 60, no. 1, p. 103088, 2023.
  * [3] W. Zhao and L. Gao, “Comer: Modeling coverage for transformer-based handwritten mathematical expression recognition,” in _Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVIII_. Springer, 2022, pp. 392–408.
  * [4] D. Zhelezniakov, V. Zaytsev, and O. Radyvonenko, “Online handwritten mathematical expression recognition and applications: A survey,” _IEEE Access_ , vol. 9, pp. 38 352–38 373, 2021.
  * [5] B. Beeton, A. Freytag, and M. Sargent III, “Unicode support for mathematics,” Unicode Technical Note 25 Version 9, Unicode, Inc., 2008. URL http://www. unicode. org/reports/tr25, Tech. Rep., 2001.
  * [6] X. Bian, B. Qin, X. Xin, J. Li, X. Su, and Y. Wang, “Handwritten mathematical expression recognition via attention aggregation based bi-directional mutual learning,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 36, no. 1, 2022, pp. 113–121.
  * [7] M. Mahdavi, R. Zanibbi, H. Mouchere, C. Viard-Gaudin, and U. Garain, “Icdar 2019 crohme+ tfd: competition on recognition of handwritten mathematical expressions and typeset formula detection,” in _2019 International Conference on Document Analysis and Recognition (ICDAR)_. IEEE, 2019, pp. 1533–1538.
  * [8] Y. Yuan, X. Liu, W. Dikubab, H. Liu, Z. Ji, Z. Wu, and X. Bai, “Syntax-aware network for handwritten mathematical expression recognition,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 4553–4562.
  * [9] L. Deng, “The mnist database of handwritten digit images for machine learning research,” _IEEE Signal Processing Magazine_ , vol. 29, no. 6, pp. 141–142, 2012.
  * [10] C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” _arXiv preprint arXiv:2207.02696_ , 2022.
  * [11] G. Jocher, A. Chaurasia, and J. Qiu, “YOLO by Ultralytics,” Jan. 2023. [Online]. Available: https://github.com/ultralytics/ultralytics
  * [12] Deci.ai, “Yolo-nas,” https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md, 2023\.
  * [13] RangiLyu, “Nanodet-plus: Super fast and high accuracy lightweight anchor-free object detection model.” https://github.com/RangiLyu/nanodet, 2021.
  * [14] Z. Tian, C. Shen, H. Chen, and T. He, “Fcos: Fully convolutional one-stage object detection,” in _Proceedings of the IEEE/CVF international conference on computer vision_ , 2019, pp. 9627–9636.
  * [15] K.-F. Chan and D.-Y. Yeung, “Mathematical expression recognition: a survey,” _International Journal on Document Analysis and Recognition_ , vol. 3, pp. 3–15, 2000.
  * [16] B. Keshari and S. Watt, “Hybrid mathematical symbol recognition using support vector machines,” in _Ninth International Conference on Document Analysis and Recognition (ICDAR 2007)_ , vol. 2. IEEE, 2007, pp. 859–863.
  * [17] B.-Q. Vuong, Y. He, and S. C. Hui, “Towards a web-based progressive handwriting recognition environment for mathematical problem solving,” _Expert Systems with Applications_ , vol. 37, no. 1, pp. 886–893, 2010.
  * [18] R. Zanibbi, D. Blostein, and J. R. Cordy, “Recognizing mathematical expressions using tree transformation,” _IEEE Transactions on pattern analysis and machine intelligence_ , vol. 24, no. 11, pp. 1455–1467, 2002.
  * [19] L. Hu and R. Zanibbi, “Segmenting handwritten math symbols using adaboost and multi-scale shape context features,” in _2013 12th International Conference on Document Analysis and Recognition_. IEEE, 2013, pp. 1180–1184.
  * [20] A. D. Le and M. Nakagawa, “A system for recognizing online handwritten mathematical expressions by using improved structural analysis,” _International Journal on Document Analysis and Recognition (IJDAR)_ , vol. 19, pp. 305–319, 2016.
  * [21] L. Hu and R. Zanibbi, “Line-of-sight stroke graphs and parzen shape context features for handwritten math formula representation and symbol segmentation,” in _2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR)_. IEEE, 2016, pp. 180–186.
  * [22] D. Fang and C. Zhang, “Multi-feature learning by joint training for handwritten formula symbol recognition,” _IEEE Access_ , vol. 8, pp. 48 101–48 109, 2020.
  * [23] S. Lavirotte and L. Pottier, “Mathematical formula recognition using graph grammar,” in _Document Recognition V_ , vol. 3305. SPIE, 1998, pp. 44–52.
  * [24] K.-F. Chan and D.-Y. Yeung, “Error detection, error correction and performance evaluation in on-line mathematical expression recognition,” _Pattern Recognition_ , vol. 34, no. 8, pp. 1671–1684, 2001.
  * [25] J. Zhang, J. Du, S. Zhang, D. Liu, Y. Hu, J. Hu, S. Wei, and L. Dai, “Watch, attend and parse: An end-to-end neural network based approach to handwritten mathematical expression recognition,” _Pattern Recognition_ , vol. 71, pp. 196–206, 2017.
  * [26] H. Mouchere, C. Viard-Gaudin, R. Zanibbi, and U. Garain, “Icfhr 2014 competition on recognition of on-line handwritten mathematical expressions (crohme 2014),” in _2014 14th International Conference on Frontiers in Handwriting Recognition_. IEEE, 2014, pp. 791–796.
  * [27] J. Zhang, J. Du, and L. Dai, “Multi-scale attention with dense encoder for handwritten mathematical expression recognition,” in _2018 24th international conference on pattern recognition (ICPR)_. IEEE, 2018, pp. 2245–2250.
  * [28] J. Zhang, J. Du, Y. Yang, Y.-Z. Song, S. Wei, and L. Dai, “A tree-structured decoder for image-to-markup generation,” in _International Conference on Machine Learning_. PMLR, 2020, pp. 11 076–11 085.
  * [29] F. Álvaro, J.-A. Sánchez, and J.-M. Benedí, “An integrated grammar-based approach for mathematical expression recognition,” _Pattern Recognition_ , vol. 51, pp. 135–147, 2016.
  * [30] A. Lods, E. Anquetil, and S. Macé, “Fuzzy visibility graph for structural analysis of online handwritten mathematical expressions,” in _2019 International Conference on Document Analysis and Recognition (ICDAR)_. IEEE, 2019, pp. 641–646.
  * [31] B. Li, Y. Yuan, D. Liang, X. Liu, Z. Ji, J. Bai, W. Liu, and X. Bai, “When counting meets hmer: Counting-aware network for handwritten mathematical expression recognition,” in _Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVIII_. Springer, 2022, pp. 197–214.
  * [32] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13_. Springer, 2014, pp. 740–755.
  * [33] T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,” in _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , 2019.
  * [34] E. Arani, S. Gowda, R. Mukherjee, O. Magdy, S. Kathiresan, and B. Zonooz, “A comprehensive study of real-time object detection networks across multiple domains: A survey,” _arXiv preprint arXiv:2208.10895_ , 2022.