# An Automatic Approach for Grouping Sunspots and Calculating Relative Sunspot
Number on SDO/HMI Continuum Images

Cui Zhao College of Applied Science and Technology, Beijing Union University,
Beijing 102200, PR China  Shangbin Yang National Astronomical Observatories,
Chinese Academy of Sciences, 20A Datun Road, Chaoyang District, Beijing
100012, PR China  Tingmei Wang  College of Applied Science and Technology,
Beijing Union University, Beijing 102200, PR China  Haiyan Zhao  College of
Applied Science and Technology, Beijing Union University, Beijing 102200, PR
China  Shiyuan Liu  College of Applied Science and Technology, Beijing Union
University, Beijing 102200, PR China  Fangyuan He  College of Applied Science
and Technology, Beijing Union University, Beijing 102200, PR China  Zhengkun
Hu  College of Applied Science and Technology, Beijing Union University,
Beijing 102200, PR China

###### Abstract

Relative Sunspot Number is one of the major parameters for the study of long-
term solar activity. The automatic calculation of the Relative Sunspot Number
is more stable and accurate as compared to manual methods. In this paper, we
propose an algorithm that can detect sunspots and divide them into groups, to
automatically calculate the Relative Sunspot Number. Mathematical Morphology
was adopted to detect sunspots then group them. The dataset used were the
continuum images from SDO/HMI. The process was carried out on the overall HMI
data available on the timespan from January 2022 to May 2023 with a time
cadence of one day. The experimental results indicated that the method
achieved high accuracy of 85.3%. It was well fitted with the international
Relative Sunspot Number provided by Solar Influences Data Analysis Center
(SIDC) (CC=0.91). We calculated the conversion factor K value of SDO/HMI for
calculating the Relative Sunspots Number(K=1.03).

Solar, Relative Sunspot Number, sunspots groups, Astronomy image processing

††journal: AJ

##  1 Introduction

The Relative Sunspot Number(SN) represents the strength of solar activity and
provides the longest record of solar activity(Clette et al., 2014, 2015).
Coronal mass ejections and strong explosions on the solar surface can be
observed more frequently when the sunspot number reaches its maximum.
Continuous observation of sunspots has made it possible to monitor solar
activity and thus predict the space environment(Hathaway & Wilson, 2004;
Lukianova & Mursula, 2011; Yan et al., 2011). SN has long also been used in
studies of solar dynamo, predictions of Solar cycle, and studies of secular
variation of the Earth’s climate(Solanki et al., 2004; Wang, 2004).

The SN was first recorded according to the equation proposed by Wolf at the
Zurich Observatory in 1849, and later taken over by the Solar Influences Data
Analysis Center (SIDC)(Vanlommel et al., 2004; Clette et al., 2007). The SN is
defined by multiplying the number of observed sunspot groups by 10, adding the
total number of observed sunspots to the product, and then multiplying by the
parameter K(Hossfield, 2002; Clette et al., 2016). K is a conversion factor,
which stipulates that the K of Zurich Observatory is 1. The values of K for
the other observatories are determined by comparing their observations with
the Zurich Observatory’s Sunspot Number. The K value is related to factors
such as weather conditions (transparency and Astronomical seeing) at the
observatory, the aperture of the telescope and the experience of the observer.
Different observation stations or instruments have different K values.

Conventionally, the calculation of SN is manual based and due to human
subjectivity, the results are not stable. In recent years, with the
development of image processing technology and deep learning technology, a
variety of automatic sunspots extraction technologies have been proposed
(Curto et al., 2008; Zhao et al., 2016; Carvalho et al., 2020; Hanaoka, 2022).
The statistical accuracy of the total number of sunspots has gradually
improved. However, there are few automatic methods for sunspots grouping, the
accuracy is also very low. For instance, Dasgupta et al. (2011) used
clustering algorithm to group sunspots, and compared their results with Wolf
Number. The difference error between the two was about 10%. Palladino et al.
(2022) used the deep learning technology to group sunspots, and the accuracy
of sunspot group recognition was 44.22%.

As the new solar observation instruments have collected massive sunspot data,
a stable and accurate approach to accurate the relative SN is needed for these
data. In this paper, we proposed an algorithm that could detect and group
sunspots so that the Relative Sunspot Number could be calculated
automatically. The dataset used in the study were SDO/HMI Continuum images. We
further compared the calculated SN with manual means, Catalogue of
Heliophysics Features (HFC) and SIDC respectively. In addition, the K value of
the HMI was also calculated. To our knowledge, it is the first time that the K
value for the HMI has been calibrated. The paper is organized as follows:
Section 2 focuses on data. The proposed model is given in Section 3. Section 4
presents the results and discussion. Section 5 provides the method to
calculate the K value. Finally, the summary is provided in Section 6.

##  2 Data

We used the continuum images from SDO/HMI as the dataset. SDO/HMI is a space
device, conducts continuous observation of the day 24 hours without
interruption. It started the regular observation on April 30, 2010, providing
high-quality sunspot full-disk data. It publishes a set of data every 15
minutes, including magnetic field data and corresponding continuous images.
There are four different scale size of images, including: 256, 512, 1024, and
4096. We selected the daily images at 08:45:00UT from January 2022 to May
2023, with the image size of 1024 and an interval of one day. There were 507
images included in total. The original images were RGB, and converted to Gray
by the standard Matlab program rgb2gray().

##  3 Methods

As SN is calculated by the number of sunspots and sunspot groups, the method
proposed in this paper consists of two pipelines: the first part is an
automatic sunspot recognition method for automatically detecting sunspots and
calculating the total number of sunspots on the solar disk. The second part is
an automatic grouping method of sunspots, for automatically calculating the
number of sunspot groups.

Despite the rapid development of deep learning and computer vision technology
used in image processing, here we still chose the traditional image morphology
processing techniques. It is mainly based on the following considerations: (1)
For images with uniform background, the target and background grayscale are
significantly different, the traditional Mathematical Morphology(MM) generally
has good performance. As sunspots are obviously different from the solar disk,
MM should be suitable for detecting sunspots. (2) Deep learning technology
requires a large amount data annotation and high computational performance,
which requires a lot of manpower and computational power. While traditional
image processing technologies do not impose these requirements.

###  3.1 Mathematical Morphology Algorithm

Mathematical morphology is a discipline of image analysis based on lattice
theory and topology, which studies spatial structure and morphology (Serra,
1982; Heijmans, 1995). The basic concept of it is to change the shape and
features of an image by performing specific operations with the structural
elements on the images. The commonly used structuring elements are crosses,
squares, and open disks.

The basic morphological operators are erosion and dilation. If we set a binary
image as $\emph{J}:u\rightarrow{0,1}$, and the foreground pixels are
$u_{f}=J^{-1}(\\{1\\})$. The erosion and dilation of the binary image _J_ by
the structuring element $S\in Z\times Z$ are defined as:

$Erosion:\mu_{f}\ominus S=\\{x|\forall_{s}\in S,x+s\in\mu_{f}\\}$ (1)
$Dilation:\mu_{f}\oplus S=\\{x+s|x\in\emph{J}\land s\in S\\}$ (2)

The other two major operations in morphology are opening and closing. Opening
operation is considered to be erosion, followed by dilation and this operation
eliminates small objects and sharpens peaks in the object. Closing operation
is first dilation and then erosion, which fuses narrow breaks and fills small
holes and gaps in the image.

Dilation is an operation that looks for local maximum and makes the
highlighted area larger. Erosion is the operation that looks for the local
minimum and makes the highlighted area smaller. For sunspot images, the
highlighted part is the solar disk, and the black parts are the sunspots. As
the dilation and erosion operations are two dual operations, the dilation and
erosion operations can be applied to the black area as well. Dilation makes
sunspots smaller, and erosion makes sunspots larger.

###  3.2 Automatic recognition of sunspots

We described the automatic sunspot identification method in detail in Zhao et
al. (2016), and here we briefly describe the processing flow with the SDO/HMI
continuum image of January 14, 2022 as an example:

(1) For the observation image shown in Figure. 1(a), the morphological closing
operation was applied to eliminate sunspots and other darker areas on the
solar surface, obtain a clean solar disk, as shown in figure 1(b). The shape
of the structuring element used in closing is disk, with a size of 10.

(2) Figure 1(a) was subtracted from Figure 1(b) yield the darker regions on
the solar disk, as shown in Figure 1(c).

(3) Figure 1(c) is segmented automatically by the threshold to obtain the
Figure 1(d), which was a binary image with white areas as candidate sunspots.
Considering the limb darkening, the threshold for the area within
[1:1000,100:900] was set to 22, and the rest was set to 15.

(4) To eliminate the noises of the solar disk that can be easily regarded as
sunspots (which may be instrument-generated noise or darker areas on the solar
disk), we set the area of each candidate region to be greater than a fixed
value of $5$. In addition, since the gray value of the noise is relatively
uniform, and the gray value of the sunspots center are lower than that at the
edge, we set that the difference between the maximum and minimum values of
pixels in the set candidate area needs to be larger than a fixed value of $5$.
With the above rules, the noises on the solar surface were removed and the
pure sunspot regions were obtained, as presented in Figure 1(e). The sunspots
are marked in red.

Figure 1: An example of continuum image on January 14, 2022 to carry out
automatic sunspots recognition: (a) the original image, some sunspots are seen
on the solar disk. (b) the result of performing a closing operation on (a).
(c) the result of subtracting (a) from (b). (d) a binary image, which is the
result of the threshold automatic segmentation of (c). (e) sunspots marked
with red color are detected on the disk.

###  3.3 Group sunspots and calculate the relative SN automatically

Since sunspots belonging to a group are generally clustered together, we
assume that the sunspots smaller than a certain distance should be classified
as a group. Therefore we designed the algorithm flow shown in Figure 2:

(1) The white regions in Figure 2(a) are the sunspots detected in the first
stage. Morphological Erosion operation was carried out on Figure 2(a), and a
suitable structural element of size $15$ was selected experimentally. The
white areas closer to each other merged into a large area, as shown in Figure
2(b).

(2) Each white region in Figure 2(b) was labelled and numbered with a red
rectangle, and then superimposed on the original Figure 1(a) to obtain the
sunspots grouping Figure 2(c). This step was made automatically by Matlab
programs. It was divided into three steps: Firstly, the program regionprops()
was used to get each white area bounding box. It was the smallest rectangle
containing the white area, which was extracted by the program automatically.
Secondly, the program rectangle() was applied to show it out. Meanwhile the
program text() was used to label it. At last, the bounding boxes were
superimposed to the original image.

(3) To show the grouping effect clearly, we enlarged the sunspot groups
numbered 3 and 9, as shown in Figure 2(d) and 2(e). It could be seen that the
smaller and darker sunspot groups and sunspot groups at the edge of the solar
disk have been identified, and the algorithm was more effective in identifying
sunspot groups.

As shown in the figure below, the total number of sunspots was 15 and the
number of sunspot groups was 9, so the relative number of sunspots was 105.

Figure 2: Group sunspots after they are be detected: (a) the sunspots detected
in the first stage. (b) the results of erosion calculation on figure (a). (c)
sunspots grouping labeled with a total 9 groups and overlayed to the original
image. (d) enlarge and display group 3. (e) enlarge the display group 4.

##  4 Results

###  4.1 Compare our results with HFC’s

Catalog of Heliophytics Features (HFC)(Bonnin et al. (2013)) is the Solar
Survey Archive of BASS2000 available at https://bass2000.obspm.fr/home.php. It
provides daily sunspot groups numbers, which were adopted here as a comparison
with our method.

Since the HFC can only select a particular date to search for the number of
sunspot groups on that day, it can not select a time span to query, which is
more laborious. Therefore only a small period of data could be used in the
study, and this was not statistically significant (we will compare
statistically with long-term data from SIDC later in Section 5). We searched
for the daily sunspot groups numbers from April 1 to April 30, 2023, and
calculated it in the same period by our algorithm. The results are shown in
Table 1.

Table 1: Compare the number of sunspot groups calculated by Our Algorithm with HFC’s  Date | Our algorithm | HFC | Difference
---|---|---|---
20230401 | 5 |  | 5 | 0
20230402 | 3 |  | 3 | 0
20230403 | 3 |  | 2 | 1
20230404 | 5 |  | 4 | 1
20230405 | 4 |  | 3 | 1
20230406 | 3 |  | 3 | 0
20230407 | 4 |  | 3 | 1
20230408 | 3 |  | 2 | 1
20230409 | 4 |  | 3 | 1
20230410 | 6 |  | 5 | 1
20230411 | 5 |  | 5 | 0
20230412 | 9 |  | 7 | 2
20230413 | 9 |  | 7 | 2
20230414 | 10 |  | 9 | 1
20230415 | 14 |  | 10 | 4
20230416 | 12 |  | 11 | 1
20230417 | 10 |  | 7 | 3
20230418 | 10 |  | 8 | 2
20230419 | 10 |  | 8 | 2
20230420 | 6 |  | 6 | 0
20230421 | 9 |  | 6 | 3
20230422 | 9 |  | 8 | 1
20230423 | 6 |  | 7 | -1
20230424 | 6 |  | 7 | -1
20230425 | 9 |  | 6 | 3
20230426 | 8 |  | 6 | 2
20230427 | 5 |  | 8 | -3
20230428 | 5 |  | 4 | 1
20230429 | 4 |  | 4 | 0
20230430 | 4 |  | 4 | 0

As can be indicated in Table 1, our results generally follow the same trend as
those of HFC. However, the numbers of sunspot groups calculated by our
algorithm were mostly larger than that of HFC. This was because our algorithm
could detect weak sunspot groups that have just appeared or were about to
disappear. The sunspot groups in the left image in Figure 3(a) were detected
by our algorithm. The sunspot groups in the right image were marked by HFC. By
comparison, the group numbered 3 was located at the edge of the solar disk,
and it was found by our algorithm but ignored by HFC. In a few case, our
algorithm mistook a large discrete sunspot group for two small groups. For
example in Figure 3(b), the sunspot group numbered 2 and 3 should actually
belong to a group. It was because our algorithm divided sunspots into a group
according to the distance between sunspots. If the distance between sunspots
in a large group was bigger than a fixed value, it would be divided into two
groups.

Figure 3: Compare the sunspot grouping difference between our algorithm with
HFC. On the left is the number of sunspot groups calculated by our algorithm
on 2023-4-3. On the right is the number of sunspot groups marked by HFC on
2023-4-7.

###  4.2 Accuracy and Efficiency

To test the accuracy of our model, we set up a benchmark for comparison using
the manual labelling results. As shown in Table 2, we manually annotated the
data from April 1, 2023 to April 30, 2023 and then calculated four parameters:
True Positive (TP), False Negative (FN), False Positive (FP), and True
Negative (TN). They are the concepts from the Confusion matrix(Visa et al.
(2011)). TP means that the sunspot groups on the solar disk are correctly
identified and predicted to be positive. FN indicates that the originally
existing sunspot group on the solar disk is not recognized, and is incorrectly
predicted as the background area of the solar disk. FP indicates that the
solar background or noisy regions are mistakenly identified as a sunspot
group. TN represents the correct recognition of the solar background or noise
regions as background, which is meaningless here.

Table 2: Sunspot groups number by our model and by manual means  Date | Our Model | Manual Means | TP | FN | FP | TN
---|---|---|---|---|---|---
20230401 | 5 |  | 5 |  | 5 | 0 | 0 | 0
20230402 | 3 |  | 3 |  | 3 | 0 | 0 | 0
20230403 | 3 |  | 3 |  | 3 | 0 | 0 | 0
20230404 | 5 |  | 5 |  | 5 | 0 | 0 | 0
20230405 | 4 |  | 4 |  | 4 | 0 | 0 | 0
20230406 | 3 |  | 3 |  | 3 | 0 | 0 | 0
20230407 | 4 |  | 3 |  | 2 | 1 | 2 | 0
20230408 | 3 |  | 3 |  | 3 | 0 | 0 | 0
20230409 | 4 |  | 4 |  | 4 | 0 | 0 | 0
20230410 | 6 |  | 6 |  | 6 | 0 | 0 | 0
20230411 | 5 |  | 5 |  | 5 | 0 | 0 | 0
20230412 | 9 |  | 7 |  | 7 | 0 | 2 | 0
20230413 | 9 |  | 9 |  | 9 | 0 | 0 | 0
20230414 | 10 |  | 9 |  | 9 | 0 | 1 | 0
20230415 | 14 |  | 11 |  | 11 | 0 | 3 | 0
20230416 | 12 |  | 10 |  | 10 | 0 | 2 | 0
20230417 | 10 |  | 8 |  | 8 | 0 | 2 | 0
20230418 | 10 |  | 8 |  | 8 | 0 | 2 | 0
20230419 | 10 |  | 8 |  | 8 | 0 | 2 | 0
20230420 | 6 |  | 5 |  | 5 | 0 | 1 | 0
20230421 | 9 |  | 7 |  | 7 | 0 | 2 | 0
20230422 | 9 |  | 7 |  | 7 | 0 | 2 | 0
20230423 | 6 |  | 5 |  | 5 | 0 | 1 | 0
20230424 | 6 |  | 6 |  | 0 | 0 | 0 | 0
20230425 | 9 |  | 7 |  | 7 | 0 | 2 | 0
20230426 | 8 |  | 7 |  | 7 | 0 | 1 | 0
20230427 | 5 |  | 5 |  | 0 | 0 | 0 | 0
20230428 | 5 |  | 5 |  | 0 | 0 | 0 | 0
20230429 | 4 |  | 4 |  | 0 | 0 | 0 | 0
20230430 | 4 |  | 4 |  | 0 | 0 | 0 | 0
Total |  |  |  |  | 151 | 1 | 25 | 0

To evaluate the model, we used machine learning classification algorithm
metrics such as Accuracy, Precision, Recall, and F1 Score, as shown in
formulas (1) - (4)(Goutte & Gaussier, 2005; Yacouby & Axman, 2020). Accuracy
refers to the ”number of correctly predicted samples/total number of samples”.
Precision refers to how many of the samples we predict to be true are indeed
true. Recall refers to how many samples that are actually true have been
selected by the model. Precision and Recall are often mutually exclusive. The
F1 score takes into account both Precision and Recall factors, achieving a
compromise between them. The F1 score can be directly used to evaluate the
performance of a model.

$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$ (3)

$Precision=\frac{TP}{TP+FP}$ (4)

$Recall=\frac{TP}{TP+FN}$ (5)

$F1Score=2*\frac{Precision*Recall}{Precision+Recall}$ (6)

After substituting the data in Table 2 into the equation, the results can be
seen in Table 3. As indicated in the table, the performance of our model is
significantly better than that of Palladino et al. (2022).Palladino et al.
(2022) proposed a method where a pipline of state-of-the-art sunspots groups
detection and classification. Deep neural networks (DNNs) was adopted in their
study to detect and classify sunspots groups in real time. They used the
McIntosh system for sunspot groups classification and the generation process
was carried out on the overall HMI data available on the timespan that went
from 2010 to 2021 with a sampling time of 1 day. In order to classify sunspots
groups, they paid more attention to the area sunspots groups in detection
stage. Therefore, the accuracy metrics they choose is Intersection over Union
(IoU) between predicted bounding box and ground truth bounding box.
Intersection over union was then computed as the ratio of the overlapping area
of the two bounding boxes over the union of their total area). Our choice of
accuracy metrics was dependent on the number of sunspots groups, which was
more reasonable as our purpose was to automatically calculate the relative SN.

As can be seen from the table, our algorithm has high recall rate of 99.3%,
indicating that almost all sunspot groups present on the solar disk can be
accurately detected. The precision rate of 85.8% is lower than the recall
rate, which could explained by the fact that faint sunspot groups that were
difficult to observe manually were now detectable by our algorithm.

Table 3: Performance on Sunspot Group Detection.  Metric | Palladino et al. (2022) | Our Model
---|---|---
Precision | 65.82% |  | 85.8%
Recall | 57.45% |  | 99.3%
F1 Score | 61.35% |  | 92.1%
Accuracy | 44.22% |  | 85.3%

In terms of computation time, our algorithm processed over 500 pieces of data
on a single core regular desktop computer, in 5 minutes, with an average
processing time of 0.6 seconds per data. In contrast, the algorithm proposed
by Palladino et al. (2022) initially took nearly 8 hours of computation time
to generate nearly 3000 images on a regular desktop computer. After
parallelizing on a 6-core CPU, the computational time was reduced to 1 hour.
Our algorithm is clearly faster and has better real-time processing
performance.

##  5 Calculate the K value of HMI Relative Sunspot Number

By writing a Python data collection program, we downloaded data between
January 2022 to March 2023 and selected a daily sample of 507 images from
http://jsoc.stanford.edu/data/hmi/images/. Then our algorithm was conducted to
calculate the relative SN. On the other hand, with SIDC as calibration, we
downloaded the SN in the same time span from https://www.sidc.be/SILSO/home.
Then the correlation between them was calculated and shown in Figure 4. The
horizontal axis represents the date (year-month), and the vertical axis
denotes the Relative SN. The correlation coefficient between the two sets of
data is 0.91. It is highly consistent, indicating that our algorithm is
reliable.

Figure 4: Correlation between the relative SN of our algorithm and of SIDC

As shown in Figure 5, we performed the least squares fit to both sets of data,
which resulted in a fit of K=1.03. This is the scaling value of the HMI
instrument.

Figure 5: Least squares fitting of the sunspot SN in HMI and SIDC to calculate
the K value of HMI

##  6 Summary

The long-term relative SN is statistically helpful in exploring the periodic
patterns of the sun. In this paper, an automatic algorithm for calculating the
relative SN is proposed. The experiments were conducted on HMI dataset. The
conclusions of this article are as follows: (1) The performance of our
algorithm: Accuracy=85.3%, Precision=85.8%, Recall=99.3%, and F1 Score=92.1%.
(2) The average processing time of our algorithm is 0.6 seconds per data,
which is remarkably fast and real-time. (3) The relative SN calculated by our
algorithm is highly consistent with SIDC with a correlation of 0.91. This
indicates our algorithm is reliable. (4) For the first time, we have
calculated and confirmed the K value of SDO/HMI, which is K=1.03.

The authors would like to thank the anonymous reviewers for comments and
suggestions that improved the quality of this manuscript. Images and data used
in this paper are provided by SDO/HMI, SIDC, and HFC, for which we are very
grateful. This work is supported by the Academic Research Projects of Beijing
University (No. ZK20202204, ZK90202106, ZK90202105), the General projects of
science and technology plan of Beijing Municipal Education Commission
(KM20211141700) and the R&D Program of Beijing Municipal Education Commission
(KM201911417003, KM202211417006).

## References

  * Bonnin et al. (2013) Bonnin, X., Fuller, N., Renié, C., et al. 2013, Proceedings of the International Astronomical Union, 8, 512
  * Carvalho et al. (2020) Carvalho, S., Gomes, S., Barata, T., Lourenço, A., & Peixinho, N. 2020, Astronomy and Computing, 32, 100385
  * Clette et al. (2007) Clette, F., Berghmans, D., Vanlommel, P., et al. 2007, Advances in Space Research, 40, 919
  * Clette et al. (2015) Clette, F., Cliver, E., Lefèvre, L., Svalgaard, L., & Vaquero, J. 2015, Space Weather, 13, 529
  * Clette et al. (2016) Clette, F., Lefèvre, L., Cagnotti, M., Cortesi, S., & Bulling, A. 2016, Solar Physics, 291, 2733
  * Clette et al. (2014) Clette, F., Svalgaard, L., Vaquero, J. M., & Cliver, E. W. 2014, Space Science Reviews, 186, 35
  * Curto et al. (2008) Curto, J., Blanca, M., & Martínez, E. 2008, Solar Physics, 250, 411
  * Dasgupta et al. (2011) Dasgupta, U., Singh, S., & Jewalikar, V. 2011, in 2011 Third National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics, IEEE, 171–174
  * Goutte & Gaussier (2005) Goutte, C., & Gaussier, E. 2005, in European conference on information retrieval, Springer, 345–359
  * Hanaoka (2022) Hanaoka, Y. 2022, Solar Physics, 297, 158
  * Hathaway & Wilson (2004) Hathaway, D. H., & Wilson, R. M. 2004, Solar Physics, 224, 5
  * Heijmans (1995) Heijmans, H. 1995, in Proceedings of Summer School on Morphological Image and Signal Processing, Citeseer, 228–231
  * Hossfield (2002) Hossfield, C. H. 2002, Journal of the American Association of Variable Star Observers, Vol. 31, No. 1, p. 48-53, 31, 48
  * Lukianova & Mursula (2011) Lukianova, R., & Mursula, K. 2011, Journal of Atmospheric and Solar-Terrestrial Physics, 73, 235
  * Palladino et al. (2022) Palladino, L., Ntagiou, E., Klug, J., Palacios, J., & Keil, R. 2022, in 2022 IEEE Aerospace Conference (AERO), IEEE, 1–10
  * Serra (1982) Serra, J. 1982, (No Title)
  * Solanki et al. (2004) Solanki, S. K., Usoskin, I. G., Kromer, B., Schüssler, M., & Beer, J. 2004, Nature, 431, 1084
  * Vanlommel et al. (2004) Vanlommel, P., Cugnon, P., Linden, R. V. D., Berghmans, D., & Clette, F. 2004, Solar Physics, 224, 113
  * Visa et al. (2011) Visa, S., Ramsay, B., Ralescu, A. L., & Van Der Knaap, E. 2011, Maics, 710, 120
  * Wang (2004) Wang, Y.-M. 2004, Solar Physics, 224, 21
  * Yacouby & Axman (2020) Yacouby, R., & Axman, D. 2020, in Proceedings of the first workshop on evaluation and comparison of NLP systems, 79–91
  * Yan et al. (2011) Yan, X., Deng, L., Qu, Z., & Xu, C. 2011, Astrophysics and Space Science, 333, 11
  * Zhao et al. (2016) Zhao, C., Lin, G., Deng, Y., & Yang, X. 2016, Publications of the Astronomical Society of Australia, 33, e018


# A Novel Dataset for Financial Education Text Simplification in Spanish

Nelson Pérez Rojas,1 2 Saúl Calderón Ramírez,3 Martín Solís Salazar,4
Mario Romero Sandoval,5 Mónica Arias Monge,6 Horacio Saggion7
1Doctorado en Ciencias Naturales para el Desarrollo (DOCINADE), Instituto
Tecnológico de Costa Rica, Costa Rica
2Escuela de Ciencias del Lenguaje, Instituto Tecnológico de Costa Rica, Costa
Rica
3Escuela de Ingeniería en Computación, Instituto Tecnológico de Costa Rica,
Costa Rica
4Escuela de Administración de Empresas, Instituto Tecnológico de Costa Rica,
Costa Rica
5Maestría en Computación, Instituto Tecnológico de Costa Rica, Costa Rica
6Instituto de Investigaciones Psicológicas, Universidad de Costa Rica, Costa
Rica
7Departamento de Tecnologías de la Información y las Comunicaciones,
Universidad Pompeu Fabra, España
<EMAIL_ADDRESS>

##  1 Introduction

Text simplification encompasses a variety of tasks, among them sentence
structure modification through different actions such as irrelevant
information suppression, splitting long sentences, changing a sentence from a
passive form to a subject–verb–object structure, and other actions [Saggion et
al.,2015, Romero et al.,2022]. Text simplification holds particular value for
individuals with visual impairments, as it enables more efficient access to
the core meaning of complex texts. Such texts frequently present syntactic and
lexical challenges [Monge, Alvarado, and Rojas,2023]. These complex texts are
often syntactically and lexically challenging, particularly in specialized
domains like finance.

One of the main challenges to generating those models is data availability
[Coster and Kauchak,2011]. Most existing datasets focus on English,
complicating research and development for text simplification in other
languages. Most datasets in other languages have been created by aligning
sentences from original texts with simpler versions in similar texts [Romero
et al.,2022]. The alignment helps to create a more extensive dataset, but it
is not always clear which simplification rules were used to make the simple
versions.

Furthermore, creating these datasets needed to adequately consider the unique
needs of specific populations, with text simplification requirements varying
across different groups [Fajardo et al.,2015, Romero et al.,2022]. For
example, deaf individuals need simplifications tailored to their experiences
with oral language as a second language and the specificities of sign language
[Monge, Alvarado, and Rojas,2023], differing significantly from the needs of
blind individuals who rely on audio formats. Blind people’s challenges in
accessing information stem from the modalities of obtaining it, not from
cognitive limitations. They depend on screen readers to convert text to audio,
which requires a different approach to information processing [Sauvan et
al.,2020, Barbieri et al.,2005]. Text simplification, therefore, is not just
about comprehension but about making content more accessible auditorily
[Parmanto et al.,2005].

In Spanish, limited datasets are available for text simplification, notably
the manually curated simplex dataset [Saggion et al.,2015] and one developed
through automatic sentence alignment from Newsela [Palmero Aprosio et
al.,2019]. This manuscript introduces a novel dataset for Spanish, mainly
catering to a specific population, a first of its kind. Our goal is to foster
research in text simplification for specialized groups. The focus on financial
education aligns with the global movement towards financial literacy,
essential for societal financial decision-making and well-being, as recognized
in various national strategies [Atkinson and Messy,2013].

Creating datasets for text simplification, especially in non-English languages
and for targeted demographic groups, presents two primary challenges. Firstly,
the scarcity of large, high-quality datasets restricts the effectiveness of
text simplification models. Secondly, variations in data quality and
consistency affect the models’ learning efficiency. This manuscript introduces
a tailored Spanish financial text simplification dataset to address these
issues, consisting of 5,314 sentence pairs developed using distinct
simplification rules, thereby enriching the resources available for improving
text simplification models.

Moreover, the limited size of manually generated datasets, such as ours, poses
a challenge for training Large Language Models (LLMs) that typically require
vast datasets [Tida, Srinivas, and Hsu,2022]. Consequently, this study also
aims to explore using models like GPT-3, mT5, and Tuner for data augmentation
in text simplification. GPT-3, having demonstrated superior results in our
evaluations, is considered even more compelling for data augmentation due to
its proprietary nature.

The following is a description of the key contributions of this work:

  1. 1.

A set of text simplification rules designed explicitly for Spanish aimed at
enhancing the comprehensibility of financial education texts for individuals
with visual impairments who rely on screen readers like NVDA and JAWS [Singh
and Goyal,2021].

  2. 2.

A novel text simplification dataset in Spanish using the rules defined
previously, which we refer to as Financial Education Corpus IN SpAnish
(FEINA).

  3. 3.

A data-oriented evaluation of the quality and reliability of the manually and
automatically generated datasets.

###  1.1 Text simplification methods

Using pre-trained models via transfer learning in NLP is critical for
improving task-specific performance. However, this approach has been
predominantly focused on English, overlooking many non-English languages. To
address such a gap, tmT5 and mC4 are multilingual versions of the T5 model and
C4 dataset, respectively. mT5, following the T5’s architecture, achieves
advanced results in diverse evaluations bolstered by mC4, which includes text
in 101 languages [Xue et al.,2020].

These models tackle the previously overlooked ’erroneous translation’
phenomenon in multilingual NLP models during zero-shot scenarios, where
generative models may unintentionally translate text into the wrong language.

TUNER, a novel model in lexical simplification focusing on Spanish,
Portuguese, Catalan, and Galician, utilizes an unsupervised approach to
achieve state-of-the-art results, especially in Spanish. Unlike deep learning
models like LSBert [Qiang et al.,2020], which rely on dynamic methods, TUNER
uses static resources such as vocabularies and thesauri. Its multilingual
architecture includes stages like Sentence Analysis, Word Sense Disambiguation
(WSD), Synonym Ranking, and Morphological Generation. It also features a
morphological generator using lexicon-based algorithms and decision-tree
predictions for grammatically accurate inflections [Štajner et al.,2022].

Recent advancements in Text Simplification (TS) systems are driven by large
datasets and powerful language models like GPT-3, enhancing their role in NLP
applications such as machine translation, text summarization, and question
answering. The effectiveness of TS systems hinges on the quality and diversity
of training data. In general, to improve this, researchers are exploring
strategies like data augmentation, transfer learning, semi-supervised learning
and active learning in Natural Language Processing (NLP) or other domains [Ott
et al.,2019, Calderon-Ramirez et al.,2022, Benavides-Mata and Calderon-
Ramirez,2022, Calderon-Ramirez, Yang, and Elizondo,2022, Calderon-Ramirez et
al.,2022, Calderon-Ramirez et al.,2021, Calderon-Ramirez et al.,2021a,
Calderon-Ramirez et al.,2022a, Calderon-Ramirez et al.,2022b, Calderon-Ramirez
et al.,2021b, Calderon-Ramirez et al.,2021b].

Developing new AI models, such as OpenAI’s ChatGPT, marks a significant
advancement. ChatGPT, based on the GPT architecture, utilizes neural networks
for natural language management, enabling context-sensitive, human-like
interactions. Beyond traditional Text Simplification (TS) uses, ChatGPT offers
applications like real-time interaction emulation. However, its adoption comes
with challenges, including potential bias in training datasets and the risk of
misinformation spread. These issues highlight the importance of careful data
curation and ongoing improvement of data enhancement strategies for TS system
integrity and accuracy [Sallam,2023].

Transfer learning in Text Simplification (TS) has become increasingly popular,
enhancing TS models by utilizing pre-trained language models (PLMs) like BERT,
RoBERTa, and T5 [Devlin et al.,2018]. This method involves fine-tuning a PLM
for specific TS tasks, enabling models to recognize complex linguistic
features better and improve generalization. Furthermore, a comparative
analysis of leading TS datasets for Spanish and English, including ALEXSIS,
EASIER/EASIER-500, Simple Wikipedia (SWiRL), Newsela, WikiLarge, and Parallel
Universal Dependencies (PUD), focuses on language, sentence count, alignment,
domain specificity, and target population [Ferrés and Saggion,2022].

###  1.2 Datasets in Text Simplification Research

Research in automatic text simplification has predominantly been in English.
However, efforts have extended to languages like Italian [Palmero Aprosio et
al.,2019], Japanese [Maruyama and Yamamoto,2020], French [Todirascu et
al.,2022], German [Gonzales et al.,2021], Spanish [Palmero Aprosio et
al.,2019], Russian [Dmitrieva and Tiedemann,2021], and Bangla [Hossain and
Ahnaf,2021].

These studies have led to the creation of new training and evaluation
datasets. Often, these datasets are formed by aligning sentences from original
texts with their simplified versions [Palmero Aprosio et al.,2019, Dmitrieva
and Tiedemann,2021], but the exact methods and rules used in simplification
are usually kept a secret. In the French context [Todirascu et al.,2022], a
parallel corpus with 4,596 sentence pairs was expanded using a translated
version of the WikiLarge dataset. The Japanese study [Maruyama and
Yamamoto,2020] utilized a parallel corpus with three simplifications: word or
phrase-level simplification, sentence-level simplification, maintaining the
original meaning, and document-level simplification.

Notable datasets in English include the Simple Wikipedia (English) and the
Turk Corpus. Simple Wikipedia was compiled by aligning sentences from
Wikipedia with its simpler version, Simple English Wikipedia, tailored for
children, English learners, and non-native speakers who might find standard
Wikipedia articles too complex. This alignment used various similarity
measures. The Turk Corpus was derived from a subset of the PWKP dataset [Xu et
al.,2016, Zhu, Bernhard, and Gurevych,2010], filtering sentences with more
than a 20% variation in token count between the complex and simplified forms,
mainly retaining paraphrase simplifications. They extracted 2,350 sentences
from regular Wikipedia and had them simplified by eight Amazon Mechanical Turk
workers, resulting in up to 18,800 complex-simple sentence pairs [Xu et
al.,2016].

Regarding Spanish, several datasets have been developed: The Aligned Newsela
corpus (Spanish) [Palmero Aprosio et al.,2019] contains nearly 1,221
professionally annotated documents. Following cleanup and sentence alignment,
55,890 sentence pairs were obtained as a Spanish benchmark, although this
dataset is not available for research. The SIMPLEXT Corpus [Saggion et
al.,2011] includes 200 news articles manually simplified by experts to aid
comprehension for those with learning disabilities. The EASIER dataset is
annotated for easy-to-read language in lexical simplification, and the ALEXSIS
dataset [Ferrés and Saggion,2022] has 381 instances for lexical simplification
tasks. Each ALEXSIS instance includes a sentence, a complex target word, and
25 substitution options.

Dataset | N. Instances | Aligned | Specific Domain | Specific Population
---|---|---|---|---
ALEXSIS | 381 | Yes | No | No
EASIER | 5130 | Yes | No | No
EASIER-500 | 500 | Yes | No | No
Newsela | 20000 | Yes | No | No
Med-EASi | 1979 | Yes | Medical | No

Table 1: Examples of datasets.

This comparison highlights the limited availability of domain-specific and
target population-focused data sets for Text Simplification (TS), particularly
for Spanish. Most of these data sets are also aligned. The latter presents a
problem for training TS models: more custom TS data sets are needed for
specific domains and populations, emphasizing alignment to speed up model
training. The data sets compared in this study will be used as a reference
point for creating and evaluating TS data sets in the future.

##  2 Methodology

This work can be split in three methodological steps, following the research
questions established in Section 1

###  2.1 Financial Education Text Dataset Construction

The first step comprises the process of building the text simplification
dataset. Initially, we selected 4 books about financial education [Torres
Salazar and Ramos Arriagada,2020, Gil,2008, Izaguirre Olmedo, Carhuancho
Mendoza, and Silva Siu,2020, for Financial Education,2017] to create the
dataset. We selected books that let us release to the scientific research
community the dataset. The second step was to extract text segments
(sentences) from the books. A text segment is defined as follows: It is a
piece of text that is separated from the next segment text by the following
punctuation symbols: a) dot, b) semicolon, c) Closing question mark, d)
Closing exclamation mark. In summary, we split the books into pieces of text
that we called text segments. For example, in table II we can see a paragraph
decomposed into text segments, using the dot symbol.

Paragraph |  Segments
---|---
“La tangibilidad es una característica muy esencial de un objeto para considerarlo bien económico. Tangible es por consiguiente un computador, un carro, una casa, zapatos y muchísimas otras mercancías que son consideradas bienes económicos. Por otro lado, para que un producto adquiera la categoría de bien económico, debe ser satisfactor de una necesidad o un deseo”. |  La tangibilidad es una característica muy esencial de un objeto para considerarlo bien económico.
Tangible es por consiguiente un computador, un carro, una casa, zapatos y
muchísimas otras mercancías que son consideradas bienes económicos.
Por otro lado, para que un producto adquiera la categoría de bien económico,
debe ser satisfactor de una necesidad o un deseo.

Table 2: Examples of text segments extracted from a paragraph.

In the third phase of our research, we formulated a set of clear, exemplified
guidelines. These guidelines outlined the attributes within a text segment
that required simplification and the necessary actions to craft a more concise
text version. Our approach to designing these textual simplification
guidelines was rigorous. We conducted an in-depth review of existing
literature, specifically focusing on the Simplext corpus developed by the
Diles Group at the Autonomous University of Madrid [Anula,2011]. This literary
exploration was further enriched by collaborations with specialists in text
adaptation for the visually impaired and linguistic experts, ensuring the
fine-tuning of simplification rules vital for optimal compatibility with
screen readers. Table A1 of the Appendix shows an extract from the manual. The
second column of the appendix has the name of attributes that should be
simplified, and the third column has a summary of the instructions used to
solve the complexity generated by the attribute. For example, the first
attribute in column two is named “word frequency” and the instruction in the
third column is to replace low-frequency words with commonly used words. In
summary, these instructions are the guidelines to generate simple text
segments from the original text segments. A total of 21 attributes with their
instruction were generated.

Finally, six advanced philology students were recruited to simplify text
segments using a manual, producing 5,314 pairs of complex/straightforward text
segments. The most frequent attributes requiring simplification were
superfluous words, word length, and complex lexical expressions, as depicted
in Figure 1. Less common attributes, like lexical ambiguity and compound
subjects, were underrepresented. The dataset, potentially biased towards more
prevalent attributes, reflects the domain-specific nature of the source texts.
For example, the abstract nature of finance implies less frequent use of
visual references compared to longer word lengths typical of financial jargon.
The final dataset, including original texts, simplified versions, and
identified attributes, is available online.

Figure 1: Histogram of the the simplification rules used to generate the
manually simplified dataset.

###  2.2 Benchmarking against machine-generated simplifications: GPT-3, Tuner
and MT5

Different methods for text simplification have been developed in the
literature, with different approaches; from rules-based systems to Large
Language Models (LLM). To identify what models are worth to test, we developed
a small qualitative test, with the simplification of 13 text segments using
the following systems: Generative Pre-trained Transformer (GPT)-3, MT5, Tuner,
FastChat, Alpaca13b and Llama 13b. A sample of the results can be seen in
Table A2 of the Appendix. In most of the examples GPT-3 outputs the most
aggressive simplifications, shortening the text segments considerably and
keeping the main idea of the segments. Llama presents inconsistent text
segments, including even words in English, as seen in the first example.
Alpaca also presents results with low semantics correctness. Similarly,
FastChat fails to deliver semantically correct results. MT5 and Tuner
delivered semantically and syntactically correct results. MT5 was able to
generate more simple text segments when compared to Tuner, as the former tends
to leave the text segment unchanged. According to our evaluation, GPT-3
yielded the best results according to correctness, semantic preservation and
simplification. Also Tuner and MT5 achieved outputs with considerable
correctness, unlike the rest of the models (FastChat, Llama and Alpaca), which
often failed in producing syntactically and semantically correct outputs.
Other LLM s were tested: Bard does not work in Spanish by the writing time of
this work, Mosaic MPT-7B model (see
https://huggingface.co/spaces/mosaicml/mpt-7b-chat) did not work well in our
preliminary tests. From the previous analysis we decided to select GPT-3, MT5
and Tuner for the second step.

As a second step in the implemented methodology of this work, we evaluated
GPT-3, MT5 and Tuner to perform automatic text simplification using the same
source text segments built in the previous. We aim to evaluate the quality of
each one of the tested models against the manually generated data. This can
shed light on how close automatic systems are to manual simplifications in a
specific domain such as finance education for blind people. Data augmentation
policies for example can be formulated upon the measured performance.

Therefore, we generated the additional text-simplifications datasets

  * •

Financial Education Text Dataset simplified by Tuner: built using the Tuner
system developed in [Ferrés, Saggion, and Gómez Guinovart,2017].

  * •

Financial Education Text Dataset simplified by MT5: Generated with the MT5
simplification model proposed in [Štajner, Sheang, and Saggion,2022].

  * •

Financial Education Text Dataset Simplified with GPT-3: using the GPT-3 model
described in [Brown et al.,2020]. By the time of writing this work, the model
was not available for free usage by OpenAI. We used the following prompt
Simplifique el siguiente texto en español:. Only one prompt result for each
input segment was included in the dataset.

###  2.3 Quality evaluation of the datasets

Three kind of evaluations were generated. First, automatic simplicity metrics
were computed for each dataset. As previously mentioned, we categorized these
metrics into three classes. The following subset of metrics were selected for
each class:

  * •

Reference-less metrics for single text segments: We selected specific metrics
for Spanish: the Fernandez-Huerta, Gutierrez-Polini and Szigriszt Pazos
indices. As complementary measurements, we also report the number of mono and
polysyllables, as also the number of syllables and words. The metrics of this
category were imported from the TextStats package
https://pypi.org/project/textstat/ [Bansal and Aggarwal,2021].

  * •

Reference-less simple/complex comparison metrics: As for the previously
mentioned metrics which account for a comparison between the simple and
complex sentences, we measured the compression ratio, number of sentence
splits, the Levenshtein similarity index, the number of exact copies and the
additions and deletions performed. We used the Easier Automatic Sentence
Simplification Evaluation (EASSE) python package [Alva-Manchego et al.,2019]
to calculate them. To measure the semantic preservation, we used the cosine
distance between the Spanish BERT (BETO) embeddings of the simple and complex
sentences. BETO is a Bidirectional Encoder Representations from Transformers
(BERT) architecture pre-trained in a Spanish corpus [Cañete et al.,2020]. For
sentence embedding extraction we used the library developed in [Reimers and
Gurevych,2019] along with the weights of BETO, made available in
espejelomar/sentece-embeddings-BETO through the HuggingFace application
programming interface.

  * •

Reference based metrics: As for the quantification of text simplicity based
upon 2 or more human created references, we use the System output Against
References and against the Input sentence (SARI) and Bilingual Evaluation
Understudy Score (BLEU) scores. We used the implementations of these metrics
provided also in the EASSE python package.

  * •

As a complementary reference-based measurement, we present metrics of
discrimination between the complex and simple text segments to determine how
much the simplified sentences differ from the original ones. The
discrimination metrics are the average Accuracy and F1 score of Random Forest
and Beto models trained with each dataset to classify the text segments into
simple or complex (the complex is the original text segment). A higher
Accuracy or F1 score of the classifier means that the text segments simplified
are easy to discriminate from the original segments.

We fine-tune the Beto model for the binary classification into simple and
complex with each dataset. The Random Forests were also trained for the same
tasks with each dataset, using TF-IDF. The training was executed with 10
partitions (cross-validation train/test) that were the same for each dataset.
Therefore, the index of the text segments in the training and testing were the
same between models. The average accuracy and F1 score of tests were presented
as metrics of discrimination between the complex and simple text segments.

In the case of BETO models, the best values of the hyperparameters learning
rate, weight decay, batch size, and the scheduler were selected using weights
and biases suite with Bayesian optimization. The training dataset in each
partition was divided into two subsets: one subset to compute the weights of
the last layer of BETO (75% of the training dataset, 3986 observations) and
the other part (25% of the training dataset, 1328 observations) to define when
the model should stop the training using early stopping with the patience of
two of the cross-entropy loss function. In the case of Random Forest models,
we selected the best values of the number of trees, maximum number of
features, and minimum sample of leaf, using Bayesian optimization and cross-
validation of 5 folds with each training dataset.

  * •

Finally, to incorporate human judgment of the text segments for each system
and the manual simplifications, we solicited feedback from two visually
impaired subjects on the simplification quality of N = 100 text segments. Both
subjects possess extensive expertise in text simplification tailored for blind
or visually impaired audiences. They provided their input after completing an
informed consent form, ensuring adherence to the ethical considerations of
their participation in this study segment. Each evaluator was asked to reply
the following items:

    * –

The output text segment is simpler than the original?

    * –

The output text segment is simpler preserves the meaning of the original?

    * –

The output text segment is better understood than the original?

    * –

The output text is more suitable than the original for blind or visually
impaired people?

The items were answered by the two judges using a Likert scale from 1 to 5,
where 1 means total agree with the affirmation of the item, 2 agree, 3
neutral, 4 disagree and 5 totally disagree. For each source text segment, four
simplification proposals were presented to the judges: the manual proposal,
the GPT-3, tuner and MT5 outputs. The four text simplification proposals were
sorted randomly to neutralize a possible initial order effect.

##  3 Results

In Subsection 3.1 the results for the calculated metrics are depicted. We
develop the corresponding analysis in order to shed light about the quality of
the automatic simplifications.

###  3.1 Simplifications metrics for the manual and automatic simplification
datasets

The first set of experiment results show the output of computing the different
reference-less lexical simplicity metrics, the simple/complex comparison of
the text segments with no reference, and the reference based metrics, as
described in Section 2.3. We calculate such measures to quantitatively compare
how the simplification was carried out for the different tested systems. We
also compute the metrics for the manually simplified dataset as a reference.
Therefore we include the results for the Manual, GPT-3, MT5 large and the
rule-based system tested. Finally the results for the human based judgements
of a subset of text segments for each generated dataset are described.

####  3.1.1 Reference-less lexical metrics results

Tables 3, 4, 5 and 6 describe the yielded results of the selected lexical
reference-less metrics. As previously mentioned the Fernandez-Huerta,
Szigriszt-Pazos, Gutierrez-Polini metrics with higher values attempt to
describe a better simplification. As for the word count, polysyllable count in
general, the lower, a better text simplification, with a number of exceptions.

| $\mu_{s}$ | $\sigma_{s}$ | $\mu_{c}$ | $\sigma_{c}$
---|---|---|---|---
fernandez_huerta | 87.13 | 14.88 | 79.09 | 20.16
szigriszt_pazos | 83.68 | 15.12 | 75.88 | 20.11
gutierrez_polini | 39.38 | 6.95 | 37.46 | 8.33
crawford | 3.41 | 1.32 | 3.72 | 1.20
syllable count | 37.51 | 22.73 | 46.12 | 27.51
word count | 22.01 | 13.14 | 27.59 | 16.03
poly count | 4.32 | 3.27 | 5.11 | 3.82
mono count | 12.27 | 7.82 | 15.97 | 9.63

Table 3: Statistics for the Manually simplified dataset for the reference-less
with no comparison metrics.

Table 3 shows the lexical complexity metrics for both the simple and complex
sentences from the manually simplified dataset. As for the Fernandez-Huerta,
Szigriszt-Pasos and the Gutierrez-Polini metrics, a trend where its values are
higher in average for the simple sentences can be seen. Using a Wilcoxon-Mann-
Whitney statistical significance test, all of the aforementioned metrics
present a statistically significant difference between the simple and complex
tests. This using a confidence interval of 95%.

A similar behavior is described in Table 5 for the GPT-3 generated dataset,
where all the derived metrics yield a higher value for the simple sentences.
Also, statistical significance of such metrics between the average of the
simple and complex sentences is achieved. The average difference for the
derived metrics between the simple and complex sentences is slightly lower
when compared to the results yielded in the manually generated dataset. This
suggests that manually simplification is better than the GPT-3 generated
simplifications.

The Tuner generated dataset exhibits a weaker lexical simplification
efectiveness, as seen in Table 6, where the average difference between the
derived metrics for the simple and complex sentences is lower when compared to
the GPT-3 and Manually generated dataset.

As for the Mt5 simplified dataset, the trend of higher values for the
simplified sentences is also visible as seen in Table 4. However, in this case
there is no statistical significance for all of them. This according to the
Wilcoxon-Mann-Whitney test carried out.

The highest improvements on lexical simplicity according to the tested metrics
were yielded by the manually and GPT-3 generated datasets by considerably
larger margin. Both Tuner and MT5 achieved the lowest values, suggesting that
both systems are considerably worse at lexical simplification.

| $\mu_{s}$ | $\sigma_{s}$ | $\mu_{c}$ | $\sigma_{c}$
---|---|---|---|---
fernandez_huerta | 83.44 | 14.20 | 82.40 | 17.65
szigriszt_pazos | 80.14 | 14.41 | 79.14 | 17.66
gutierrez_polini | 38.91 | 6.73 | 38.55 | 7.43
crawford | 3.71 | 1.15 | 3.61 | 1.17
syllable count | 39.03 | 13.07 | 48.97 | 27.61
word count | 23.37 | 7.59 | 29.36 | 16.13
poly count | 4.26 | 2.33 | 5.41 | 3.86
mono count | 13.45 | 5.06 | 17.04 | 9.74

Table 4: Statistics for the mT5 text simplifications reference-less metrics.

| $\mu_{s}$ | $\sigma_{s}$ | $\mu_{c}$ | $\sigma_{c}$
---|---|---|---|---
fernandez_huerta | 86.02 | 16.73 | 79.09 | 20.16
szigriszt_pazos | 82.67 | 16.97 | 75.88 | 20.11
gutierrez_polini | 39.55 | 7.66 | 37.46 | 8.33
crawford | 3.44 | 1.40 | 3.72 | 1.20
syllable count | 37.41 | 23.48 | 46.12 | 27.51
word count | 22.22 | 13.57 | 27.59 | 16.03
poly count | 4.18 | 3.33 | 5.11 | 3.82
mono count | 12.64 | 8.06 | 15.97 | 9.63

Table 5: Statistics for the GPT-3 text simplifications referenceless metrics.

| $\mu_{s}$ | $\sigma_{s}$ | $\mu_{c}$ | $\sigma_{c}$
---|---|---|---|---
fernandez_huerta | 85.80 | 16.17 | 82.13 | 17.52
szigriszt_pazos | 82.48 | 16.31 | 78.86 | 17.52
gutierrez_polini | 39.93 | 7.11 | 38.49 | 7.38
crawford | 3.44 | 1.20 | 3.64 | 1.14
syllable count | 50.48 | 28.99 | 49.20 | 27.62
word count | 30.17 | 16.80 | 29.48 | 16.12
poly count | 5.52 | 4.04 | 5.44 | 3.87
mono count | 17.21 | 9.97 | 17.10 | 9.73

Table 6: Statistics for the Tuner text simplifications reference-less metrics.

###  3.2 Simple-complex comparison based metrics

Table 7 shows the simple-complex comparison based metrics calculated for the 4
datasets: the compression ratio, the number of sentence splits, the
Levenshtein similarity, the number of exact copies, the additions/deletions
proportion, the lexical complexity score and the BERT and BETO based
similarity scores. For the compression ratio, the Manually and GPT-3 generated
datasets show a clear lead over the MT5 and Tuner generated datasets. This
suggests how the two former systems are not as effective as the manual and
GPT-3 based simplifications, fro m a lexical simplification perspective. The
SBert and SBETO based similarity scores suggest that the semantics of the
simplified text segments are fairly similar for the GPT-3 and manual
simplifications, which advocates for a consistent simplification process of
both systems. In the case of the Tuner and MT5 systems, the very high Sbert
and SBETO similarities of the source and simplified text segments, along with
the higher compression ratio and the number of exact copies suggest for a
frequently absent text simplification. This means that identical simple text
segments are generated for the MT5 and Tuner systems, when compared to the
source text segment.

Table 7 also shows that the exact copies metric for the GPT-3 and manual
simplification systems are 0 for both, meaning that there is always an
effective text transformation performed in both cases. The proportion of
additions and deletions for GPT-3 and the manual outputs also correlates with
this trend, as both generate text segments with significantly higher
transformations of both types, when compared to the Tuner and MT5 systems. The
Levenshtein similarity score strengthens this argument, showing lower scores
for the manually and GPT-3 when compared to the MT5 and Tuner systems.
Finally, according to the lexical complexity score, also the GPT-3 and Manual
systems managed to generate less lexically complex text segments, when
compared to the Tuner and MT5 systems. A similar trend regarding the sentence
splits metric is revealed, where the manually and GPT-3 systems are able to
perform more sentence splits when compared to the Tuner and MT5 systems.

For the 9 tested simple-complex text segment based metrics we carried out a
normality and homoscedasticity (equal variances) statistical test, to choose
which statistic tests to use to compare these 9 variables across the 4
treatments (manually, MT5, Tuner and GPT-3 simplified text segments) used in
the experimental setting. We used the pingouin python package to implement the
normality and the homoscedasticity tests. According to them, no of the
variables for the 4 treatments are normal or fulfill homoscedasticity.
Therefore we chose to carry out a Wilcoxon signed test.

Table 7 shows the statistical analysis using the aforementioned Wilcoxon test
of all the datasets for each of the 9 evaluated metrics. Value pairs within
the same row underlined or with the * symbol are not statistically according
to the Wilcoxon signed test. It describes how for every comparison between
text simplification systems, there is a statistically significant difference,
except in the case when comparing the compression ratio between the manually
and the GPT-3 generated datasets, as also is the case for the comparison
between both systems using the lexical complexity score. Finally, when
comparing the MT5 and the Tuner datasets, the lexical complexity score is also
not statistically significantly different.

In summary, according to Table 7 we can consider that the manual
simplifications are slightly better according to the Levenshtein similarity,
sentence splits and the number of deletions and additions, when compared to
the GPT-3 based simplification system. As for the compression ratio and
lexical complexity score, the GPT-3 system performs very similar when compared
to the manual simplifications. Both the manual and GPT-3 text simplifications
are considerably better compared to the Tuner and MT5 simplifications.

| Manually Ds. | GPT-3 Ds. | Tuner Ds. | MT5 Ds.
---|---|---|---|---
Compression ratio | 0.84$\pm$0.05 | 0.84$\pm$0.13 | 1.02$\pm$0.01 | 0.89$\pm$0.03
Sentence splits | 1.36$\pm$0.49 | 1.16$\pm$0.59 | 1.26$\pm$0.27 | 0.9$\pm$0.1
Levenshtein similarity | 0.76$\pm$0.03 | 0.77$\pm$0.03 | 0.94$\pm$0.01 | 0.91$\pm$0.02
Exact copies | 0.00$\pm$0.0 | 0.00$\pm$0.0 | 0.37$\pm$0.23 | 0.29$\pm$0.21
Additions proportion | 0.18$\pm$0.02 | 0.16$\pm$0.03 | 0.07$\pm$0.01 | 0.05$\pm$0.01
Deletions proportion | 0.37$\pm$0.04 | 0.34$\pm$0.04 | 0.05$\pm$0.01 | 0.17$\pm$0.04
Lexical complexity score | 9.55$\pm$0.58 | 9.55$\pm$0.59 | 9.61$\pm$0.45* | 9.62$\pm$0.44*
Sbert cos. sim. | 0.87$\pm$0.01 | 0.89$\pm$0.01 | 0.97$\pm$0.00 | 0.94$\pm$0.01
Sbeto cos. sim. | 0.92$\pm$0.01 | 0.92$\pm$0.01 | 0.98$\pm$0.00 | 0.96$\pm$0.00

Table 7: Results for all the tested systems using the reference-less
comparison based metrics. Value pairs in the same row underlined or with * are
not statistically different according to the Wilcoxon test, with a $p=0.05$.

###  3.3 Reference based metrics

As for the reference based metrics SARI and BLEU, are depicted in Tables 8 and
9, respectively. Specifically for the BLEU metric, Table 9 reveals that the
manual simplifications perform best through all the dataset variation cases:
using two text segment references (with a sample size of 149 observations) up
to five text segment references (with a sample size of 94 observations). In
second place, the GPT-3 simplifications are the better ones, using the SARI
metric according to Table 8 for all the tested datasets with 2 to 5
references. The decrease of the SARI value when compared to the manual
simplifications is around 6 units. In third place, the MT5 simplifications
yield a value near to 33, for all the tested dataset samples with different
number of references. Similarly, a drop of around 6 units for the SARI metric
is perceived between the GPT-3 and MT5 simplifications. Finally, the
simplifications with the lowest SARI values in average, were yielded by the
Tuner system.

As for the statistical relevance of the difference between each system
according to the SARI and BLEU metrics, we verified that nor the BLEU or SARI
metric present a normal distribution and equal variances for both the manual
and GPT-3 systems. Therefore, similar as the previous set of metrics, we
executed a non-parametric Wilcoxon signed test to verify whether there is a
statistically significant difference between the manual and GPT-3 metrics. In
the same fashion as the results for the previous metrics set, we marked in
Tables 8 and 9 the pairs of statistically similar results (using a $p=0.05$ as
criteria). According to the statistical analysis results, only the BLEU metric
based comparison of the manual and GPT-3 systems can be considered not to be
statistically different.

N. r. | N. | Manual | GPT-3 | MT5 | Tuner
---|---|---|---|---|---
2 | 149 | 48.7$\pm$12.6 | 41.6$\pm$12.2 | 34$\pm$11.3 | 26.4$\pm$7.9
3 | 134 | 48.2$\pm$11.9 | 42.4$\pm$11.7 | 34.1$\pm$11.4 | 26.7$\pm$8.1
4 | 119 | 48.1$\pm$11.9 | 42.1$\pm$11.3 | 33.9$\pm$11.2 | 26.7$\pm$8.2
5 | 94 | 48.8$\pm$10.7 | 41.1$\pm$10 | 34.4$\pm$11.1 | 27.4$\pm$8.5

Table 8: Average and standard deviation of the SARI scores for the simplified
text segments generated by the manual, GPT-3, Tuner and MT5 systems. The
higher the better.

N. r. | N. | Manual | GPT-3 | MT5 | Tuner
---|---|---|---|---|---
2 | 149 | 49$\pm$23.1 | 44.9$\pm$22.1 | 56.4$\pm$17.8 | 51.1$\pm$17.7
3 | 134 | 48.5$\pm$22.5 | 45.5$\pm$22.1 | 56.2$\pm$17.1 | 51.4$\pm$17.1
4 | 119 | 47.5$\pm$22.3 | 44.9$\pm$22.1 | 54.9$\pm$17 | 50.4$\pm$17.1
5 | 94 | 47.9$\pm$20.9 | 44.81$\pm$21.6 | 55.6$\pm$16 | 51$\pm$15.2

Table 9: Average and standard deviation of the BLEU scores for the simplified
text segments generated by the manual, GPT-3, Tuner and MT5 systems. The
higher the better. Value pairs underlined are no statistically different
according to the Wilcoxon test with a $p=0.05$

###  3.4 Discrimination between complex and simple versions

Table 10shows the performance metrics of the models trained to classify text
segments as complex or simple. Our assumption is that the classifier should
yield higher accuracy with better simplifications. Using the Wilcoxon test, we
found that the Beto and Random Forest classifiers trained with the manually
dataset, have better accuracy and F1 score than the models trained with the
other datasets. Therefore, we can conclude that the manual dataset has more
clear discrimination between simple and complex sentences. It is important to
clarify that we considered the original text segments as complex because they
had complex attributes that had to be simplified according to the advanced
students in Philology.

On the other hand, we found that GPT-3 classifiers tend to get similar
performance to turner classifiers, and better performance to MT5 with the
exception of the accuracy in BETO. Therefore, we conclude that GPT-3 and
Turner discriminate between complex and simple text segments in a similar way.

Dataset BETO Random Forest Acc. F1-score Acc. F1-score Manual 0.830 $\pm$0.012
0.830$\pm$0.012 0.681 $\pm$0.011 0.681$\pm$0.011 GPT-3 0.753$\pm$0.009
0.750$\pm$0.010 0.637$\pm$0.010 0.636$\pm$0.012 mT5-large 0.724$\pm$0.012
0.697$\pm$0.010 0.600$\pm$0.019 0.597$\pm$0.018 Tuner 0.744$\pm$0.006
0.765$\pm$0.008 0.630$\pm$0.007 0.629$\pm$0.007

Table 10: 10-Folds mean statistics of the accuracy and F1-score for the BETO
based classifier and Random Forest model with a TF-IDF based representation.
Value pairs underlined are no statistically different according to the
Wilcoxon test with a $p=0.05$

###  3.5 Human evaluation based metrics

Table 11 presents the human judgment evaluation results, showing that both
manual and GPT-3 datasets received similar and optimistic assessments. The
first judge’s average scores, being less than two or close to it, suggest that
the simplified text segments from both datasets generally enhance the original
texts in terms of simplicity, understandability, and appropriateness for blind
people while retaining the original meaning. The second judge assigned
slightly higher scores, still below 3, indicating similar improvements. While
both datasets scored comparably, specific differences emerged: the manual
dataset was rated higher for preserving original sentence meanings, whereas
the GPT-3 dataset excelled in understandability. The other two datasets
received lower ratings across all four dimensions; in many instances, the
simplified versions were not deemed superior to the originals.

Manually Ds. GPT-3 Ds. Tuner Ds. MT5 Ds. Judge1 Simplicity 1.83$\pm$1.19
1.83$\pm$1.26 4.30$\pm$1.09 2.62$\pm$1.81 Meaning Pres. 1.46$\pm$1.30
1.67$\pm$1.30 3.37$\pm$1.66* 3.44$\pm$1.79* Understand. 2.05$\pm$1.30
1.93$\pm$1.22 4.26$\pm$1.14 2.93$\pm$ 1.70 Suitability 2.00$\pm$1.35
2.57$\pm$1.48 4.34$\pm$1.12 2.96 $\pm$1.72 Judge2 Simplicity 2.46$\pm$1.48
2.39$\pm$1.48 4.55$\pm$0.94 3.46$\pm$1.51 Meaning Pres. 1.99$\pm$1.44
2.07$\pm$1.54 4.02$\pm$1.50* 4.00$\pm$1.40* Understand. 2.59$\pm$1.45
2.57$\pm$1.48 4.58$\pm$0.87 4.01$\pm$1.29 Suitability 2.56$\pm$1.44
2.53$\pm$1.43 4.55$\pm$0.88 4.02$\pm$1.26

Table 11: Average of human scores based judgements using the Likert scale. The
lower the better. Value pairs in the same row underlined or with * are not
statistically different according to the Wilcoxon test, with a $p=0.05$.

##  4 Conclusions

In this work we approached the problem of text simplification within the
financial education domain for blind people in Spanish with a datacentric
perspective. The presented dataset, which we refer to as FEINA, is the largest
dataset up to date to our knowledge, consisting of simple/complex text segment
pairs generated in a completely manual fashion, following a set of
simplification rules also described in this work.

We are concerned for the lack of high quality simplification datasets in
Spanish for specific domains, in this case for financial education.
Simplifying financial education texts in Spanish can increase the readability
of this kind of materials for a specific population such as visually disabled
people. We adhere to the conception of Artificial Intelligence (AI) as a task
creator instead of a mere automator, as argued by Acemoglu et. al. in
(Acemoglu and Restrepo, 2020). Using AI to increase inclusivity is an
important endeavour which requires high quality and open datasets.

In this work we carried out an extensive datacentric analysis of the tested
simplification systems. Four types of metrics were calculated: reference-less,
reference-less pair based, reference and human evaluation based. The manual
and GPT-3 generated simplifications were the best according to all the types
of metrics used.

The findings indicate that while our manually curated dataset sets a high
standard, GPT-3’s results are close in quality, with MT5 and Tuner needing to
catch up. These insights suggest GPT-3’s potential in augmenting
simplification datasets with minimal manual adjustments, enhancing financial
education accessibility for the visually impaired in Spanish-speaking regions

Within the framework of performance evaluation, it is observed that the
reference-based SARI metric suggests a significantly better performance of
manual simplifications compared to GPT-3 results. However, our findings also
indicate no significant difference between the simplifications generated by
GPT-3 and the manual ones for visually impaired subjects. This result invites
reflection on the suitability of current metrics for evaluating the
effectiveness of text simplification systems, particularly regarding
preserving meaning and lexical simplification.

In summary, this study offers valuable insight and opens new paths for future
work in automatic text simplification in Spanish within the domain of
financial education for visually impaired individuals. It is a call to action
for researchers in artificial intelligence and natural language processing to
promote inclusion by developing increasingly effective and accessible systems
and approaches.

## References

  * [Alva-Manchego et al.,2019] Alva-Manchego, F., L. Martin, C. Scarton, and L. Specia.  2019\.  EASSE: Easier automatic sentence simplification evaluation.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, pages 49–54, Hong Kong, China, November. Association for Computational Linguistics.
  * [Anula,2011] Anula, A.  2011\.  Pautas básicas de simplificación textual y diseño del corpus simplext.  Technical report, Technical report, Grupo DILES. Madrid, Spain: Universidad Autónoma de Madrid.
  * [Atkinson and Messy,2013] Atkinson, A. and F.-A. Messy.  2013\.  Promoting financial inclusion through financial education: Oecd/infe evidence, policies and practice.  OECD Working Papers on Finance, Insurance and Private Pensions No. 34.
  * [Bansal and Aggarwal,2021] Bansal, S. and C. Aggarwal.  2021\.  textstat.  Retrieved September 1st.
  * [Barbieri et al.,2005] Barbieri, T., A. Bianchi, L. Sbattella, F. Carella, M. Ferra, et al.  2005\.  Multiabile: A multimodal learning environment for the inclusion of impaired e-learners using tactile feedbacks, voice, gesturing, and text simplification.  Assist Technol: From Virtuality to Real, 16(1):406–10.
  * [Benavides-Mata and Calderon-Ramirez,2022] Benavides-Mata, I. and S. Calderon-Ramirez.  2022\.  Improving semi-supervised deep learning by using automatic thresholding to deal with out of distribution data for covid-19 detection using chest x-ray images.  arXiv preprint arXiv:2211.02142.
  * [Brown et al.,2020] Brown, T., B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al.  2020\.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901.
  * [Calderon-Ramirez et al.,2021a] Calderon-Ramirez, S., R. Giri, S. Yang, A. Moemeni, M. Umana, D. Elizondo, J. Torrents-Barrena, and M. A. Molina-Cabello.  2021a.  Dealing with scarce labelled data: Semi-supervised deep learning with mix match for covid-19 detection using chest x-ray images.  In 2020 25th International Conference on Pattern Recognition (ICPR), pages 5294–5301. IEEE.
  * [Calderon-Ramirez et al.,2021b] Calderon-Ramirez, S., D. Murillo-Hernandez, K. Rojas-Salazar, L.-A. Calvo-Valverd, S. Yang, A. Moemeni, D. Elizondo, E. López-Rubio, and M. A. Molina-Cabello.  2021b.  Improving uncertainty estimations for mammogram classification using semi-supervised learning.  In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE.
  * [Calderon-Ramirez et al.,2022a] Calderon-Ramirez, S., D. Murillo-Hernandez, K. Rojas-Salazar, D. Elizondo, S. Yang, A. Moemeni, and M. Molina-Cabello.  2022a.  A real use case of semi-supervised learning for mammogram classification in a local clinic of costa rica.  Medical & biological engineering & computing, 60(4):1159–1175.
  * [Calderon-Ramirez et al.,2022b] Calderon-Ramirez, S., L. Oala, J. Torrents-Barrena, S. Yang, D. Elizondo, A. Moemeni, S. Colreavy-Donnelly, W. Samek, M. A. Molina-Cabello, and E. Lopez-Rubio.  2022b.  Dataset similarity to assess semisupervised learning under distribution mismatch between the labeled and unlabeled datasets.  IEEE Transactions on Artificial Intelligence, 4(2):282–291.
  * [Calderon-Ramirez, Yang, and Elizondo,2022] Calderon-Ramirez, S., S. Yang, and D. Elizondo.  2022\.  Semisupervised deep learning for image classification with distribution mismatch: A survey.  IEEE Transactions on Artificial Intelligence, 3(6):1015–1029.
  * [Calderon-Ramirez et al.,2022] Calderon-Ramirez, S., S. Yang, D. Elizondo, and A. Moemeni.  2022\.  Dealing with distribution mismatch in semi-supervised deep learning for covid-19 detection using chest x-ray images: A novel approach using feature densities.  Applied Soft Computing, 123:108983.
  * [Calderon-Ramirez et al.,2021] Calderon-Ramirez, S., S. Yang, A. Moemeni, D. Elizondo, S. Colreavy-Donnelly, L. F. Chavarria-Estrada, and M. A. Molina-Cabello.  2021\.  Correcting data imbalance for semi-supervised covid-19 detection using x-ray chest images.  Applied Soft Computing, 111:107692.
  * [Cañete et al.,2020] Cañete, J., G. Chaperon, R. Fuentes, J.-H. Ho, H. Kang, and J. Pérez.  2020\.  Spanish pre-trained bert model and evaluation data.  In PML4DC at ICLR 2020.
  * [Coster and Kauchak,2011] Coster, W. and D. Kauchak.  2011\.  Simple english wikipedia: a new text simplification task.  In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 665–669.
  * [Devlin et al.,2018] Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova.  2018\.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805.
  * [Dmitrieva and Tiedemann,2021] Dmitrieva, A. and J. Tiedemann.  2021\.  Creating an aligned russian text simplification dataset from language learner data.  In Proceedings of the 8th Workshop on Balto-Slavic Natural Language Processing. ACL Anthology.
  * [Fajardo et al.,2015] Fajardo, I., A. Ferrer, V. Ávila, L. Gil, L. Salmerón, and M. Gómez.  2015\.  Aportes de la psicolingüística a la simplificación de textos contributions from psycholinguistics to text simplification.  IX JORNADAS CIENTÍFICAS INTERNACIONALES DE INVESTIGACIÓN SOBRE PERSONAS CON DISCAPACIDAD.
  * [Ferrés and Saggion,2022] Ferrés, D. and H. Saggion.  2022\.  Alexsis: a dataset for lexical simplification in spanish.  In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 3582–3594.
  * [Ferrés, Saggion, and Gómez Guinovart,2017] Ferrés, D., H. Saggion, and X. Gómez Guinovart.  2017\.  An adaptable lexical simplification architecture for major Ibero-Romance languages.  In E. Bender, H. Daumé III, A. Ettinger, and S. Rao, editors, Proceedings of the First Workshop on Building Linguistically Generalizable NLP Systems, pages 40–47, Copenhagen, Denmark, September. Association for Computational Linguistics.
  * [for Financial Education,2017] for Financial Education, N. E.  2017\.  Tus gastos, tus ahorros, tu futuro: Guía de preparación financiera para principiantes.  National Endowment for Financial Education.
  * [Gil,2008] Gil, G.  2008\.  Libro maestro de educación financiera. sistema para vivir mejor.
  * [Gonzales et al.,2021] Gonzales, A. R., N. Spring, T. Kew, M. Kostrzewa, A. Säuberli, M. Müller, and S. Ebling.  2021\.  A new dataset and efficient baselines for document-level text simplification in german.  In Proceedings of the Third Workshop on New Frontiers in Summarization, pages 152–161.
  * [Hossain and Ahnaf,2021] Hossain, N. and A. Ahnaf.  2021\.  Bert-based text simplification approach to reduce linguistic complexity of bangla language.  In 2021 International Conference on Intelligent Technology, System and Service for Internet of Everything (ITSS-IoE), pages 1–5. IEEE.
  * [Izaguirre Olmedo, Carhuancho Mendoza, and Silva Siu,2020] Izaguirre Olmedo, J., I. M. Carhuancho Mendoza, and D. Silva Siu.  2020\.  Finanzas para no financieros.  GUAYAQUIL/UIDE/2020.
  * [Maruyama and Yamamoto,2020] Maruyama, T. and K. Yamamoto.  2020\.  Extremely low-resource text simplification with pre-trained transformer language model.  International Journal of Asian Language Processing, 30(01):2050001.
  * [Monge, Alvarado, and Rojas,2023] Monge, M. A., L. S. Alvarado, and G. R. Rojas.  2023\.  Análisis de idoneidad de un banco de ítems para personas con discapacidad auditiva y visual en una prueba estandarizada de acceso a la educación superior en costa rica.  Actualidades Investigativas en Educación, 23(2):1–35.
  * [Ott et al.,2019] Ott, M., S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli.  2019\.  fairseq: A fast, extensible toolkit for sequence modeling.  arXiv preprint arXiv:1904.01038.
  * [Palmero Aprosio et al.,2019] Palmero Aprosio, A., S. Tonelli, M. Turchi, M. Negri, and A. Di Gangi Mattia.  2019\.  Neural text simplification in low-resource conditions using weak supervision.  In Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation (NeuralGen), pages 37–44. Association for Computational Linguistics (ACL).
  * [Parmanto et al.,2005] Parmanto, B., R. Ferrydiansyah, A. Saptono, L. Song, I. W. Sugiantara, and S. Hackett.  2005\.  Access: accessibility through simplification & summarization.  In Proceedings of the 2005 international cross-disciplinary workshop on web accessibility (W4A), pages 18–25.
  * [Qiang et al.,2020] Qiang, J., Y. Li, Y. Zhu, Y. Yuan, and X. Wu.  2020\.  Lsbert: a simple framework for lexical simplification.  arXiv preprint arXiv:2006.14939.
  * [Reimers and Gurevych,2019] Reimers, N. and I. Gurevych.  2019\.  Sentence-bert: Sentence embeddings using siamese bert-networks.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11.
  * [Romero et al.,2022] Romero, M., S. Calderón-Ramírez, M. Solís, N. Pérez-Rojas, M. Chacón-Rivas, and H. Saggion.  2022\.  Towards text simplification in spanish: A brief overview of deep learning approaches for text simplification.  In 2022 IEEE 4th International Conference on BioInspired Processing (BIP), pages 1–7. IEEE.
  * [Saggion et al.,2011] Saggion, H., E. Gómez-Martínez, E. Etayo, A. Anula, and L. Bourg.  2011\.  Text simplification in simplext: Making texts more accessible.  Procesamiento del lenguaje natural, (47):341–342.
  * [Saggion et al.,2015] Saggion, H., S. Štajner, S. Bott, S. Mille, L. Rello, and B. Drndarevic.  2015\.  Making it simplext: Implementation and evaluation of a text simplification system for spanish.  ACM Transactions on Accessible Computing (TACCESS), 6(4):1–36.
  * [Sallam,2023] Sallam, M.  2023\.  The utility of chatgpt as an example of large language models in healthcare education, research and practice: Systematic review on the future perspectives and potential limitations.  medRxiv.
  * [Sauvan et al.,2020] Sauvan, L., N. Stolowy, C. Aguilar, T. François, N. Gala, F. Matonti, E. Castet, and A. Calabrèse.  2020\.  Text simplification to help individuals with low vision read more fluently.  In N. Gala and R. Wilkens, editors, Proceedings of the 1st Workshop on Tools and Resources to Empower People with REAding DIfficulties (READI), pages 27–32, Marseille, France, May. European Language Resources Association.
  * [Singh and Goyal,2021] Singh, M. and P. Goyal.  2021\.  Chartsight: An automated scheme for assisting visually impaired in understanding scientific charts.  In VISIGRAPP (5: VISAPP), pages 309–318.
  * [Štajner et al.,2022] Štajner, S., D. Ferrés, M. Shardlow, K. North, M. Zampieri, and H. Saggion.  2022\.  Lexical simplification benchmarks for english, portuguese, and spanish.  Frontiers in Artificial Intelligence, 5:991242.
  * [Štajner, Sheang, and Saggion,2022] Štajner, S., K. C. Sheang, and H. Saggion.  2022\.  Sentence simplification capabilities of transfer-based models.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 12172–12180.
  * [Tida, Srinivas, and Hsu,2022] Tida, V. Srinivas, and S. Hsu.  2022\.  Universal spam detection using transfer learning of bert model.  arXiv preprint arXiv:2202.03480.
  * [Todirascu et al.,2022] Todirascu, A., R. Wilkens, E. Rolin, T. François, D. Bernhard, and N. Gala.  2022\.  Hector: A hybrid text simplification tool for raw texts in french.  In 12th International Conference on Language Resources and Evaluation (LREC).
  * [Torres Salazar and Ramos Arriagada,2020] Torres Salazar, G. and R. Ramos Arriagada.  2020\.  Manual de finanzas personales y de familia. cómo usar bien mi dinero y el tuyo.
  * [Xu et al.,2016] Xu, W., C. Napoles, E. Pavlick, Q. Chen, and C. Callison-Burch.  2016\.  Optimizing statistical machine translation for text simplification.  Transactions of the Association for Computational Linguistics, 4:401–415.
  * [Xue et al.,2020] Xue, L., N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Siddhant, A. Barua, and C. Raffel.  2020\.  mt5: A massively multilingual pre-trained text-to-text transformer.  arXiv preprint arXiv:2010.11934.
  * [Zhu, Bernhard, and Gurevych,2010] Zhu, Z., D. Bernhard, and I. Gurevych.  2010\.  A monolingual tree-based translation model for sentence simplification.  In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1353–1361.

##  Appendix A Appendix

Original |  Attribute |  Guideline for simplification
---|---|---
1 |  Word Frequency |  Replace low-frequency words with commonly used words.
2 |  Abstract vs. Concrete Language |  When possible, replace words with abstract or less imaginable referents with concrete or imaginable referents. It can be suppressed if the word is in an unnecessary idea to convey the message.
3 |  Visual Reference Dependence |  When possible, replace discourse segments with visual references. It can be suppressed if the word is in an unnecessary idea to convey the message.
4 |  Ambiguity in Word Meaning |  Replace polysemic words according to their main meanings.
5 |  Unnecessary Words |  Eliminate words and phrases that are not necessary for understanding the message of the text.
6 |  Complex Phrases |  Replace phrases (complex lexical expressions) with their one-word equivalents.
7 |  Use of Nominalizations |  Replace nominalizations with verbs that express the same states, processes, and actions.
8 |  Conjunctions |  Coordinating: Preferably use the conjunctions ”y” and ”ni”. However, replacement is unnecessary in short sentences and those that do not employ other attributes that make the text difficult. Disjunctive : Use the conjunction ”o”. However, modification is only recommended when the marker is infrequently used or when a less explanatory style is used for a financial education text. Adversative : Use the conjunction ”pero” in affirmative constructions and ”sino” in negative constructions. However, modification is only recommended when the marker is infrequently used or when a less explanatory style is used for a financial education text. Other Coordination Connectors: Modify these coordination links only when the marker is infrequently used or when a less explanatory style is used for a financial education text.
9 |  Subordinating Conjunctions |  Relative: Use the simple relative forms ”que”, ”quien”, and ”cuanto”. Replace the simple form ”cuyo”. Temporal: Preferably use ”cuando” and ”mientras”. Locative: Use ”donde”. Modal: Use ”como” and ”según”. Conditional: Use ”si” and ”por si”, provided important logical expressions for financial texts are unaffected. Concessive: Use ”aunque”, or other frequently used ones such as ”a pesar de que” and ”incluso cuando”.
10 |  Past anterior and future perfect tense |  Use the forms of present, simple past, imperfect past, simple future, simple conditional, present perfect, pluperfect, and conditional perfect.
11 |  Subjunctive Mood Verb Forms |  Limit the use of subjunctive to its forms of present, imperfect past, simple future, and pluperfect.
12 |  Use of Gerund |  Replace the gerund forms in the sentences.
13 |  Abbreviations, Shortenings, Acronyms, and Initialisms |  Use full expressions instead of abbreviations, shortenings, acronyms, and initialisms.
14 |  Use of Numerical Expressions |  Write numerical expressions with letters.
15 |  Sentence Length |  Use segments of 20 words or less. Dividing the discourse segment into several segments is recommended.
16 |  Parenthetical Phrases |  Reserve use when they are essential for understanding the message.
17 |  Relative Clauses |  Reduce the number of relative clauses per sentence (maximum one).
18 |  Sentences of formal and aesthetic use |  Modify the use of these constructions when they are used solely for stylistic or aesthetic reasons.
19 |  Sentence Order |  Preferably use the SVO order.
20 |  Common Discourse Markers |  Use the most frequent markers in the cultural context.
21 |  Use of Complex Punctuation Marks |  Eliminate using semicolons, brackets, dashes, single quotes, and angle quotes. Do not change the semicolon to period if the linguistic units separated with the first punctuation mark are words or phrases (enumerating lexical elements).

Table A1: Attributes and Guidelines for Simplifying Discourse Segments of
Financial Texts for Visually Impaired Individuals

Original |  GPT-3 |  MT5 |  Tuner |  FastChat |  Alpaca13b |  Llama 13b
---|---|---|---|---|---|---
Cuando se intenta definir el dinero, las personas piensan en monedas o billetes, lo cual es muy reducido porque el dinero es más que eso. |  El dinero es más que monedas o billetes. |  Cuando se intenta definir el dinero, las personas piensan en monedas o billetes, lo cual es muy reducido. |  Cuando se intenta definir el dinero, las personas piensan en monedas o billetes, lo cual es muy reducido porque el dinero es mas que eso. |  Cuando se trata de definir el dinero, las personas pensan en monedas o billetes, pero el dinero es más que esos. No se trata sólo de monedas o billetes. |  El dinero es un concepto más amplio que simplemente monedas o billetes. Se define como una mediodinero es más que eso. Se define como una mediodinero es más que eso. |  The dinero es una forma de intercambiar bienes y servicios. Todos los bienes y servicios tienen un precio y se pueden comprar y vender. Algunos ejemplos de bienes son los hogares, los automóviles, los muebles de la casa, los alimentos y los cosméticos. Algunos ejemplos de servicios son la limpieza, la cocina, los negocios, la atención médica y la música.
A fin de mes, podrías tener unos $50 para depositar en tu cuenta de ahorro, que suman $600 al año (sin incluir los intereses, si los depositaras en una cuenta que pague intereses). |  A fin de mes, podrás depositar $50 en tu cuenta de ahorro, lo que sumará $600 al año. |  A fin de mes, podrías tener unos 50 para depositar en tu cuenta de ahorro, que suman $600 al año (sin incluir los intereses, si los depositaras en una cuenta que pague intereses). |  A fin de mes, podrías tener unos $50 para depositar en tu cuenta de ahorro. Esta tu cuenta de ahorro suman $600 al año (sin admitir los intereses, si los depositases en una cuenta). Esta una cuenta pague intereses. |  A finales de mes, puedes tener $50 para depositar en tu cuenta de ahorro, que puede agregarse a $600 al año (sin incluir los intereses, si los encuentres en una cuenta que pague intereses). |  Puedes tener unos $50 para depositar en tu cuenta de ahorro, que suman $600 al año (sin incluir los intereses, si los depositaras en una cuenta que pague intereses). |  Fin de mes podrías tener unos $50 para depositar en tu cuenta de ahorro, que suman $600 al año sin incluir los intereses, si los depositaras en una cuenta que pague intereses.
Ingresos de tipo residual: Son ingresos producto de rendimientos financieros, arriendos de bienes raíces, dividendos, acciones de empresas; es decir, las personas no tienen que realizar un trabajo para recibir su ingreso. |  Ingresos residuales: se generan a partir de los rendimientos financieros, los arrendamientos de inmuebles, los dividendos y las acciones de las empresas; es decir, las personas no tienen que realizar un trabajo para obtener este ingreso. |  Ingresos de tipo residual: Son ingresos producto de rendimientos financieros, arriendos de bienes raíces, dividos, acciones de empresas; es decir, las personas no tienen que realizar ese tipo de transacciones. |  Ingresos de tipo residual: Son ingresos producto de rendimientos financieros, arriendos de bienes raíces, dividendos, acciones de empresas; es decir, las personas no tienen que realizar un trabajo para recibir su ingreso. |  Residuos residuales: Son medios de ingreso que no se llevarán en acciones o dividendos, y son hechos a causa de rendimientos financieros. Las personas no tienen que hacer trabajo para recibir estos medios. |  Ingresos de tipo residual: Son ingresos producto de rendimientos financieros, arriendos de bienes raíces, dividendos, acciones de empresas; es decir, las personas no tienen que realizar un trabajo para recibir su ingreso. |  Ingresos de tipo residual: Son ingresos producto de rendimientos financieros, arriendos de bienes raíces, dividendos, acciones de empresas; es decir, las personas no tienen que realizar un trabajo para recibir su ingreso.

Table A2: Comparison of the Spanish text simplification datasets.