0 Research

# Empowering People with Intellectual and Developmental Disabilities through
Cognitively Accessible Visualizations

Keke Wu
University of North Carolina at Chapel Hill  e-mail<EMAIL_ADDRESS>Danielle Albers Szafir
University of North Carolina at Chapel Hill e-mail<EMAIL_ADDRESS>
###### Abstract

Data has transformative potential to empower people with Intellectual and
Developmental Disabilities (IDD). However, conventional data visualizations
often rely on complex cognitive processes, and existing approaches for day-to-
day analysis scenarios fail to consider neurodivergent capabilities, creating
barriers for people with IDD to access data and leading to even further
marginalization. We argue that visualizations could be an equalizer for people
with IDD to participate in data-driven conversations. Drawing on preliminary
research findings and our experiences working with people with IDD and their
data, we introduce and expand on the concept of cognitively accessible
visualizations, unpack its meaning and roles in increasing IDD individuals’
access to data, and discuss two immediate research objectives. Specifically,
we argue that cognitively accessible visualizations should support people with
IDD in personal data storytelling for effective self-advocacy and self-
expression, and balance novelty and familiarity in data design to accommodate
cognitive diversity and promote inclusivity.

Human-centered computingVisualization; Human-centered computingAccessibility

Two immediate objectives of cognitively accessible visualizations are to
design for personal storytelling and to balance novelty and familiarity in
data design. Particularly, the visualizations need to facilitate empathy in
the viewer and support therapeutic expressions for people with IDD. In
addition, designers may leverage multi-sensory approaches to representing
data, and create culturally relevant interfaces to be more in tune with the
needs of this community.

##  1 Introduction

Data plays an important role in understanding, addressing, and advocating for
the needs and rights of people with Intellectual and Developmental
Disabilities (IDD) [26]. Researchers, advocates, and organizations can use
data to create interventions to improve lives of people with IDD [17, 1, 8].
Collecting and analyzing data about people with IDD provides insights into
challenges faced by the community [17], helps identify disparities and
inequalities in resource allocation [1], and guides evidence-based policy-
making and drives positive social change [8]. Data can also empower
individuals with IDD. It allows the monitoring of personal health, behaviours
and progress, assists with autonomy and enables people with IDD to make
informed decisions and develop effective advocacy strategies [26]. However,
working with data is complicated and requires various cognitive skills to
effectively analyze, interpret, and draw insights [25]. People with IDD face
significant limitations in many cognitive areas; such as learning, reasoning,
abstract thinking, attention, concentration, memory and recall [2], which
prevent them from effectively using, understanding, and making decisions with
data, often excluding them from data analytics and leaving them vulnerable to
many ethical issues. Still, little progress has been made to improve data
accessibility for people with IDD: society tends to focus on their
disabilities rather than abilities [24], underestimate their potential and
trivialize their needs for data analytics. In addition, each individual with
IDD can have a significantly different cognitive profile, which makes
developing universal solutions particularly challenging.

Since 2018, we have been working with this population to design cognitively
accessible visualizations. Our investigation started from a collaboration with
an initiative that aims to support people with IDD in data-driven financial
self-advocacy [8]. In past work, we conducted a graphical perception
experiment to understand how various visualizations may impact data
accessibility in the context of budgetary data analysis [25]. Overall, the
study underlined the need to design visualizations that attend to the specific
abilities and preferences of people with IDD and demonstrated that cognitively
accessible visualizations may require a different set of guidelines. We then
explored more broadly the lived data experiences of people with IDD through
semi-structured interviews [26]. We found that people with IDD frequently used
personal data for self-advocacy, self-expression and everyday functioning.
However, many of these data encounters were invisible and inaccessible to
people with IDD, and were not well supported by current data visualizations,
leaving them with limited access to data important to their well-being.
Collectively, these studies suggest that visualizations have the potential to
empower people with IDD and that visualization research needs to pay more
attention to the specific needs of this population to develop truly accessible
solutions.

In this article, we build on these findings and our own experiences to
identify a critical near-term research agenda for increasing cognitive access
to data. We argue that cognitively accessible visualizations could serve as an
equalizer for people with IDD to participate in society. Specifically, we
discuss two immediate research priorities: supporting people with IDD in
personal data storytelling for effective self-advocacy and self-expression,
and balancing novelty and familiarity in data design to accommodate cognitive
diversity and promote inclusivity.

##  2 Supporting Personal Storytelling with Data

People with IDD use data in a variety of contexts, such as advocating for
one’s needs and rights, sharing personal experiences, or expressing thoughts
and feelings through data as a creative outlet. However, our preliminary
findings suggest that people assemble and visualize data sets to tell personal
stories in a manual and ad hoc manner. With limited visualization literacy,
most participants found traditional storytelling approaches inaccessible and
often intimidating to consume and author, leaving them unable to leverage data
to their full benefit in self-advocacy. Therefore, visualization research
needs to understand how to support effective personal data storytelling for
people with IDD (Fig. 1). Visualizations need to represent personal data of
people with IDD to help build empathy through shared lived experience and turn
this empathy into actions. Visualization tools should empower people with IDD
to better communicate personal stories for creative expression, self-
reflection, and compassion.

###  2.1 Designing for Empathetic Persuasion

Visualization research should aim to develop best practices for designing data
stories that build empathy in the viewers to help them better understand the
lived experience of people with IDD. Many individuals with IDD experience
stigma, prejudice and discrimination: they are largely viewed as liabilities
to be managed rather than assets that contribute unique perspectives to their
community[24]. These individuals usually live in systems that were designed
without their unique needs in mind and, as a result, this population often
faces barriers to engagement and participation, ranging from the health-care
system to the criminal justice system to employment and education [22]. Our
research showed that data could be an empowering tool for self-advocacy and
help shift these negative narratives for better social inclusion.
Specifically, we found that people with IDD were trying to make compelling
arguments through collecting and sharing data from their personal lives. These
could be written logs of daily activities and accomplishments, multimedia
files that demonstrate competence and capability, or even survey results that
illustrate the life expectancy gap between disabled and non-disabled
populations.

Figure 1: Cognitively accessible visualizations should support people with IDD
in telling personal stories through data to elicit empathy in the viewer for
more effective self-advocacy and facilitate therapeutic expression and self-
compassion by helping them make sense of personal data and life experiences.

To translate this data into powerful arguments, cognitively accessible
visualizations should more vividly represent the stories of people with IDD;
they could help communicate their life aspects, challenges, thoughts and
feelings to those who may not share the experiences; and evoke empathy in the
viewers. To design such compelling data visualizations, or to more effectively
move a viewer to action, we recommend designers use storytelling techniques,
but center the design of these stories around the particular person described
by the data and/or the argument to make rather than the data per se. This
means that designers may need to co-design with people with IDD to build
better empathy and an understanding of their lives, and then base the design
on IDD individuals’ point of view. Visualizations may help a viewer better
relate to the story by using true-to-life imagery and having a character that
carries the IDD individual’s values and beliefs. To better elicit empathetic
responses, future authoring tools may learn from principles of the arts to
create an aesthetic experience with data. For example, different colors can be
mapped to emotions and feelings; visual emphasis and contrast can be made to
highlight certain piece of data or make a statement; music, sound effects and
narrations can be layered into a multimedia data visualization to establish a
setting, develop characters, or advance the plot. These aesthetic experiences
may allow a viewer without IDD to feel the emotions and perspectives of people
with IDD, being able to peek into their inner worlds and daily lives, getting
a sense of “us” rather than “them.” Such stories may help people develop a
strong sense of engagement [21] and deeper connection with characters depicted
in the story (i.e., people with IDD), and activate empathy, which then
translates to action.

###  2.2 Designing for Therapeutic Expression

Visualization can serve as a therapeutic tool for people with IDD to regulate
emotions and develop self-compassion by encouraging creative expression and
meaning-making with personal data. People with IDD are three to four times
more likely to develop psychiatric disorders compared with the general
population [14]. Approximately one third of adults with ASD have emotion
dysregulation and challenging behaviours (CBs) [9], negatively affecting their
quality of life and social participation. We found that data was commonly used
by people with IDD, especially those with ASD, as a way of self-expression and
reflection, a proxy to communicate thoughts and feelings in psychiatric
treatments, and occasionally an educational device to improve self-regulation
[26]. For example, participants kept written notes of their symptoms and
reactions to social situations, they created a values checklist to reflect
everyday behaviors, curated video clips that depict human interactions to
discuss with their therapists, and created drawings, photos, and multimedia
files to express how they feel about themselves and the world around them.

People with IDD, particularly those with ASD, usually experience anxiety as a
result of sensory processing issues [4]. Our studies indicate that the anxiety
may also stem from the social awareness that they are different: they may
experience feelings of loneliness and isolation as a result of not conforming
to social norms or an inability to fit in with their peers. For example, one
participant described how they personally relate to the movie Beauty and the
Beast, _“My understanding is that the beast is autistic to a certain extent.
He has a lot of books. He loves knowledge. That’s essentially me. And he’s
nice, but he has that inner world that’s not being understood. And Belle is
just this oddball. She just seems to be ahead of her time that she understands
the beast. She’s like the therapist that he needs and she ends up falling in
love with the beast.”_ Similarly, another participant from our workshop on the
theme of “Aliens from the VisuaLand” expressed how they “truly feel like an
alien to the human world.” The sense of alienation can magnify feelings of
self-doubt, resulting in social anxiety, low self-esteem, and depression,
ultimately leading people with ASD to shut down. Due to the many shared
behavioral, conceptual, communicative, and social difficulties, individuals
with IDD in general experience high risks of co-morbid mental health
conditions such as affective, anxiety, psychotic, and impulse control
disorders [27]. However, research indicates a common lack of clinical
knowledge and training about the needs of people with IDD among mental health
professionals, resulting in misdiagnosis and mistreatment that are associated
with an exacerbation of dysfunction and disability [27]. Non-pharmacological
therapeutic support in IDD—such as social prescribing, behavioral and
educational interventions, or psychotherapy—seem to be more effective than
other interventions [13]. While beyond traditional visual analytics, data
visualizations do not have to be purely analytical. We argue that cognitively
accessible visualizations could be a therapeutic device, providing a creative
outlet for individuals with IDD to express themselves, communicate ideas, and
make sense of complex life experiences for positive meaning-making. This can
happen in several ways:

(1) Supporting self-expression and storytelling. Data visualization can be a
creative way to tell personal stories or narratives. By combining data and
personal experiences, individuals with IDD can create visual representations
that highlight their unique perspectives or journeys. By utilizing design
choices such as colors, shapes, and visual metaphors, individuals can infuse
their data visualizations with emotion and personal expression. This can help
overcome communication challenges and enable people with IDD to create
visualizations that evoke specific feelings, connect with the audience on a
deeper level, and convey the subjective aspects of the data being represented.
Visualization research may further investigate the connection between
different design elements and emotions to provide templates for people with
IDD to readily communicate feelings and generate stories from the data.

(2) Encouraging self-reflection and compassion. Engaging in data visualization
can prompt people with IDD to reflect on their own data, habits and
behaviours. People with IDD can gain insights into their patterns, progress,
or areas for improvement. Visualization can serve as a means to explore
personal interests or passions. Whether it’s visualizing personal fitness
data, travel experiences, or reading habits, individuals can create visual
representations of their hobbies, interests, or personal pursuits. This allows
them to share their enthusiasm, express their unique perspectives, and
potentially inspire others who resonate with their visualizations.
Visualization research may explore different encoding strategies and
representation formats for expressing these personal experiences to help
people with IDD intelligently manage their data and arrive at useful insights.

##  3 Balancing Novelty and Familiarity in Data Design

Despite being everywhere, data is invisible to many people with IDD, who often
consider data as a remote subject and show apathy or even aversion to it. Most
of them are usually unconscious of everyday data experiences and disinterested
in improving data literacy [26]. Moreover, data is abstract and usually comes
in diverse formats in large volumes, which requires significant cognitive
resources and specialized training to work with. Many participants in our
studies complained about the complexity of data and their lack of abilities
and resources to make sense of it [26]. Evidence from our research shows that
when designed carefully, visualizations are able to help people with IDD
better reason with data [25], and that the awareness of and willingness to
work with data is largely driven by personal motivation [26]. However, we
found that most participants associate data with “hard drive” and
“technology”, treating it as something used by “a computer type of gal” or
“people who do studies” but not themselves. These stereotype-driven beliefs
about data can have detrimental effect on people’s ability and confidence in
taking control of data. To change this stereotypical view and encourage
interests in working with data, cognitively accessible visualization should
strike for a balance of novelty and familiarity in design. We anticipate that
this can best be achieved by creating novel multisensory experiences to
increase the awareness and appeal of “mundane data”, and also by having IDD
individuals’ shared knowledge and experiences in mind to design culturally
relevant interfaces that help them digest and relate to new information.

###  3.1 Creating Multisensory Data Experiences

Visualization should accommodate the diverse needs and abilities of people
with IDD by providing alternative representation options and novel data
experiences. On top of being highly susceptible to psychiatric disorders,
individuals with IDD are also far more likely to develop multiple physical
health conditions, such as epilepsy, visual impairment, hearing loss, mobility
issues, and sensory processing disorders [10, 18, 20]. Our data interview
highlighted a common yet unconscious use of data among this population,
especially those less independent participants: tracking personal routine data
to adjust daily regimen [26]. However, this data is usually invisible and
inaccessible to people with IDD, as it is considered as part of the daily
routine and is collected for other people (e.g., caregivers, psychologists,
dietitians, physicians etc.) rather than themselves, which significantly
reduces the agency and control of their own data and even bodies. We argue
that cognitively accessible visualizations should help IDD individuals improve
awareness of personal data, encouraging them to explore this data and
participate in decision-making about their own lives.

Figure 2: Cognitively accessible visualizations may take in multisensory forms
for better data awareness and accessibility.

Discussions with caregivers and therapists suggested that individuals with IDD
will likely benefit from a combination of visual, verbal, and tactile cues to
understand prompts and complete everyday tasks. Given the heterogeneity of
this population, we suggest visualization designers consider using
multisensory data representation strategies to help people with IDD reason
with data 2. For example, data can be transformed into physical sculptures or
installations to provide a tactile and visual experiences [7]. Individuals
with IDD can physically interact with the sculpture, manipulating different
elements to explore and reason with the data through a constructionist
approach [15, 12]. The sculpture could use various textures, shapes, or colors
to represent different data points or categories, allowing individuals to
physically rearrange and compare them [11]. Data can also be sonified, with
different data attributes being mapped to various auditory elements such as
pitch, rhythm, or volume. This allows individuals to listen to patterns,
trends, or relationships in the data, providing an alternative perspective for
reasoning and analysis [16]. Utilizing tactile interfaces, such as
touchscreens, haptic feedback devices, or braille displays, can enable
individuals to reason with personal data through touch. People can feel
patterns, textures, or vibrations associated with different data points,
enabling a tactile exploration and analysis of the data [16]. By turning data
exploration and analysis into interactive games or challenges, individuals can
utilize visual, auditory, and even kinesthetic senses to make decisions, solve
problems, and reason with personal data in an enjoyable and immersive manner.
In addition, creating multisensory data dashboards that combine visual
representations with auditory cues, interactive touchscreens, or haptic
feedback can provide a comprehensive data experience. This allows individuals
to reason with personal data using a combination of visual, auditory, and
tactile senses, enhancing the depth of understanding and analysis. Future
visualization research may better understand how different modalities can be
used to accommodate IDD individuals’ complex sensory needs and preferences,
and more thoroughly examine the impact of various modalities on cognitive
accessibility.

###  3.2 Creating Culturally Relevant Data Interfaces

Since IDD is usually a set of disabilities rather than single diagnosis, it
may impact each individual differently [2]. In our experiences working with
this population, almost everyone has a distinctive set of conditions and faces
unique challenges despite being characterized under the umbrella term “IDD,”
which makes developing a universally accessible solution extremely difficult.
Yet our graphical perception experiment (Fig. 3) demonstrated that
thoughtfully designed visualizations can serve as a cognitive amplifier and
help people with a range of disabilities better reason with data [25]. For
example, people with IDD tend to relate abstract visual elements to real-world
objects (e.g., treemaps as colored pieces of paper, stacked bars as rising
stairs etc.). They benefit from familiar visual metaphors, such as dollar
signs and other semantically meaningful imagery, to understand the context of
data. Additionally, axis-aligned isotype visualizations and discrete,
countable representations can also largely improve their accuracy and
confidence in data interpretation. While pie charts are extremely
inaccessible, stacked bars and treemaps can make proportion data twice as
accessible. However, our experiment only covered a niche application of data
visualization (fiscal analysis). Most conventional tools and guidelines were
developed for neurotypical consumers without considering the differences of
people with IDD, introducing unintentional yet ongoing challenges for this
population. Cognitively accessible visualizations will require a different set
of guidelines and likely take on different shapes [25]. They should be
designed with respect to the unique needs, abilities, and preferences of
people with IDD, tapping into their shared cultural knowledge and experiences.
Designing such a culturally relevant interface may involve several aspects:

Figure 3: Guidelines for designing cognitively accessible visualizations
drawing from the graphical perception experiment in support of financial self-
advocacy for people with IDD.

(1) Visual Design. Incorporating culturally appropriate visuals, colors,
symbols and imagery that are familiar and meaningful to people with IDD in the
design. Our studies suggest that individuals with IDD generally prefer true-
to-life representations over abstract visualization when reasoning with data.
When designing for a particular scenario, designers may first need to
understand what visuals make sense to people with IDD, and then consider
maximizing the details of the representation, using photos, icons and other
relevant pictorials to communicate data context. Future research should
examine the role of semiotics in designing accessible visualizations or even
explore what “useful” chart junk [6] would look like for people with IDD,
identifying meaningful signs and symbols to include in visualization design to
cater to their needs.

(2) Navigation & Layout. Organizing the navigation and layout of a
visualization to match IDD individuals’ expectations and mental models. People
with IDD usually have attention and memory issues [2], whereas those with ASD
specifically are highly susceptible to cognitive overload and can easily get
overwhelmed by unexpected changes and transitions [4]. However, both groups
mentioned the potential benefit of guided sensemaking, such as “step-by-step”
imagery, to help direct their attention [26]. Future research should explore
the impact of turning abstract visualizations into more concrete formats, such
as data comics [5], data videos[3], or games [23], as they may enable self-
paced navigation and help people with IDD break down multidimensional data
into digestible pieces.

(3) Content, Language & Representation. Adapting the texts and communication
styles of the visualization to reflect the preferences of people with IDD. To
ensure the content is inclusive and make people feel represented and
respected, designers need to be careful with their data selection and use
unbiased data sources, paying close attention to the visual encoding and
design choices to not skew or distort the data. They need to provide
contextual information and consider alternative perspectives, encouraging
critical thinking and interpretation of the data. The visualization needs to
be an objective, accurate, and inclusive representation of the data, and the
people it describes. Future research may explore how affective design elements
[19] can meaningfully reflect the cultural identities of this population and
create emotionally engaging visualizations.

##  4 The Need for Cognitively Diverse Visualizations

People with Intellectual and Developmental Disabilities (IDD) encompass a
diverse population with unique characteristics and abilities, which often
translate to more complex and intense challenges compared with other
disabilities. The solution for developing cognitively accessible visualization
is not a straightforward one, as it needs to cater to diverse cognitive needs
and abilities, accounting for different information processing, comprehension
and interaction preferences and challenges, and will require flexible and
adaptable approaches to working with this population and presenting
information. Unlike physical disabilities that can often be addressed through
standardized accessibility guidelines, cognitive disabilities are highly
individualized and context-dependent. There is no one-size-fits-all approach
to cognitive accessibility, making it challenging to develop universal
guidelines or standards.

Yet, our preliminary investigations show that visualization could be a
promising tool to improve cognitive access to data and provide concrete
support for people with IDD in various areas of life. However, designing
cognitively accessible visualizations will require a different set of
guidelines and concerted efforts dedicated to addressing their particular
challenges. We argue that designing for cognitive accessibility is to design
for cognitive diversity, and it is a matter of equity and respect. By
providing equitable access to information, individuals can gain knowledge,
express their opinions, and contribute to decision-making processes. And by
designing visualizations that cater to diverse cognitive needs, we can foster
a more inclusive and equitable society where data is accessible to all.

##  5 Conclusion

Data can empower people with IDD in many ways. However, working with data is
difficult and often out of the reach of this population. In this paper, we
argue for a near-term research agenda for cognitively accessible
visualizations to serve as an equalizer for people with IDD to better
participate in society. Based on preliminary findings and our own reflections
working with this population and their data, we discuss two immediate research
goals for cognitively accessible visualizations to improve data accessibility.
First, visualizations should support people with IDD with better personal data
storytelling for more effective self-advocacy and self-expression. Second,
visualizations should balance novelty and familiarity in design to increase
IDD individuals’ awareness of and agency over personal data and ultimately
make this population more visible in the inclusive data visualization space.

###### Acknowledgements.

We would like to thank our colleagues, collaborators, and participants for
their support. This work was funded by NSF #2046725, #2320920, #2233316, and
#1933915, as well as the Ray Hauser Award at University of Colorado Boulder.

## References

  * [1] Dataset on Intellectual and Developmental Disabilities: Linking Data to Enhance Person-Centered Outcomes Research.
  * [2] AAIDD.  FAQs on Intellectual Disability — aaidd.org.  https://www.aaidd.org/intellectual-disability/faqs-on-intellectual-disability, 2022\.
  * [3] F. Amini, N. Riche, B. Lee, C. Hurter, and P. Irani.  Understanding data videos: Looking at narrative visualization through the cinematography lens.  04 2015.
  * [4] J. Ashburner, L. Bennett, S. Rodger, and J. Ziviani.  Understanding the sensory experiences of young people with autism spectrum disorder: A preliminary investigation.  Australian occupational therapy journal, 60:171–80, 06 2013. doi: 10 . 1111/1440-1630 . 12025
  * [5] B. Bach, N. Henry Riche, S. Carpendale, and H. Pfister.  The emerging genre of data comics.  IEEE Computer Graphics and Applications, 38:6–13, 05 2017. doi: 10 . 1109/MCG . 2017 . 33
  * [6] S. Bateman, R. Mandryk, C. Gutwin, A. Genest, D. Mcdine, and C. Brooks.  Useful junk? the effects of visual embellishment on comprehension and memorability of charts.  vol. 4, pp. 2573–2582, 04 2010. doi: 10 . 1145/1753326 . 1753716
  * [7] R. Bhargava and C. D’Ignazio.  Data sculptures as a playful and low-tech introduction to working with data.  In Designing Interactive Systems Conference, DIS ’17. Association for Computing Machinery, 2017. doi: 1721 . 1/123453
  * [8] D. L. Braddock, R. Hemp, E. S. Tanis, J. Wu, L. Haffer, A. A. O. Intellectual, and D. Disabilities.  The state of the states in intellectual and developmental disabilities, 2017.  American Association On Intellectual And Developmental Disabilities, 2017\.
  * [9] J. Brown, M. Brown, and P. Dibiasio.  Treating individuals with intellectual disabilities and challenging behaviors with adapted dialectical behavior therapy.  Journal of mental health research in intellectual disabilities, 6:280–303, 10 2013. doi: 10 . 1080/19315864 . 2012 . 700684
  * [10] S.-A. Cooper, G. McLean, B. Guthrie, A. McConnachie, S. Mercer, F. Sullivan, and J. Morrison.  Multiple physical and mental health comorbidity in adults with intellectual disabilities: Population-based cross-sectional analysis.  BMC family practice, 16:110, 08 2015. doi: 10 . 1186/s12875-015-0329-3
  * [11] T. K. K. H. Dumičić, Ž. and G. Joost.  Design elements in data physicalization: A systematic literature review.  jun 2022. doi: 10 . 21606/drs . 2022 . 660
  * [12] C. D’Ignazio and R. Bhargava.  Creative data literacy: A constructionist approach to teaching information visualization.  Digit. Humanit. Q., 12, 2018.
  * [13] P. Hamers, D. Festen, and H. Hermans.  Non-pharmacological interventions for adults with intellectual disabilities and depression: a systematic review: Non-pharmacological interventions for depression.  Journal of Intellectual Disability Research, 62, 05 2018. doi: 10 . 1111/jir . 12502
  * [14] J. Harris.  Intellectual Disability: Understanding Its Development, Causes, Classification, Evaluation, and TreatmentUnderstanding Its Development, Causes, Classification, Evaluation, and Treatment.  11 2005. doi: 10 . 1093/oso/9780195178852 . 001 . 0001
  * [15] S. Huron, S. Carpendale, A. Thudt, A. Tang, and M. Mauerer.  Constructive visualization.  Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, DIS, 06 2014. doi: 10 . 1145/2598510 . 2598566
  * [16] N. W. Kim, S. C. Joyner, A. Riegelhuth, and Y.-S. Kim.  Accessible Visualization: Design Space, Opportunities, and Challenges.  Computer Graphics Forum, 2021. doi: 10 . 1111/cgf . 14298
  * [17] G. Krahn.  A call for better data on prevalence and health surveillance of people with intellectual and developmental disabilities.  Intellectual and developmental disabilities, 57:357–375, 10 2019\. doi: 10 . 1352/1934-9556-57 . 5 . 357
  * [18] H. Lantman and P. Walsh.  Managing health problems in people with intellectual disabilities.  BMJ (Clinical research ed.), 337:a2507, 02 2008. doi: 10 . 1136/bmj . a2507
  * [19] E. Lee-Robbins and E. Adar.  Affective learning objectives for communicative visualizations.  IEEE Transactions on Visualization and Computer Graphics, 29(01):1–11, jan 2023. doi: 10 . 1109/TVCG . 2022 . 3209500
  * [20] P. Liao, C. Vajdic, J. Trollor, and S. Reppermund.  Prevalence and incidence of physical health conditions in people with intellectual disability – a systematic review.  PLOS ONE, 16(8):1–19, 08 2021. doi: 10 . 1371/journal . pone . 0256294
  * [21] P. Ognjenović and K. Andre.  Psihološka teorija umetnosti.  Institut za psihologiju, 1997.
  * [22] S. Pappas.  Despite the ADA, equity is still out of reach, 2020.
  * [23] A. Rapp.  Gamification for self-tracking: From world of warcraft to the design of personal informatics systems.  In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, p. 1–15. Association for Computing Machinery, New York, NY, USA, 2018. doi: 10 . 1145/3173574 . 3173654
  * [24] J. J. van Vuuren and H. M. Aldersey.  Stigma, acceptance and belonging for people with idd across cultures.  Current Developmental Disorders Reports, 7:163 – 172, 2020.
  * [25] K. Wu, E. Petersen, T. Ahmad, D. Burlinson, S. Tanis, and D. A. Szafir.  Understanding data accessibility for people with intellectual and developmental disabilities.  In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21. Association for Computing Machinery, New York, NY, USA, 2021. doi: 10 . 1145/3411764 . 3445743
  * [26] K. Wu, M. H. Tran, E. Petersen, V. Koushik, and D. A. Szafir.  Data, data, everywhere: Uncovering everyday data experiences for people with intellectual and developmental disabilities.  In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23. Association for Computing Machinery, New York, NY, USA, 2023. doi: 10 . 1145/3544548 . 3581204
  * [27] Y. Zisman-Ilani.  The mental health crisis of individuals with intellectual and developmental disabilities.  Psychiatric Services, 73:245–246, 03 2022. doi: 10 . 1176/appi . ps . 202200022


# Real-time Forecasting of Time Series in Financial Markets Using Sequentially
Trained Many-to-one LSTMs

Kelum Gajamannage<EMAIL_ADDRESS>Yonggi Park
<EMAIL_ADDRESS>Department of Mathematics and Statistics, Texas A&M
University - Corpus Christ, 6300 Ocean Dr., Corpus Christi, TX 78412, USA

###### Abstract

Financial markets are highly complex and volatile; thus, learning about such
markets for the sake of making predictions is vital to make early alerts about
crashes and subsequent recoveries. People have been using learning tools from
diverse fields such as financial mathematics and machine learning in the
attempt of making trustworthy predictions on such markets. However, the
accuracy of such techniques had not been adequate until artificial neural
network (ANN) frameworks were developed. Moreover, making accurate real-time
predictions of financial time series is highly subjective to the ANN
architecture in use and the procedure of training it. Long short-term memory
(LSTM) is a member of the recurrent neural network family which has been
widely utilized for time series predictions. Especially, we train two LSTMs
with a known length, say $T$ time steps, of previous data and predict only one
time step ahead. At each iteration, while one LSTM is employed to find the
best number of epochs, the second LSTM is trained only for the best number of
epochs to make predictions. We treat the current prediction as in the training
set for the next prediction and train the same LSTM. While classic ways of
training result in more error when the predictions are made further away in
the test period, our approach is capable of maintaining a superior accuracy as
training increases when it proceeds through the testing period. The
forecasting accuracy of our approach is validated using three time series from
each of the three diverse financial markets: stock, cryptocurrency, and
commodity. The results are compared with those of an extended Kalman filter,
an autoregressive model, and an autoregressive integrated moving average
model.

###### keywords:

Many-to-one LSTM , sequential training , real-time forecasting , time series ,
financial markets

##  1 Introduction

Financial markets refer broadly to any marketplace that entitles the trading
of securities, commodities, and other fungibles, and the financial security
market includes stock market, cryptocurrency market, etc. Bahadur et al.
[2019]. Among the three markets, which are stock, cryptocurrency, and
commodity, the stock markets are well known to people while the other two are
not. A cryptocurrency market exchanges digital or virtual currencies between
peers without the need for a third party such as a bank [Squarepants, 2022],
but a commodity market trades raw materials such as gold and oil rather than
manufactured products. These markets are both highly complex and volatile due
to diverse economical, social, and political conditions [Qiu et al., 2020].
Learning such markets for the sake of making predictions is vital because that
aids market analysts to make early alerts about crashes and subsequent
recoveries so that the investors can either obtain better precautions for
future crashes or gain more profit under future recoveries. Since it is
unreliable and inefficient to rely only on a trader’s personal experience and
intuition for the analysis and judgment of such markets, traders need smart
trading recommendations derived from scientific research methods.

The classical methods of making predictions on time series data are mostly
linear statistical approaches such as linear parametric autoregressive (AR),
moving average (MA), and autoregressive integrated moving average (ARIMA)
[Zhao et al., 2018] where they assume linear relationships between the current
output and previous outputs. Thus, they often do not capture non-linear
relationships in the data and cannot cope with certain complex time series.
Because financial time series are nonstationary, nonlinear, and contaminated
with high noise [Bontempi et al., 2013], traditional statistical models have
some limitations in predicting financial time series with high precision.
Purely data-driven approaches such as Artificial Neural Networks (ANNs) are
adopted to forecast nonlinear and nonstationary time series data with both
high efficiency and better accuracy and have become a popular predictor due to
adaptive self-learning [Gajamannage et al., 2021].

Recurrent Neural Networks (RNNs) are powerful and robust types of ANNs that
belong to the most promising algorithms in use because of their internal
memory [Park et al., 2022]. This internal memory remembers its inputs and
helps RNN to find solutions for a vast variety of problems [Ma & Principe,
2018]. RNN is optimized with respect to its weights to fit the training data
by adopting a technique called backpropagation that requires the gradient of
the RNN. However, the gradient of RNN may vanish and explode during the
optimization routing which hampers RNN’s learning ability of long data
sequences [Allen-Zhu et al., 2019]. As a solution to these two problems [Le &
Zuidema, 2016], the LSTM architecture [Hochreiter & Schmidhuber, 1997], which
is a special type of RNN, is often used. LSTMs are explicitly designed to
learn long-term dependencies of time-dependent data by remembering information
for long periods. LSTM performs faithful learning in applications such as
speech recognition [Tian et al., 2017, Kim et al., 2017] and text processing
[Shih et al., 2018, Simistira et al., 2015]. Moreover, LSTM is also suitable
for complex data sequences such as stock time series extracted from financial
markets because it has internal memory, has capability of customization, and
is free from gradient-related issues.

We adopt a real-time iterative approach to train an LSTM that makes only one
prediction for each iteration. For that, we train this LSTM with a known
length, sat $T$ time steps, of previous data while setting the loss function
to be the mean square error between labels and predictions. The LSTM predicts
only one time step ahead during the current iteration that we treat as an
observation for the next training dataset. We train the same LSTM over all the
iterations where the number of iterations is equal to the number of total
predictions. This real-time LSTM model is capable of incorporating every new
future observation of the time series into the ongoing training process to
make predictions. Since we use a sequence of observed time series to predict
only one time step ahead, the prediction accuracy increases significantly.
Moreover, the $T-1$ previous observations along with the current prediction
predict for the next time step, and so the prediction error associated with
the current prediction is further minimized as it runs through iterations.
While classic ways of training result in more error when the predictions are
made further away in the test period, our approach is capable of maintaining a
superior accuracy as training increases when it proceeds through the testing
period.

This paper is structured with four sections, namely, introduction (Sec.1),
methods (Sec. 2), performance analysis (Sec. 3), and discussion (Sec. 4). In
Sec.1, first, we present the notion of real-time time series predictions.
Then, we provide mathematical formulation of many-to-one LSTM architecture for
sequential training. Finally, for the state-of-the-art time series prediction
methods, we provide the formulation of one nonlinear statistical approach
called extended Kalman filters (EKF), and two linear statistical approaches
called AR and ARIMA. Sec. 3 provides a detailed analysis of the performance of
our LSTM architecture against that of EKF, AR, and ARIMA using three financial
stocks (Apple, Microsoft, Google), three cryptocurrencies (Bitcoin, Ethereum,
Cardano), and three commodities (gold, crude oil, natural gas). We present the
conclusions along with a discussion in Sec. 4.

Notation |  Description
---|---
$t$ |  Index for time steps
$T$ |  Length of the training period
$N$ |  Forecasting length
$L$ |  Number of epochs
$K$ |  Number of stacked LSTMs
$\mathcal{L}^{(t)}_{l}$ |  Training loss at the $l$-th epoch of the $t$-th iteration
$\sigma$ |  Sigmoid function in LSTM
$p$ |  Order of the AR model
$q$ |  Number of past innovations in MA model
$\mathcal{E}$ |  Relative root mean square error
$\boldsymbol{x}^{(t)}$ |  The observation at the $t$-th time step where $1\leq t\leq T$
$\boldsymbol{X}^{(t)}=\left[\boldsymbol{x}^{(t)},...,\boldsymbol{x}^{(T+t-2)}\right]$ |  $t$-th input training window
$\hat{\boldsymbol{x}}^{(t)}$ |  The prediction at the $t$-th time step where $T<t\leq T+N$
$\boldsymbol{w}^{(t)}$ |  White Gaussian noise vector with zero mean in EKF
$\boldsymbol{y}^{(t)}$ |  Observation vector at the $t$-th time step in EKF
$\boldsymbol{v}^{(t)}$ |  White Gaussian noise vector with zero mean in EKF
$\alpha_{i},1\leq i\leq p$ |  Parameters of AR model
$\boldsymbol{\epsilon}^{(t)}$ |  White Gaussian noise vector with zero mean in AR model
$\beta_{i},1\leq i\leq q$ |  Parameters of MA model
$\boldsymbol{a}^{(t)}$ |  $t$-th past innovation of MA model
$\boldsymbol{c}$ |  Biased vector in ARIMA
$\boldsymbol{b}_{i},\boldsymbol{b}_{f},\boldsymbol{b}_{c}$ |  Bias vectors in LSTM
$f$ |  System dynamics in EKF
$h$ |  Measurement function in EKF
$L^{i}$ |  $i$-th level lag operator
$\Delta^{D}(\cdot)$ |  $D$-th differential time series
$Q^{(t)}$, $R^{(t)}$, $P^{(t)}$ |  Covariance matrices of $\boldsymbol{w}^{(t)}$, $\boldsymbol{v}^{(t)}$, $\boldsymbol{x}^{(t)}$, respectively, in EKF
$J_{f}$, $J_{h}$ |  Jacobian matrices of $f(\cdot)$, $h(\cdot)$, respectively, in EKF
$W_{i},W_{f},W_{o}$ |  Weight matrices in LSTM
Table 1: Notations used in this paper and their descriptions Abbreviations |  Description
---|---
LSTM |  Long Short-Term Memory
KF |  Kalman Filter
EKF |  Extended Kalman Filter
AR |  AutoRegressive
MA |  Moving Average
ARMA |  AutoRegressive Moving Average
ARIMA |  AutoRegressive Integrated Moving Average
Table 2: Abbreviations used in this paper and their descriptions

##  2 Methods

In this section, first, we provide technical details of the real-time time
series prediction scheme. Then, we present the LSTM architecture to cater to
the real-time time series prediction, and LSTM’s training and predicting
procedures. Moreover, we apply this real-time prediction scheme for three
other time series prediction methods, namely, EKF, AR, and ARIMA. These three
methods will be our state-of-the-art methods to compare the performance of
LSTM.

###  2.1 Real-time time series prediction

We adopt a “sequential” approach to efficiently train time series models and
predict for future. For a fixed-length input sequential data, the model is set
to predict only one future time step at an iteration where the process runs
until the required length of the prediction is performed. This real-time
prediction approach is capable of incorporating every new data point of the
time series into the ongoing training process to make predictions for the next
time step. Let, the current observed time series is
$[\boldsymbol{x}^{(1)},\dots,\boldsymbol{x}^{(T)}]$ for some $T$, the
unobserved future portion of the time series is
$\left[\boldsymbol{x}^{(T+1)},\dots,\boldsymbol{x}^{(T+N)}\right]$ for some
$N<T$, and the time series model is $\mathcal{F}$, see Fig. 1. For the first
iteration, we train the time series forecasting model with the
$x^{(T)}=\mathcal{F}\left(\boldsymbol{X}^{(1)}\right)$ where
$\boldsymbol{X}^{(1)}=\left[\boldsymbol{x}^{(1)},\dots,\boldsymbol{x}^{(T-1)}\right]$.
Then, we predict for the time step $(T+1)$, denoted by
$\hat{\boldsymbol{x}}^{(T+1)}$, as
$\hat{\boldsymbol{x}}^{(T+1)}=\mathcal{F}\left(\boldsymbol{X}^{(2)}\right)$
where
$\boldsymbol{X}^{(2)}=\left[\boldsymbol{x}^{(2)},\dots,\boldsymbol{x}^{(T)}\right]$.
In the second iteration, we train the same model $\mathcal{F}$ with
$\hat{\boldsymbol{x}}^{(T+1)}=\mathcal{F}\left(\boldsymbol{X}^{(2)}\right)$
where
$\boldsymbol{X}^{(2)}=\left[\boldsymbol{x}^{(2)},\dots,\boldsymbol{x}^{(T)}\right]$
and predict for the time step $(T+2)$, denoted by
$\hat{\boldsymbol{x}}^{(T+2)}$, as
$\hat{\boldsymbol{x}}^{(T+2)}=\mathcal{F}\left(\boldsymbol{X}^{(3)}\right)$
where
$\boldsymbol{X}^{(3)}=\left[\boldsymbol{x}^{(3)},\dots,\hat{\boldsymbol{x}}^{(T+1)}\right]$.
We keep on doing this process until the predictions are performed for all the
$N$ time steps .

Figure 1: Real-time time series prediction scheme where the currently observed
time series, $\left[\boldsymbol{x}^{(1)},\dots,\boldsymbol{x}^{(T)}\right]$
for some $T$, is shown by blue color. The unobserved future time series,
$\left[\boldsymbol{x}^{(T+1)},\dots,\boldsymbol{x}^{(T+N)}\right]$ for some
$N<T$, is shown by white color while the prediction for the future is shown in
red color. For the first iteration, we train the time series forecasting model
with the $\boldsymbol{x}^{(T)}=\mathcal{F}\left(\boldsymbol{X}^{(1)}\right)$
where
$\boldsymbol{X}^{(1)}=\left[\boldsymbol{x}^{(1)},\dots,\boldsymbol{x}^{(T-1)}\right]$.
Then, we predict for the time step $(T+1)$ as
$\hat{\boldsymbol{x}}^{(T+1)}=\mathcal{F}\left(\boldsymbol{X}^{(2)}\right)$
where
$\boldsymbol{X}^{(2)}=\left[\boldsymbol{x}^{(2)},\dots,\boldsymbol{x}^{(T)}\right]$.
In the second iteration, we train the same model with
$\hat{\boldsymbol{x}}^{(T+1)}=\mathcal{F}\left(\boldsymbol{X}^{(2)}\right)$
where
$\boldsymbol{X}^{(2)}=\left[\boldsymbol{x}^{(2)},\dots,\boldsymbol{x}^{(T)}\right]$
and predict for the time step $(T+2)$ as
$\hat{\boldsymbol{x}}^{(T+2)}=\mathcal{F}\left(\boldsymbol{X}^{(3)}\right)$
where
$\boldsymbol{X}^{(3)}=\left[\boldsymbol{x}^{(3)},\dots,\hat{\boldsymbol{x}}^{(T+1)}\right]$.
We keep on doing this process until all the predictions are performed.

###  2.2 Many-to-one LSTM architecture with sequential training

Since we make predictions only for one time step ahead at a time for an input
time series, the LSTM architecture implemented here is the many-to-one type,
see Fig. 2(a) for $K$-stacked LSTM architecture. An LSTM consists of a series
of nonlinear recurrent modules, denoted as $M^{(t)}_{j}$ for $t=1,\dots,T$ and
$j=1,\dots,N$ in Fig. 2, where each module processes data related to one time
step. LSTM introduces a memory cell, a special type of the hidden state, that
has the same shape as the hidden state which is engineered to record
additional information. Each recurrent module in an LSTM filters information
through four hidden layers where three of them are gates, namely, forgotten
gate, input gate, and output gate, and the other is called the cell state that
maintains and updates long-term memory, see 2(b).

The forgotten gate resets the content of the memory cell by deciding what
information should be forgotten or retained. This gate produces a value
between zero and one where zero means completely forgetting the previous
hidden state and one means completely retaining that. Information from the
previous hidden state, i.e., $\boldsymbol{h}^{(t-1)}$, and the information
from the current input, i.e., $\boldsymbol{x}^{(t)}$, is passed through the
$sigmoid$ function, denoted as $\sigma$, according to

$\boldsymbol{f}^{(t)}=\sigma\left(W_{f}\cdot[\boldsymbol{h}^{(t-1)},\boldsymbol{x}^{(t)}]+\boldsymbol{b}_{f}\right),$
(1)

where $W_{f}$ and $\boldsymbol{b}_{f}$ are weighting matrix and biased vector,
respectively. The input gate consisting of two components decides what new
information is to be stored in the cell state. The first component is a
$sigmoid$ layer that decides which values to be updated based on the previous
hidden state and the information from the current input such that

$\boldsymbol{i}^{(t)}=\sigma\left(W_{i}\cdot[\boldsymbol{h}^{(t-1)},\boldsymbol{x}^{(t)}]+\boldsymbol{b}_{i}\right),$
(2)

where $W_{i}$ and $\boldsymbol{b}_{i}$ are weighting matrix and biased vector,
respectively. The next component is a $tanh$ layer that creates a vector of
new candidate values, $\tilde{c}^{(t)}$, based on the previous hidden state
and the information from the current input as

$\tilde{\mathbf{c}}^{(t)}=tanh\left(W_{c}\cdot[\boldsymbol{h}^{(t-1)},\boldsymbol{x}^{(t)}]+\boldsymbol{b}_{c}\right),$
(3)

where $W_{c}$ and $\boldsymbol{b}_{c}$ are weighting matrix and biased vector,
respectively.

Figure 2: $K$-stacked LSTMs for many-to-one forecasting of a single feature
time series where each LSTM is a collection of recurrent modules, denoted as
$M$’s. (a) the left figure shows a folded version of the artificial neural
network (ANN) whereas the right figure shows its unfolded version of it. Here,
the input for the ANN is
$\boldsymbol{X}=\left[\boldsymbol{x}^{(1)},\dots,\boldsymbol{x}^{(t)},\dots,\boldsymbol{x}^{(T-1)}\right]$
and output from that is $\hat{\boldsymbol{x}}^{(T)}$. (b) Each $M$ filters
information through four hidden layers where three of them are gates, namely,
forgotten, input, and output, and the other is called the cell state. The
forgotten gate resets the content of the memory cell, the input gate decides
what new information is stored in the memory cell, the cell state stores long-
term information in the memory, and the output gate sends out a filtered
version of the memory cell’s stored information from the $M$. Operations in an
$M$ are given as follows:
$\boldsymbol{f}^{(t)}=\sigma\left(W_{f}\cdot[\boldsymbol{h}^{(t-1)},\boldsymbol{x}^{(t)}]+\boldsymbol{b}_{f}\right)$,
$\boldsymbol{i}^{(t)}=\sigma\left(W_{i}\cdot[\boldsymbol{h}^{(t-1)},\boldsymbol{x}^{(t)}]+\boldsymbol{b}_{i}\right)$,
$\tilde{\mathbf{c}}^{(t)}=tanh\left(W_{C}\cdot[\boldsymbol{h}^{(t-1)},\boldsymbol{x}^{(t)}]+\boldsymbol{b}_{c}\right)$,
$\boldsymbol{c}^{(t)}=\boldsymbol{f}^{(t)}\odot\boldsymbol{c}^{(t-1)}\oplus\boldsymbol{i}^{(t)}\odot\tilde{\boldsymbol{c}}^{(t)}$,
$\boldsymbol{o}^{(t)}=\sigma\left(W_{o}\cdot[\boldsymbol{h}^{(t-1)},\boldsymbol{x}^{(t)}]+\boldsymbol{b}_{o}\right)$,
and $\boldsymbol{h}^{(t)}=\boldsymbol{o}^{(t)}\odot
tanh\left(\boldsymbol{c}^{(t)}\right)$, where $\odot$ and $\oplus$ are point-
wise multiplication and point-wise addition, respectively.

Cell state updates the LSTM’s memory with new long-term information. For that,
first, it multiplies point wisely the old cell state $c^{(t-1)}$ by the
forgetting state $\boldsymbol{f}^{(t)}$, i.e.,
$\boldsymbol{f}^{(t)}\odot\boldsymbol{c}^{(t-1)}$, to assure that the
information retains from the old cell state is what is allowed by the
forgetting state. Then, we add the pointwise product
$\boldsymbol{i}\odot\tilde{c}^{(t)}$ into
$\boldsymbol{f}^{(t)}\odot\boldsymbol{c}^{(t-1)}$, i.e.,

$\boldsymbol{c}^{(t)}=\boldsymbol{f}^{(t)}\odot\boldsymbol{c}^{(t-1)}\oplus\boldsymbol{i}^{(t)}\odot\tilde{\boldsymbol{c}}^{(t)},$
(4)

as the information from the current input state which is found relevant by the
ANN. The output gate determines the value of the next hidden state with the
information from the current cell state, current input state, and previous
hidden state. First, a $sigmoid$ layer decides how much of the current input
and the previous hidden state are going to output. Then, the current cell
state is passed through the $tanh$ layer to scale the cell state value between
-1 and 1. Thus, the output $\boldsymbol{h}^{(t)}$ is

$\boldsymbol{h}^{(t)}=\boldsymbol{o}^{(t)}\odot
tanh\left(\boldsymbol{c}^{(t)}\right),\ \text{with}\ \
\boldsymbol{o}^{(t)}=\sigma\left(W_{o}\cdot[\boldsymbol{h}^{(t-1)},\boldsymbol{x}^{(t)}]+\boldsymbol{b}_{o}\right),$
(5)

where $W_{o}$ and $\boldsymbol{b}_{o}$ are weighting matrix and biased vector,
respectively. Based upon $\boldsymbol{h}^{(t)}$, the network decides which
information from the current hidden state should be carried out to the next
hidden state where the next hidden state is used for prediction. To conclude,
the forget gate determines which relevant information from the prior steps is
needed. The input gate decides what relevant information can be added from the
current cell state, and the output gates finalize the input to the next hidden
state.

####  2.2.1 Optimization of LSTM

Training an LSTM is the process of minimizing a relevant reconstruction error
function, also called loss function, with respect to weights and bias vectors
of Eqns. (1), (2), (3), (4), and (5). Such a minimization problem is
implemented in four steps: first, forward propagation of input data through
the ANN to get the output; second, calculate the loss, between forecasted
output and the true output; third, calculate the derivatives of the loss
function with respect to the LSTM’s weights and bias vectors using
backpropagation through time (BTT) Werbos [1990]; and fourth, adjusting the
weights and bias vectors by gradient descent method Gruslys et al. [2016].

BTT unrolls backward all the dependencies of the output onto the weights of
the ANN Manneschi & Vasilaki [2020], which is represented from left side to
right side in Fig. 2(a). At each iteration, say $t\in[1,N+1]$, we train the
LSTM by only one instance of input-label where the input is
$\boldsymbol{X}^{(t)}=\left[\boldsymbol{x}^{(t)},\dots,\boldsymbol{x}^{(T)},\hat{\boldsymbol{x}}^{(T+1)},\dots,\hat{\boldsymbol{x}}^{(T+t-2)}\right]$
and the label is $\hat{\boldsymbol{x}}^{(T+t-1)}$. Due to this process, at the
$t$-th iteration, the ANN is trained with $t$-th input-label instance and
predicts for the ($T+t-1$)-th time step. Thus, we formulate the loss function
at the $t$-th iteration of the LSTM as the relative mean square error,

$\mathcal{L}^{(t)}=\frac{\left\|\hat{\boldsymbol{x}}^{(T+t-1)}-\boldsymbol{x}^{(T+t-1)}\right\|_{F}^{2}}{\left\|\boldsymbol{x}^{(T+t-1)}\right\|_{F}^{2}},$
(6)

where $F$ denotes the Frobenius norm and $\hat{\boldsymbol{x}}^{(T+t-1)}$ is
the output of the LSTM for the input $\boldsymbol{X}^{(t-1)}$. We use BTT to
compute the derivatives of Eqn. (6) with respect to the weights and bias
vectors. We update the weights using the gradient descent-based method, called
Adaptive Moment Estimation (ADAM) Kingma & Ba [2015]. ADAM is an iterative
optimization algorithm used in recent machine learning algorithms to minimize
loss functions where it employs the averages of both the first-moment
gradients and the second-moment of the gradients for computations. It
generally converges faster than standard gradient descent methods and saves
memory by not accumulating the intermediate weights.

To assure better convergence of the loss function, we integrate epochs into
the training process in a unique way that we explain here for the $t$-th
iteration. However, if the loss function is non-convex or semi-convergence
choosing the best number of epochs is challenging. Fig. 3 illustrates the non-
convex behavior of an LSTM’s loss function that is trained with the closing
prices of the Apple stock. Here, we input a sequence of 1227 days of prices
into the LSTM and generate the price for the 1228-th day where the loss is
computed as the relative mean square error between the predicted price and the
observed prices for the 1228-th day. We proceed with this single-day training
for 60 epochs as shown in Fig. 3. Since the loss varies non-convexly with
respect to epochs, we came up with a unique way of training the LSTM through
epochs. Particularly, we maintain two LSTMs, denoted as $LSTM_{1}$ and
$LSTM_{2}$, that are trained through each iteration. We assume that those two
LSTMs corresponding to the $(t-1)$-th iteration are given for the $t$-th
iteration. For the $t$-th iteration, we train $LSTM_{1}$ with the input
$\boldsymbol{X}^{(t)}$ and the label $\boldsymbol{x}^{T+t-1}$ for fix number
of epochs, say $L$. Here, we record $LSTM_{1}$’s optimum weights and biased
vectors corresponding to each of the epoch. We reformulate $LSTM_{2}$ with the
weights and biased vectors corresponding to the least loss among $L$ epochs.
Finally, we redefine $LSTM_{1}$ as $LSTM_{2}$ and proceed to the $(t+2)$-th
iteration. Algorithm 1 summarizes the training and prediction procedure of our
sequentially trained many-to-one LSTM scheme.

Figure 3: Non-convex behavior of the loss function. We apply our training
scheme to train an LSTM with closing prices of the Apple stock. We input a
sequence of 1227 days of prices into the LSTM and generate the price for the
1228-th day. The loss is computed as the relative mean square error between
the predicted price and the observed prices for the 1228-th day. We proceed
with this single-day training 60 times, also called epochs, where the loss for
the $l$-th epoch is denoted as $L^{(t)}_{l}$. Algorithm 1 : Many-to-one LSTM
architecture with sequential training.
Denotation:
$\boldsymbol{X}^{(1)}=\left[\boldsymbol{x}^{(1)},\dots,\boldsymbol{x}^{(T-1)}\right]$,
$\boldsymbol{X}^{(2)}=\left[\boldsymbol{x}^{(2)},\dots,\boldsymbol{x}^{(T)}\right]$,
and
$\boldsymbol{X}^{(t)}=\left[\boldsymbol{x}^{(t)},\dots,\boldsymbol{x}^{(T)},\hat{\boldsymbol{x}}^{(T+1)},\dots,\hat{\boldsymbol{x}}^{(T+t-2)}\right]$
for $3\leq t\leq N+1$.
Input: training time series
$\left(\left[\boldsymbol{x}^{(1)},\dots,\boldsymbol{x}^{(T)}\right]\right)$;
forecast length ($N$); number of maximum epochs ($L$).
Output: time series forecast
$\left(\left[\hat{\boldsymbol{x}}^{(T+1)},\dots,\hat{\boldsymbol{x}}^{(T+N)}\right]\right)$;
trained $LSTM$ ($LSTM_{1}$ or $LSTM_{2}$).

1: Initialization: two $LSTM$s, denoted as $LSTM_{1}$ and $LSTM_{2}$, with the
weights $W_{f}=W_{i}=W_{c}=W_{o}=0$ and biased vectors
$\boldsymbol{b}_{f}=\boldsymbol{b}_{i}=\boldsymbol{b}_{c}=\boldsymbol{b}_{o}=\boldsymbol{h}^{(0)}=\textbf{0}$.

2: for $t\in[1,N+1]$ do

3: for $l\in[1,L]$ do

4: Compute
$\hat{\boldsymbol{x}}^{(T+t-1)}=LSTM_{1}\left(\boldsymbol{X}^{(t)}\right)$
according to the map in Fig 2(a).

5: Minimize
$L^{(t)}_{l}=\left\|\hat{\boldsymbol{x}}^{(T+t-1)}-\boldsymbol{x}^{(T+t-1)}\right\|_{F}^{2}\Big{/}\left\|\boldsymbol{x}^{(T+t-1)}\right\|_{F}^{2}$
using BTT with respect to the weights $W_{i}$, $W_{i}$, $W_{c}$, and $W_{o}$,
and bias vectors $\boldsymbol{b}_{f}$, $\boldsymbol{b}_{i}$,
$\boldsymbol{b}_{c}$, and $\boldsymbol{b}_{o}$ of the composite representation
of the functions in Eqns. (1), (2), (3), (4), and (5).

6: Record $L^{(t)}_{l}$ along with $W_{i}$, $W_{i}$, $W_{c}$, $W_{o}$,
$\boldsymbol{b}_{f}$, $\boldsymbol{b}_{i}$, $\boldsymbol{b}_{c}$, and
$\boldsymbol{b}_{o}$.

7: Update the weights and bias vectors of $LSTM_{1}$ using the gradient
descent-based method ADAM.

8: end for

9: Reformulate $LSTM_{2}$ with $W_{i}$, $W_{i}$, $W_{c}$, $W_{o}$,
$\boldsymbol{b}_{f}$, $\boldsymbol{b}_{i}$, $\boldsymbol{b}_{c}$, and
$\boldsymbol{b}_{o}$ corresponding to the least $\left\\{L^{(t)}_{l}\Big{|}\
\forall l\right\\}$.

10: Replicate $LSTM_{2}$ and define it as $LSTM_{1}$.

11: Forecast for $(T+t)$-th time step, i.e.,
$\hat{\boldsymbol{x}}^{(T+t)}=LSTM_{1}\left(\boldsymbol{X}^{(t+1)}\right)$

12: end for

###  2.3 State-of-the-art methods

Here, we present three state-of-the-art time series prediction methods,
namely, extended Kalman filter (EKF), autoregression, and autoregressive
integrated moving average (ARIMA), that we use to validate the performance of
our LSTM scheme. Here, we utilize the same sequential training as we did for
LSTMs to make real-time predictions on the same financial time series.

####  2.3.1 Extended Kalman Filter (EKF)

EKF is a nonlinear version of the standard Kalman filter where the formulation
of EKF is based on the linearization of both the state and the observation
equations. In an EKF, the state Jacobian and the measurement Jacobian replace
the state transition matrix and the measurement matrix, respectively, of a
linear KF (Valade et al. [2017]). This process essentially linearizes the non-
linear function around the current estimate. Linearization enables the
propagation of both the state and state covariance in an approximately linear
format. Here, the extended Kalman filter is presented in three steps, namely,
dynamic process, model forecast step, and data assimilation step.

##### Dynamic Process

Here, we present both the state model and the observation model of a nonlinear
dynamic process. The current state, $\boldsymbol{x}^{(t+1)}$, is modeled as a
sum of the nonlinear function of the previous state, $\boldsymbol{x}^{(t)}$,
and the noise, $\boldsymbol{w}^{(t)}$, as

$\boldsymbol{x}^{(t+1)}=f\left(\boldsymbol{x}^{(t)}\right)+\boldsymbol{w}^{(t)},$
(7)

where $\boldsymbol{x}^{(t)}$, $\boldsymbol{w}^{(t)}\in\mathbb{R}^{n}$. Here,
the random process $\\{\boldsymbol{w}^{(t)}\\}$ is Gaussian white noise with
zero mean and covariance matrix of
$Q^{(t)}=E\left[\boldsymbol{w}^{(t)}\left(\boldsymbol{w}^{(t)}\right)^{T}\right]$.
The initial state $\boldsymbol{x}^{(0)}$ is a random vector with known mean
$\boldsymbol{\mu}_{0}=E\left(\boldsymbol{x}^{(0)}\right)$ and covariance
$P^{(0)}=E\left[(\boldsymbol{x}^{(0)}-\boldsymbol{\mu}_{0})(\boldsymbol{x}^{(0)}-\boldsymbol{\mu}_{0})^{T}\right]$.
The Jacobian of the predicted state with respect to the previous state,
denoted as $J_{f}$, is obtained by partial derivatives as
$J_{f}=f_{\boldsymbol{x}}(\cdot)$.

The current observation, $\boldsymbol{y}^{(t+1)}$, is modeled as a sum of the
nonlinear function of the current state, $\boldsymbol{x}^{(t+1)}$, and the
noise, $\boldsymbol{v}^{(t+1)}$, as

$\boldsymbol{y}^{(t)}=h\left(\boldsymbol{x}^{(t)}\right)+\boldsymbol{v}^{(t)},$
(8)

where $\boldsymbol{y}^{(t+1)}$, $\boldsymbol{v}^{(t+1)}\in\mathbb{R}^{n}$.
Here, the random process $\\{\boldsymbol{v}^{(t+1)}\\}$ is Gaussian white
noise with zero mean and covariance matrix of
$R^{(t)}=E\left[\boldsymbol{v}^{(t)}\left(\boldsymbol{v}^{(t)}\right)^{T}\right]$.
The Jacobian of the predicted observation with respect to the previous state,
denoted as $J_{h}$, is obtained by partial derivatives as
$J_{h}=h_{\boldsymbol{x}}(\cdot)$.

##### Model Forecast Step

The state Jacobian and the measurement Jacobian replace linear KF’s state
transition matrix and the measurement matrix, respectively Valade et al.
[2017]. Let, the initial estimates of the state and the covariance are
$\boldsymbol{x}^{(0|0)}$ and $P^{(0|0)}$, respectively. The state and the
covariance matrix are propagated to the next step using

$\displaystyle\hat{\boldsymbol{x}}^{(t+1)}\approx
f\left(\hat{\boldsymbol{x}}^{(t)}\right)$ (9)

and

$\displaystyle
P^{(t+1)}=J_{f}\left(\hat{\boldsymbol{x}}^{(t)}\right)P^{(t)}\left[J_{f}\left(\hat{\boldsymbol{x}}^{(t)}\right)\right]^{T}+Q^{(t)},$
(10)

respectively.

##### Data Assimilation Step

The measurement at the $t+1$ step is given by

$\displaystyle\boldsymbol{y}^{(t+1)}\approx
h\left(\hat{\boldsymbol{x}}^{(t+1)}\right)$ (11)

Use the difference between the actual measurement and the predicted
measurement to correct the state at the $t+1$ step. To correct the state, the
filter must compute the Kalman gain. First, the filter computes the
measurement prediction covariance (innovation) as

$\displaystyle
S^{(t+1)}=J_{h}\left(\boldsymbol{x}^{(t+1)}\right)P^{(t+1)}\left[J_{h}\left(\boldsymbol{x}^{(t+1)}\right)\right]+R^{(t+1)}$
(12)

Then, the filter computes the Kalman gain as

$\displaystyle
K_{g}^{(t+1)}=P^{(t+1)}\left[J_{h}\left(\boldsymbol{x}^{(t+1)}\right)\right]^{T}\left[S^{(k+1)}\right]^{-1}$
(13)

The filter corrects the predicted estimate by using observation. The estimate,
after the correction using the observation $\boldsymbol{y}^{(t+1)}$, is

$\displaystyle\begin{split}\hat{\boldsymbol{x}}^{(t+1)}=\boldsymbol{x}^{(t+1)}+K^{(t+1)}\left[\boldsymbol{y}^{(t+1)}-h\left(\boldsymbol{x}^{(t+1)}\right)\right]\\\
P^{(t+1)}=\left[I-K_{g}^{(t+1)}J_{h}\left(\hat{\boldsymbol{x}}^{(t+1)}\right)\right]P^{(t+1)}\end{split}$
(14)

The corrected state is often called the a posteriori estimate of the state,
because it is derived after including the observation.

####  2.3.2 Autoregression (AR) model

Many observed time series exhibit serial autocorrelation which is known to be
the linear association between lagged observations. The AR model predicts the
value for the current time step, $\boldsymbol{x}^{(t)}$, based on a linear
relationship between $p$-recent observations, $\boldsymbol{x}^{(t-1)}$,
$\boldsymbol{x}^{(t-2)}$, $\dots$, $\boldsymbol{x}^{(t-p)}$, where this $p$ is
known as the order of the model Geurts et al. [1977]. Let,
$\alpha_{1},\dots,\alpha_{p}\in\mathbb{R}$ are the coefficients, order $p$ AR
model is given by

$\displaystyle\boldsymbol{x}^{(t)}=\boldsymbol{c}+\alpha_{1}\boldsymbol{x}^{(t-1)}+\alpha_{2}\boldsymbol{x}^{(t-2)}+...+\alpha_{p}\boldsymbol{x}^{(t-p)}+\boldsymbol{\epsilon}^{(t)},$
(15)

where $\boldsymbol{\epsilon}^{(t)}$ is uncorrelated noise with a zero mean.
Let, the lag operator polynomial notation is
$L^{i}\boldsymbol{x}^{(t)}=\boldsymbol{x}^{(t-i)}$. We define order $p$
autoregression lag operator polynomial as
$\alpha(L)=(1-\alpha_{1}L-\dots-\alpha_{p}L^{p})$. Thus, AE model is given by

$\displaystyle\alpha(L)\boldsymbol{x}^{(t)}=c+\boldsymbol{\epsilon}^{(t)}.$
(16)

The solution for the AR model is given by

$\displaystyle\boldsymbol{x}^{(t)}=\alpha^{-1}(L)\left(c+\boldsymbol{\epsilon}^{(t)}\right).$
(17)

####  2.3.3 Autoregressive integrated moving average (ARIMA) model

ARIMA model is made by combining a differential version of AR model into a
moving average (MA) model. MA model captures serial autocorrelation in a time
series
$\boldsymbol{x}^{(1)},\dots,\boldsymbol{x}^{(t)},\dots,\boldsymbol{x}^{(T)}$
by expressing the conditional mean of $\boldsymbol{x}^{(t)}$ as a function of
past innovations,
$\boldsymbol{a}^{t()},\boldsymbol{a}^{(t-1)},...,\boldsymbol{a}^{(t-q)}$. An
MA model that depends on $q$ past innovations is called an MA model of order
$q$, denoted by MA($q$). In general, the MA($q$) model can be represented by
the formula

$\displaystyle\boldsymbol{x}^{(t)}=\boldsymbol{\mu}+\boldsymbol{a}^{(t)}+\beta_{1}\boldsymbol{a}^{(t-1)}+...+\beta_{q}\boldsymbol{a}^{(t-q)},$
(18)

where $\boldsymbol{a}^{(t)}$’s are uncorrelated innovation processes with a
zero mean and $\mu$ is the unconditional mean of $\boldsymbol{x}^{(t)}$ for
all $t$.

For some observed time series, a higher order AR or MA model is needed to
capture the underlying process well. In this case, a combined ARMA model can
sometimes be a parsimonious choice. An ARMA model expresses the conditional
mean of $\boldsymbol{x}^{(t)}$ as a function of both recent observations,
$\boldsymbol{x}^{(t-1)},\boldsymbol{x}^{(t-2)},\dots,\boldsymbol{x}^{(t-p)}$,
and recent innovations,
$\boldsymbol{a}^{(t)},\boldsymbol{a}^{(t-1)},\dots,\boldsymbol{a}^{(t-q)}$.
The ARMA model with AR degree of $p$ and MA degree of $q$ is denoted by ARMA
($p,q$), which is given by

$\displaystyle\begin{split}\boldsymbol{x}^{(t)}=\boldsymbol{c}+\alpha_{1}\boldsymbol{x}^{(t-1)}+\alpha_{2}\boldsymbol{x}^{(t-2)}+\dots+\alpha_{p}\boldsymbol{x}^{(t-p)}\\\
+\beta_{0}\boldsymbol{a}^{(t)}+\beta_{1}\boldsymbol{a}^{(t-1)}+\dots+\beta_{q}\boldsymbol{a}^{(t-q)},\end{split}$
(19)

Shumway & Stoffer [2017].

The ARIMA process generates nonstationary series that are integrated of order
$D$ where that nonstationary process can be made stationary by taking $D$
differences. A series that can be modeled as a stationary ARMA($p,q$) process
after being differenced $D$ times is denoted by ARIMA($p,D,q$), which is given
by

$\begin{split}\Delta^{D}\boldsymbol{x}^{(t)}=\mu+\alpha_{1}\Delta^{D}\boldsymbol{x}^{(t-1)}+\alpha_{2}\Delta^{D}\boldsymbol{x}^{(t-2)}+...+\alpha_{p}\Delta^{D}\boldsymbol{x}^{(t-p)}\\\
+\boldsymbol{a}^{(t)}-\beta_{1}\boldsymbol{a}^{(t-1)}-\beta_{2}\boldsymbol{a}^{(t-2)}-...-\beta_{q}\boldsymbol{a}^{(t-q)},\end{split}$
(20)

where $\Delta^{D}\boldsymbol{x}^{(t)}$ denotes a $D$-th differential time
series, $\boldsymbol{a}^{(t)}$’s are uncorrelated innovation processes with a
zero mean, and $\mu$ is the unconditional mean of $\boldsymbol{x}^{(t)}$ for
all $t$ [Newbold, 1983]. With the lag operator
$L^{i}\boldsymbol{x}^{(t)}=\boldsymbol{x}^{(t-i)}$, the ARIMA model can be
written as

$\displaystyle\alpha(L){(1-L)}^{D}\boldsymbol{x}^{(t)}=c+\beta(L)\boldsymbol{a}^{(t)}$
(21)

where $\alpha(L)=(1-\alpha_{1}L-\dots-\alpha_{p}L^{p})$ and
$\beta(L)=(1+\beta_{1}L+\dots+\beta_{q}L^{q})$. Thus, the solution for ARIMA
model is given by

$\displaystyle\boldsymbol{x}^{(t)}=\left(\alpha(L){(1-L)}^{D}\right)^{-1}\left(c+\beta(L)\boldsymbol{a}^{(t)}\right).$
(22)

##  3 Performance Analysis

The performance analysis of LSTM is conducted using nine financial time series
obtained from three markets, namely, stocks, cryptocurrencies, and
commodities. We chose Apple, Google, and Microsoft for stocks; Bitcoin,
Ethereum, Cardano for cryptocurrencies; and Gold, Oil, and Natural Gas for
commodities. These diverse examples validate the broad applicability of LSTMs
in analyzing and predicting financial time series.

We follow the procedure given in Fig. 1 to train the real-time many-to-one
LSTM architecture given in Fig. 2. Setting the LSTM to run for a specific
number of epochs and then using that trained network to make predictions often
do not perform the best training and then do not perform accurate predictions
since the loss function undergoes semi-convergence as shown in Fig. 3. To
avoid this issue; first, we train the LSTM for 100 epochs; second, we compute
the best number of epochs associated with the least loss; and finally, train
again a new LSTM with that many epochs. Moreover, the parameter choices for
the training length and prediction length are shown in Table 3.

Table 3: Parameter choices for the training length, prediction length, and number of epochs used in real-time many-to-one LSTMs. time series | Training length | Prediction length
---|---|---
| ($T$) | ($N$)
Apple | 1228 | 30
Microsoft | 1228 | 30
Google | 1228 | 30
Bitcoin | 1064 | 30
Ethereum | 1064 | 30
Cardano | 1064 | 30
Oil | 8248 | 200
Natural gas | 5802 | 150
Gold | 816 | 30

Now, we incorporate the same one-day recursive prediction procedure in Fig. 1
into the other three state-of-the-art methods, namely, EKF, AR, and ARIMA, to
predict the above financial time series. After a trial and error process, we
found that the best $p$’s of AR are 300, 400, and 400, for Apple, Microsoft,
and Google, respectively; and the best ($p,D,q$)’s of ARIMA are (10, 0, 2),
(10, 2, 1), and (0, 1 , 1), for Apple, Microsoft, and Google, respectively.
Then, the best $p$’s of AR were found to be 100, 100, and 300, for Bitcoin,
Ethereum, and Cardano, respectively; and the best ($p,D,q$)’s of ARIMA were
found to be (6, 0, 2), (6, 1, 1), and (8, 2 , 1), for Bitcoin, Ethereum, and
Cardano, respectively. Finally, the best $p$’s of AR were 200, 200, and 100,
for Oil, Natural gas, and Gold, respectively; and the best ($p,D,q$)’s of
ARIMA were (4, 1, 1), (10, 1, 2), and (8, 2 , 0), for Oil, Natural gas, and
Gold, respectively. Thus, we set the methods with the best parameter values
and executed them with the corresponding time series.

We compute the mean of the relative absolute difference between the predicted
and the observed time series for the prediction period using

$\mathcal{E}=\frac{1}{N}\sum^{T+N}_{t=T+1}\frac{\big{\|}\hat{\boldsymbol{x}}^{(t)}-\boldsymbol{x}^{(t)}\big{\|}_{2}}{\big{\|}\boldsymbol{x}^{(t)}\big{\|}_{2}},$
(23)

as an error measure of the prediction that we show in Table 4. Hereby, we
observe that the order of the best to the worst prediction performance is
LSTM, EKF, AR, and ARIMA.

Table 4: This table shows the prediction error, quantified as the means of the relative difference between the predicted and the observed time series for the prediction period, of four methods LSTM, EKF, AR, and ARIMA. The analysis is conducted on three stocks Apple, Microsoft, and Google; three cryptocurrencies, Bitcoin, Ethereum, and Cardano; and three commodities, oil, natural gas, and gold. On average, LSTM performs 17 times better than EKF, 7 times better than AR, and 4 times better than ARIMA. Moreover, the average prediction errors of LSTM are 0.05, 0.22, and 0.14 for stocks, cryptocurrencies, and commodities, respectively. time series | LSTM | EKF | AR | ARIMA
---|---|---|---|---
Apple | $5.9*10^{-2}$ | $4.5*10^{-1}$ | $2.5*10^{-1}$ | $7.5*10^{-2}$
Microsoft | $5.5*10^{-2}$ | $4.2*10^{-1}$ | $4.3*10^{-1}$ | $6.3*10^{-2}$
Google | $3.5*10^{-2}$ | $4.7*10^{-1}$ | $4.7*10^{-1}$ | $5.5*10^{-2}$
Bitcoin | $2.1*10^{-1}$ | $2.7*10^{0}$ | $4.0*10^{-1}$ | $3.8*10^{-1}$
Ethereum | $1.8*10^{-1}$ | $2.9*10^{0}$ | $8.1*10^{-1}$ | $1.0*10^{0}$
Cardano | $2.7*10^{-1}$ | $4.3*10^{0}$ | $1.2*10^{0}$ | $1.7*10^{0}$
Oil | $1.5*10^{-1}$ | $4.6*10^{0}$ | $6.3*10^{-1}$ | $3.0*10^{-1}$
Natural gas | $2.0*10^{-1}$ | $3.4*10^{0}$ | $8.0*10^{-1}$ | $1.0*10^{0}$
Gold | $7.7*10^{-2}$ | $2.4*10^{0}$ | $1.2*10^{0}$ | $1.1*10^{0}$

Fig. 4 shows the price predictions of the three stocks, Apple, Microsoft, and
Google, using our real-time many to one LSTM, EKF, AR, and ARIMA. Since some
of the predictions closely mimic the observed time series to overlap, we
compute the absolute difference between the observations and the predictions,
see Figs. 4(c), 4(f), and 4(i). We observe that all four methods are capable
of capturing the pattern of the time series with the order of the best to the
worst prediction performance is LSTM, ARIMA, AR, and EKF. Moreover, while LSTM
and ARIMA perform similarly good, EKF and AR perform similarly weak.

Figure 4: Price prediction of three stocks, Apple (first row), Microsoft
(second row), and Google (third row), using our real-time many to one LSTM
(blue), EKF (yellow), AR (green), and ARIMA (purple). The first, second, and
third columns show the entire time series, observed and predicted time series
for the last 30 days of the prediction period, and the absolute difference
between observed and predicted time series, respectively. We observe that the
order of the best to the worst performance is LSTM, ARIMA, AR, and EKF. Note
that some of the time series are not visible as they are covered by others.

Fig. 5 shows the price predictions of the three cryptocurrencies, Bitcoin,
Ethereum, and Cardano, using LSTM, EKF, AR, and ARIMA. Since some of the
predictions closely mimic the observed time series to overlap, we compute the
absolute difference between the observations and the predictions, see Figs.
4(c), 4(f), and 4(i). We observe that LSTM, ARIMA, and AR are capable of
capturing the pattern of the time series in contrast to the weak prediction of
EKF. The order of the best to the worst prediction performance is LSTM, AR,
ARIMA, and EKF.

Figure 5: Price prediction of three cryptocurrencies, Bitcoin (first row),
Ethereum (second row), and Cardano (third row), using our real-time many to
one LSTM (blue), EKF (yellow), AR (green), and ARIMA (purple). The first,
second, and third columns show the entire time series, observed and predicted
time series for the last 30 days of the prediction period, and the absolute
difference between observed and predicted time series, respectively. We
observe that the order of the best to the worst performance is LSTM, AR,
ARIMA, and EKF. Note that some of the time series are not visible as they are
covered by others.

Fig. 6 shows the price predictions of the three commodities, Oil, Natural gas,
and Gold, using LSTM, EKF, AR, and ARIMA. We compute the absolute difference
between the observations and the predictions, see Figs. 5(c), 5(f), and 5(i)
since some of the predictions are similar to observations. We observe that
mostly LSTM, ARIMA, and AR are capable of capturing the pattern of the time
series. The order of the best to the worst prediction performance is LSTM, Ar,
ARIMA, and EKF.

Figure 6: Price prediction of three commodities, Oil (first row), Natural gas
(second row), and Gold (third row), using our real-time many to one LSTM
(blue), EKF (yellow), AR (green), and ARIMA (purple). The first, second, and
third columns show the entire time series, observed and predicted time series
for the last 30 days of the prediction period, and the absolute difference
between observed and predicted time series, respectively. We observe that the
order of the best to the worst performance is LSTM, AR, ARIMA, and EKF. Note
that some of the time series are not visible as they are covered by others.

The performance of this real-time many-to-one LSTM is highly influenced by the
number of epochs that it is executed. To check this assertion, we compute the
prediction performance of the LSTM with respect to different numbers of epochs
for Apple, Bitcoin, and Gold. The prediction performance is computed as the
mean of the relative absolute difference, i.e., $\mathcal{E}$, between the
prediction and the observed time series. Since EKF, AR, and ARIMA are
independent of epochs, we represent their $\mathcal{E}$ as a straight line. We
observe that the performance of LSTM improves from worst to the best when the
number of epochs is increased.

Figure 7: Prediction performance of the real-time many-to-one LSTM with
respect to the different numbers of epochs. The first row shows the mean of
the relative absolute difference, denoted as $\mathcal{E}$, between the
prediction and the observed time series for Apple, Bitcoin, and Gold. Note
that, EKF, AR, and ARIMA are independent of epochs; however, we represent a
straight line for their $\mathcal{E}$ in the first row. The second, the third,
and the fourth rows show the prediction and the observed time series of the
prices in the prediction periods of Apple, Bitcoin, and Gold for 10 epochs, 20
epochs, and 50 epochs, respectively. We observe that the performance of LSTM
improves from worst to the best when the number of epochs is increased. Note
that some of the time series and plots are not visible as they are covered by
others.

##  4 Discussion

The classical methods of solving temporal chaotic systems are mostly linear
models which assume linear relationships between systems’ previous outputs for
stationary time series. Thus, they often do not capture non-linear
relationships in the data and cannot cope with certain non-stationary signals.
Because financial time series are often nonstationary, nonlinear, and contain
noise [Bontempi et al., 2013], traditional statistical models encounter some
limitations in predicting them with high precision. In this paper, we have
presented a real-time forecasting technique for financial markets using
sequentially trained many-to-one LSTM. We applied this technique for some time
series obtained from stock market, cryptocurrency market, and commodity
market, then, compared the performance against three state-of-the-art methods,
namely, EKF, AR, and ARIMA.

Here, we train a many-to-one LSTM with sequential data sampled using a moving
window approach such that the succeeding window is shifted forward by one data
instance from the preceding window. Such sequential window training plays an
important role in time series predictions since it, 1) helps generate more
data from a given limited time series and then thorough training of the ANN;
2) makes the data heterogeneous so that the overfitting issue of the ANN can
be reduced; and 3) facilitates the learning of patterns of the data not only
for the entire time series but also for short segments of sequential data.
Sequential window training maximizes the performance of this LSTM as
accelerates LSTM’s learning capability as well as it increases LSTM’s
robustness to new data.

The performance analysis of this study covers the LSTM applied to nine time
series obtained from three financial markets, stocks (Apple, Microsoft,
Google), cryptocurrencies (Bitcoin, Ethereum, Cardano), and commodities (gold,
crude oil, natural gas). We observed that the LSTM performs exceptionally
better than the other three methods for all the nine datasets where the
performance of EKF was significantly weak. We have seen in Table 4, on
average, LSTM performs 17 times better than EKF, 7 times better than AR, and 4
times better than ARIMA. The average prediction errors of LSTM are 0.05, 0.22,
and 0.14 for stocks, cryptocurrencies, and commodities, respectively. The
reason for that is while the prediction on less volatile time series like in
the stock market is easy, the prediction on high volatile time series like in
the cryptocurrency market is challenging.

In future work, we are planning to extend this sequentially trained many-to-
one LSTM to employ as a real-time fault detection technique in industrial
production processes. This real-time fault detection scheme will be capable of
producing an early alarm to alert a shift in the production process so the
quality controlling team can take necessary actions. Moreover, trajectories of
collectively moving agents can be represented on a low-dimensional manifold
that underlies on a high-dimensional data cloud Gajamannage et al. [2019],
Gajamannage & Paffenroth [2021], Gajamannage et al. [2015]. However, some
segments of these trajectories are not tracked by multi-object tracking
methods due to natural phenomena such as occlusions. Thus, we are planning to
utilize our LSTM architecture to make predictions for the fragmented segments
of the trajectories.

We empirically validated that our real-time LSTM outperforms the performance
of EKF, AR, and ARIMA. In the future, we are planning to compare the
performance of our real-time LSTM with that of the other famous ANN-based
methods such as Facebook developed Prophet [Taylor & Letham, 2018], Amazon
developed DeepAR [Salinas et al., 2020], Google developed Temporal Fusion
Transformer [Lim et al., 2021], and Element AI developed N-BEATS [Oreshkin et
al., 2019]. Prophet was designed for automatic forecasting of univariate time
series data. DeepAR is a probabilistic forecasting model based on recurrent
neural networks. Temporal Fusion Transformer is a novel attention-based
architecture that combines high-performance multi-horizon forecasting with
interpretable insights into temporal dynamics. N-BEATS is a custom deep
learning algorithm that is based on backward and forward residual links for
univariate time series point forecasting.

We presented both nonlinear and real-time prediction technique for financial
time series that is made by a many-to-one LSTM which is sequentially trained
with windows of data. The sequential window training approach has
significantly improved LSTM’s learning ability while dramatically reducing
LSTM’s over-fitting issues. We empirically justified that our LSTM possesses
superior performance even for highly volatile time series such as those in
cryptocurrencies and commodities.

## Acknowledgments

The authors would like to thank the Google Cloud Platform for granting
Research Credit to access its GPU computing resources under project number
397744870419.

## References

  * Allen-Zhu et al. [2019] Allen-Zhu, Z., Li, Y., & Song, Z. (2019).  On the convergence rate of training recurrent neural networks.  In Advances in Neural Information Processing Systems (pp. 1310–1318).  PMLR volume 32.  arXiv:1810.12065.
  * Bahadur et al. [2019] Bahadur, N., Paffenroth, R., & Gajamannage, K. (2019).  Dimension Estimation of Equity Markets.  In Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019 (pp. 5491--5498).  Institute of Electrical and Electronics Engineers Inc.  doi:10.1109/BigData47090.2019.9006343.
  * Bontempi et al. [2013] Bontempi, G., Ben Taieb, S., & Le Borgne, Y. A. (2013).  Machine learning strategies for time series forecasting.  In Lecture Notes in Business Information Processing (pp. 62--77).  Springer volume 138 LNBIP.  doi:10.1007/978-3-642-36318-4_3.
  * Gajamannage et al. [2015] Gajamannage, K., Butail, S., Porfiri, M., & Bollt, E. M. (2015).  Identifying manifolds underlying group motion in Vicsek agents.  European Physical Journal: Special Topics, 224, 3245--3256. doi:10.1140/epjst/e2015-50088-2.
  * Gajamannage & Paffenroth [2021] Gajamannage, K., & Paffenroth, R. (2021).  Bounded manifold completion.  Pattern Recognition, 111, 107661. doi:https://doi.org/10.1016/j.patcog.2020.107661.
  * Gajamannage et al. [2019] Gajamannage, K., Paffenroth, R., & Bollt, E. M. (2019).  A nonlinear dimensionality reduction framework using smooth geodesics.  Pattern Recognition, 87, 226--236. doi:10.1016/j.patcog.2018.10.020.
  * Gajamannage et al. [2021] Gajamannage, K., Park, Y., Paffenroth, R., & Jayasumana, A. P. (2021).  Reconstruction of Fragmented Trajectories of Collective Motion using Hadamard Deep Autoencoders.  arXiv preprint arXiv:2110.10428, . doi:10.48550/arxiv.2110.10428. arXiv:2110.10428.
  * Geurts et al. [1977] Geurts, M., Box, G. E. P., & Jenkins, G. M. (1977).  Time Series Analysis: Forecasting and Control.  Journal of Marketing Research, 14, 269. doi:10.2307/3150485.
  * Gruslys et al. [2016] Gruslys, A., Munos, R., Danihelka, I., Lanctot, M., & Graves, A. (2016).  Memory-efficient backpropagation through time.  Advances in Neural Information Processing Systems, 29, 4132--4140. arXiv:1606.03401.
  * Hochreiter & Schmidhuber [1997] Hochreiter, S., & Schmidhuber, J. (1997).  Long short-term memory.  Neural computation, 9, 1735--1780. doi:10.1162/neco.1997.9.8.1735.
  * Kim et al. [2017] Kim, J., El Khamy, M., & Lee, J. (2017).  Residual LSTM: Design of a deep recurrent architecture for distant speech recognition.  Proceedings of the Annual Conference of the International Speech Communication Association, 2017-Augus, 1591--1595. doi:10.21437/Interspeech.2017-477.
  * Kingma & Ba [2015] Kingma, D. P., & Ba, J. L. (2015).  Adam: A method for stochastic optimization.  3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, . doi:10.48550/arXiv.1412.6980. arXiv:1412.6980.
  * Le & Zuidema [2016] Le, P., & Zuidema, W. (2016).  Quantifying the Vanishing Gradient and Long Distance Dependency Problem in Recursive Neural Networks and Recursive LSTMs.  arXiv preprint arXiv:1603.00423, (pp. 87--93). doi:10.18653/v1/w16-1610. arXiv:1603.00423.
  * Lim et al. [2021] Lim, B., Arık, S., Loeff, N., & Pfister, T. (2021).  Temporal Fusion Transformers for interpretable multi-horizon time series forecasting.  International Journal of Forecasting, 37, 1748--1764. doi:10.1016/j.ijforecast.2021.03.012. arXiv:1912.09363.
  * Ma & Principe [2018] Ma, Y., & Principe, J. (2018).  Comparison of Static Neural Network with External Memory and RNNs for Deterministic Context Free Language Learning.  In Proceedings of the International Joint Conference on Neural Networks (pp. 1--7).  IEEE volume 2018-July.  doi:10.1109/IJCNN.2018.8489240.
  * Manneschi & Vasilaki [2020] Manneschi, L., & Vasilaki, E. (2020).  An alternative to backpropagation through time.  Nature Machine Intelligence, 2, 155--156. doi:10.1038/s42256-020-0162-9.
  * Newbold [1983] Newbold, P. (1983).  ARIMA model building and the time series analysis approach to forecasting.  Journal of Forecasting, 2, 23--35. doi:10.1002/for.3980020104.
  * Oreshkin et al. [2019] Oreshkin, B. N., Carpov, D., Chapados, N., & Bengio, Y. (2019).  N-BEATS: Neural basis expansion analysis for interpretable time series forecasting.  arXiv preprint arXiv:1905.10437, . URL: http://arxiv.org/abs/1905.10437. arXiv:1905.10437.
  * Park et al. [2022] Park, Y., Gajamannage, K., Jayathilake, D. I., & Bollt, E. M. (2022).  Recurrent Neural Networks for Dynamical Systems : Applications to Ordinary Differential Equations , Collective Motion , and Hydrological Modeling, .  (pp. 1--15). doi:10.48550/arxiv.2202.07022.
  * Qiu et al. [2020] Qiu, J., Wang, B., & Zhou, C. (2020).  Forecasting stock prices with long-short term memory neural network based on attention mechanism.  PLoS ONE, 15, e0227222. doi:10.1371/journal.pone.0227222.
  * Salinas et al. [2020] Salinas, D., Flunkert, V., Gasthaus, J., & Januschowski, T. (2020).  DeepAR: Probabilistic forecasting with autoregressive recurrent networks.  International Journal of Forecasting, 36, 1181--1191. doi:10.1016/j.ijforecast.2019.07.001. arXiv:1704.04110.
  * Shih et al. [2018] Shih, C. H., Yan, B. C., Liu, S. H., & Chen, B. (2018).  Investigating Siamese LSTM networks for text categorization.  In Proceedings - 9th Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2017 (pp. 641--646).  IEEE volume 2018-Febru.  doi:10.1109/APSIPA.2017.8282104.
  * Shumway & Stoffer [2017] Shumway, R. H., & Stoffer, D. S. (2017).  ARIMA Models.  (pp. 75--163).  Springer, Cham.  doi:10.1007/978-3-319-52452-8_3.
  * Simistira et al. [2015] Simistira, F., Ul-Hassan, A., Papavassiliou, V., Gatos, B., Katsouros, V., & Liwicki, M. (2015).  Recognition of historical Greek polytonic scripts using LSTM networks.  In Proceedings of the International Conference on Document Analysis and Recognition, ICDAR (pp. 766--770).  IEEE volume 2015-Novem.  doi:10.1109/ICDAR.2015.7333865.
  * Squarepants [2022] Squarepants, S. (2022).  Bitcoin: A Peer-to-Peer Electronic Cash System.  SSRN Electronic Journal, (p. 21260). doi:10.2139/ssrn.3977007.
  * Taylor & Letham [2018] Taylor, S. J., & Letham, B. (2018).  Forecasting at Scale.  American Statistician, 72, 37--45. doi:10.1080/00031305.2017.1380080.
  * Tian et al. [2017] Tian, X., Zhang, J., Ma, Z., He, Y., Wei, J., Wu, P., Situ, W., Li, S., & Zhang, Y. (2017).  Deep LSTM for large vocabulary continuous speech recognition, .  doi:10.48550/arXiv.1703.07090. arXiv:1703.07090.
  * Valade et al. [2017] Valade, A., Acco, P., Grabolosa, P., & Fourniols, J. Y. (2017).  A study about kalman filters applied to embedded sensors.  Sensors (Switzerland), 17, 2810. doi:10.3390/s17122810.
  * Werbos [1990] Werbos, P. J. (1990).  Backpropagation Through Time: What It Does and How to Do It.  Proceedings of the IEEE, 78, 1550--1560. doi:10.1109/5.58337.
  * Zhao et al. [2018] Zhao, Y., Ge, L., Zhou, Y., Sun, Z., Zheng, E., Wang, X., Huang, Y., & Cheng, H. (2018).  A new Seasonal Difference Space-Time Autoregressive Integrated Moving Average (SD-STARIMA) model and spatiotemporal trend prediction analysis for Hemorrhagic Fever with Renal Syndrome (HFRS).  PLoS ONE, 13, e0207518. doi:10.1371/journal.pone.0207518.