#  FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and
Character Design

Yangyang Yu∗, Haohang Li∗, Zhi Chen∗, Yuechen Jiang∗, Yang Li∗
Denghui Zhang, Rong Liu, Jordan W. Suchow, Khaldoun Khashanah†
Stevens Institute of Technology
Hoboken, NJ, United States
{yyu44, hli113, zchen100, yjiang52, yli269, dzhang42, rliu20<EMAIL_ADDRESS>

###### Abstract

Recent advancements in Large Language Models (LLMs) have exhibited notable
efficacy in question-answering (QA) tasks across diverse domains. Their
prowess in integrating extensive web knowledge has fueled interest in
developing LLM-based autonomous agents. While LLMs are efficient in decoding
human instructions and deriving solutions by holistically processing
historical inputs, transitioning to purpose-driven agents requires a
supplementary rational architecture to process multi-source information,
establish reasoning chains, and prioritize critical tasks. Addressing this, we
introduce FinMem, a novel LLM-based agent framework devised for financial
decision-making. It encompasses three core modules: Profiling, to customize
the agent’s characteristics; Memory, with layered message processing, to aid
the agent in assimilating hierarchical financial data; and Decision-making, to
convert insights gained from memories into investment decisions. Notably,
FinMem’s memory module aligns closely with the cognitive structure of human
traders, offering robust interpretability and real-time tuning. Its adjustable
cognitive span allows for the retention of critical information beyond human
perceptual limits, thereby enhancing trading outcomes. This framework enables
the agent to self-evolve its professional knowledge, react agilely to new
investment cues, and continuously refine trading decisions in the volatile
financial environment. We first compare FinMem with various algorithmic agents
on a scalable real-world financial dataset, underscoring its leading trading
performance in stocks. We then fine-tuned the agent’s perceptual span and
character setting to achieve a significantly enhanced trading performance.
Collectively, FinMem presents a cutting-edge LLM agent framework for automated
trading, boosting cumulative investment returns.

††footnotetext: Corresponding author. Email:
<EMAIL_ADDRESS>Equal contribution, with author order
decided by dice roll.22footnotetext: The source code of this project can be
found via: FinMem LLM Trading

KEYWODS: Financial AI, Large Language Model, Trading Algorithms, Deep
Learning, Financial Technology

##  1 Introduction

With the influx of diverse financial data streams from the web, traders face a
deluge of information from various sources. This requires them rapidly to
understand, memorize, and filtrate crucial events for investment decisions.
However, innate cognitive limitations restrict human traders from processing
information within their perception and memory capacity, a span much narrower
than the actual volume of available information [5]. Consequently,
insufficiently considering or even dismissing critical events affecting
trading decisions becomes increasingly concerning as data availability
expands. To overcome the physical limitations in the memory systems of human
traders, researchers have been consistently working on designing autonomous
trading agent systems. These systems need to thoroughly integrate all
available information and possess a sophisticated design in the agent’s
backbone algorithm to deliver enhanced trading performance.

The evolution of autonomous trading systems has transitioned from the initial
rule-based trading strategies [12] to more advanced machine-learning-based
algorithms [19]. In recent years, Reinforcement Learning (RL)-based agents
[14], especially those employing Deep Reinforcement Learning (DRL) [29] as
backbone algorithms, garner joint attention of both academia and industry.
Leveraging both RL principles and deep learning, DRL agents effectively handle
and learn from scalable and diverse financial data, including stock prices,
key financial indicators, and market sentiments. They utilize deep neural
networks to extract expressive features from input data, representing the
complex financial market environment, which enhances their comprehension
ability. Retaining the key features of RL agents, they learn through
interaction with a predefined environment to maximize investment gain over
time. Research suggests DRLs can meet the crucial needs of trading agents to
process and make informed decisions from large volumes of data. However,
certain inherent features of DRL algorithms exhibit notable deficiencies in
financial applications. Firstly, DRL agents exhibit a lack of interpretability
concerning the rationale behind their decisions [4]. They are often described
as “black boxes,” where the internal processes and computational layers
leading to a specific decision are neither easily understandable nor
transparent. Secondly, DRL agents find it challenging to effectively integrate
textual data with numerical features. Text data plays a vital role in finance,
since the majority of market information is conveyed through news articles and
financial reports. However, transforming text data to embeddings considerably
increases input space dimensionality, making learning more computationally
demanding. Plus, empirical studies have shown that combining textual
representations with numerical financial indicators often leads to convergence
challenges [15]. Therefore, a backbone algorithm with transparent reasoning
and the enhanced ability to capture investment-related textual insights
comprehensively is essential.

Recent advancements in Large Language Models (LLMs), like Generative Pre-
trained Transformers (GPTs) [34], offer viable solutions for developing
trading agents, alleviating previous concerns. With carefully designed
prompts, the LLM-based agent is able to provide reasons and outcomes in plain
text. This allows for immediate observation and prompt adjustment of its
reasoning process. Employing LLMs as the backbone algorithms for agents also
overcomes the constraint of isolated environments. This is achieved through
their vast pre-existing knowledge and the effective integration of valuable
insights from a variety of data sources, including both textual and numerical.
When equipped with suitable prompt templates, this approach significantly
enhances decision-making capabilities [50]. Studies indicate that prompt-
guided reasoning significantly improves problem-solving rates across various
domains [24]. Notably, a growing body of research has focused on utilizing
LLMs to make informed trading decisions for stocks and funds by continuously
interacting with financial environment information [55, 52]. However, in
currently available approaches, LLMs primarily serve as a QA role rather than
functioning as autonomous agents. The potential issue with these approaches is
the incapability of fully understanding the varying timeliness associated with
different types of financial data. These financial LLM agents, despite
outperforming traditional trading benchmarks, generally process information
indiscriminately through QA iterations, lacking the ability to memorize
influential messages. Furthermore, their method of acknowledging the
timeliness of financial data is heavily dependent on the uncertain and
laborious LLM fine-tuning process. These insufficiencies undermine their
ability to update the knowledge base in a daily manner, meaning they lack a
memory component. As a result, they may struggle to prioritize significant and
influential memory events effectively. Additionally, current literature on
LLM-based trading agents lacks a comparative analysis between these
applications and other autonomous trading systems, such as DRL agents.

To bridge this gap, we present FinMem, an innovative LLM-based autonomous
trading agent with a novel layered memory system and dynamic character design.
Unlike previous LLM agents in finance, FinMem encompasses a memory module
adept at processing multi-source financial data with varying timeliness and
self-adaptive character setting for fitting into volatile market environments.

Our concept is initially inspired by the Generative Agents framework by Park
et al. [35], aimed at enhancing the efficient retrieval of key events for
general-purpose LLM agents. This framework features a unique character design
and seed memory, activating the agent upon specific query through prompts. It
prioritizes events in a unified memory stream, ranked by a linear combination
of recency, relevancy, and importance. The framework outlined in [35] provides
a foundational structure for LLM agent design. It includes a profiling module
for character definition, a memory module for experience recording and
critical information retrieval, and an action module to guide actions based on
the retrieved memories. This structure effectively facilitates goal
achievement for the agent in a general social environment. However, Park et
al.’s framework struggles with comprehending financial data with varying
timeliness and importance, like daily news versus quarterly and annual
reports. Key challenges involve quantifying the timeliness of different
information sources, optimizing information retrieval, and enhancing trading
decisions with detailed analysis. To tackle these challenges, we further
propose FinMem with the following improvements.

FinMem maintains a modular approach similar to Park et al. [35], but features
novel design of profiling and memory modules. Its specialized profiling module
equips FinMem with a trading-task-specific professional background, enhancing
robustness to market fluctuations via offering self-adaptive risk inclination
option. FinMem’s memory module innovatively incorporates working memory and
layered long-term memory components, ideal for stratified information
processing. Its working memory acts as a dynamic “workspace,” enabling
operations like summarization, observation, and reflection on multi-source
information to facilitate trading decisions. Its long-term memory, structured
into shallow, intermediate, and deep layers [9], manages varied decay rates to
satisfy the need to retain distinct types of financial information within
different time scales based on their corresponding timeliness. For instance,
daily news, with its immediate effects on stock markets, is channeled into the
shallow processing layer. Meanwhile, annual company reports, exerting a more
prolonged impact, are processed in the deep layer by FinMem. Each layer in
FinMem prioritizes memory events based on the assemble of recency, relevancy,
and importance close to Park et al.’s method. However, it introduces new
measurements for recency and importance, specifically tailored to better rank
financial data according to their unique time sensitivity. FinMem’s memory
mechanism can also transit significantly impactful investment memory events to
deeper processing layers, ensuring their retention for extended periods.
FinMem’s memory module can mirror the human cognitive system [47] and
facilitate agile, real-time decisions [46]. It enables continuous evolution in
professional knowledge through structured summarizing, retrospecting past
experiences, and reacting to new trading scenarios. Additionally, FinMem
includes a decision-making module capable of deriving investment decisions by
considering top-ranked memory events and current market conditions.

Through experiments, we show that FinMem exhibits an outstanding ability to
stratify and leverage the various levels of market insights, significantly
improving the quality of trading decisions. We claim that FinMem provides
these key contributions:

FinMem presents a state-of-the-art LLM-based trading agent with a human-
aligned memory mechanism and character design, particularly crafted to capture
investment insights from the financial market. In its agent memory module
design, FinMem innovatively emulates human working and layered long-term
memory mechanisms. This approach effectively harnesses the time-sensitive
aspects of financial data, capturing crucial investment insights and thereby
boosting trading performance. FinMem’s profiling module includes a dynamic
character setting feature, offering seed information about professional
backgrounds and adjustable risk inclinations. Additionally, it continuously
updates domain knowledge as the trading experience grows, thereby enhancing
FinMem’s capabilities. Ablation studies demonstrate that FinMem can learn from
past trading experiences and evolve its knowledge base through continuous
market interaction, maintaining robustness in complex markets for profitable
trading decisions.

FinMem can utilize its distinctive features to expand the agent’s perceptual
range beyond the human limitation to make well-informed trading decisions.
Cognitive research indicates that human working memory can recall only five to
nine events at a time [30]. This limitation, while preventing information
overload, can yield insufficient insight for precise decision-making. In
contrast, FinMem’s memory module transcends this constraint. It allows
adjusting cognitive load by selecting a flexible number of top-ranked events
from each layer of its hierarchical long-term memory, allowing FinMem to
maintain agility and deliver superior trading decisions in data-rich contexts.

FinMem achieves impressive trading performance using training data that is
limited in volume and spans a short time period. Our experiments indicate that
training FinMem with daily collected data over just six months to a year is
enough to produce robust and notable trading results. This timeframe is
considerably shorter than what other comparable models require. This
efficiency is achieved through the optimal utilization of multi-source
incoming data and the precise identification of key signals for trading
decisions. Furthermore, it’s noteworthy that FinMem’s effectiveness is
demonstrated on smaller datasets and with general-purpose LLMs. Its
capabilities are anticipated to be further amplified with access to larger,
higher-quality financial datasets and LLMs fine-tuned specifically for
financial applications.

In this paper, we begin by explaining the three core modules of FinMem.
Subsequently, we emphasize its superior trading performance compared to a
range of representative algorithmic agents. We further explore how FinMem
achieves its optimal performance, examining adjustments in three key aspects:
backbone algorithms, working memory capacity, and character settings.

##  2 Related Work

###  2.1 Backbone Algorithms of Contemporary Autonomous Trading Agents

The development of trading agents has evolved over several decades, influenced
by advancements in technology, finance, and computational methodologies.
Conventionally, a rule-based algorithm for trading stocks is an automated
strategy that operates based on a predefined set of rules[7, 49, 37]. These
rules are often derived from historical market patterns and trading
experience. Compared with rule-based algorithms that use predefined rules and
conditions, Reinforcement Learning provides a way for agents to learn by
interacting with an environment and receiving feedback in the form of rewards
or penalties. [10, 22]. Deep learning models can be integrated with RL to
handle large and complex state spaces, like those in stock markets. Such
models are often referred to as Deep Reinforcement Learning (DRL) [53, 54].
For example, Deep Q-Network (DQN) [44], Advantage Actor-Critic (A2C) [56], and
Proximal Policy Optimization (PPO) [27] are popular algorithms for such tasks.
Using DRL agents as automated financial trading backbones, face two key
issues: 1) A lack of interpretability, as their decisions, rooted in complex
computations and high-dimensional representations, are challenging to
articulate [4]. 2) They struggle to fully leverage textual financial
information due to the high-dimensional nature and computational intensity of
rich text embeddings [11, 13]. Consequently, DRL agents often rely on
extracting textual sentiment [39], sidestepping the direct use of embeddings
[6, 3], leading to an incomplete representation of crucial market information
embedded in news and macroeconomic policies.

###  2.2 Advancements from LLMs to LLM Autonomous Agents

The evolution of LLMs has reshaped artificial intelligence and natural
language processing. From foundational embeddings like Word2Vec [16] and GloVe
[38], the field advanced with the introduction of sequential modeling like
Long Short-Term Memory (LSTM) [18] and early transformer models of
Bidirectional Encoder Representations from Transformers (BERT) [11]. Today,
the new-generation LLMs, like Generative Pre-trained Transformer series (GPTs)
[40, 34] and LLM Meta AI (Llamas) [48], stand out in diverse QA tasks. The
trend leans towards LLM agents. While LLM agents for domain-specific tasks
have been extensively researched [20, 25, 36], their application in financial
trading remains underexplored. Existing studies in this domain, such as [52,
55], often lack open-source availability or have not considered an
architecture specifically tailored to fit the unique environment of finance
markets. Thus, there’s significant value in further investigating advanced,
transparent LLM agents for trading.

###  2.3 Architecture Design of LLM Autonomous Agent

As Wang et al.[50] emphasizes, an effective architecture for LLMs serving
autonomous agents is essential. Typically, this structure comprises modules
like profiling, memory, planning, and actions, though not all may be essential
for every application. There are cases of two modules (e.g., planning and
action modules) being integrated as one component ([35]). The design
variations are numerous. For instance, profiling has been achieved through
methods like handcrafting [57], generation by LLMs [51], and alignment with
real-world datasets [2]. Among these modules, the memory component is
essential. Acting as the operational core, it aligns an agent’s actions with
real-world tasks. Research indicates that leveraging insights from cognitive
science studies on human memory [50, 45] can enhance this alignment. Thus, a
well-structured trading LLM agent, comprising aptly designed modules, can
sharply tackle the complexities of financial markets to make informed
decisions.

##  3 Architecture of FinMem

In this section, we comprehensively detail the three core components of
FinMem, namely the profiling, memory, and decision-making modules. The
profiling module empowers FinMem to adaptively tailor character setting for
specific trading tasks. Memory module leverages diverse time-efficiency
attributes of financial data, enhancing its trading efficacy. The decision-
making module enables FinMem to synchronize its memory streams with market
facts, facilitating high-quality trading decisions. The details and notations
associated with these three modules are provided in the subsequent sections.

Figure 1: The prompt template for FinMem’s profiling module. It includes two
key elements of its character setting: professional background knowledge and
three distinct investment risk inclinations. In the self-adaptive risk
inclination option, the omitted texts align with the detailed descriptions
provided for the risk-seeking and risk-averse inclinations.

###  3.1 Profiling Module

The profiling module empowers FinMem to develop a dynamic agent character
specifically designed to navigate the complex dynamics of financial markets
effectively.

The dynamic character of FinMem comprises two principal components, as
depicted in Figure 1: firstly, a foundational professional knowledge base akin
to a trading expert, and secondly, an agent with three distinct investment
risk inclinations. The first component includes two types of information: an
introduction to the primary trading sectors relevant to the company stock
FinMem will trade in, and a concise overview of the historical financial
performance of the specified ticker, spanning from the beginning to the end of
the training period. Before initiating trades in a new company’s stock, FinMem
accesses and updates this sector-specific and historical financial data from a
backend database. This professional background setting narrows down
information and memory events pertinent to specific trading tasks.

The second component of FinMem’s design, illustrated in Figure 1, encompasses
three distinct risk inclination options: risk-seeking, risk-averse, and a
self-adaptive risk character. The risk-seeking setting gears FinMem towards an
aggressive, high-reward approach, while the risk-averse setting gears it
towards a conservative, lower-risk strategy. A distinctive aspect of FinMem is
its ability to dynamically alternate between these risk settings in response
to current market conditions. Specifically, it shifts risk preferences when
the Cumulative Return falls to below zero within a brief period, such as three
days, and reversely. This flexible design functions as a protective mechanism,
mitigating prolonged downturns in turbulent market environments. During the
initial stage of the training phase, FinMem is configured with a chosen risk
preference, each supplemented with comprehensive textual explanations through
LLM prompts. These guidelines shape how FinMem processes incoming messages and
determines its subsequent actions in alignment with its designated risk
inclination. The system maintains a catalog of all risk inclinations and their
detailed explanations in a backlog, enabling seamless adaptation to different
stocks by switching among these risk profiles as needed.

The dynamic character setting in FinMe’s profiling module provides subjective
and professional background knowledge and flexible choice of risk
inclinations. It provides crucial context for filtering and retrieving
trading-relevant information and memory events, thus improving accurate
inferencing and adaptability to fluctuating market conditions.

###  3.2 Memory Module

The memory module of FinMem emulates a human trader’s cognitive system so that
it can efficiently process hierarchical financial information and prioritize
the critical messages for high-quality investment decisions. Furthermore, it
adjusts the memory span flexibly, enabling the agent to operate on a wider
range of events over a longer retrieval period. FinMem’s memory module,
illustrated in Figure 2, comprises working and long-term memory with layered
processing capability and is initiated by a specific investment inquiry.

Figure 2: Memory module structure of FinMem with a detailed view of
components, operations, and workflow. The cognitive architectures of FinMem’s
memory module have two core components – Working Memory and Layered Long-term
Memory.

####  3.2.1 Working memory

Working memory refers to the human cognitive system’s functions for temporary
storage and diverse operations. We incorporate this concept into FinMem’s
memory module development, creating a central workspace for informed decision-
making. Unlike human working memory, having a maximum capacity of seven plus
or minus two memory events [30], FinMem has the ability to expand the capacity
based on specific requirements. Tailored for converting financial data into
trading actions, FinMem’s working memory encompasses three key operations:
summarization, observation, and reflection. The mechanisms by which they
interact and operate as an integrated decision-making workflow are detailed in
the middle box of Figure 2. Additionally, the LLM prompt template that
underpins these processes is thoroughly outlined in 3.

Summarization: FinMem leverages external market data to derive critical
investment insights and sentiments tailored to specific stock trading queries,
such as “Can you make an investment decision on TSLA on 1/24/2023?”. As
illustrated in Figure 3 (2), this system condenses the original text into a
compact yet informative paragraph, thereby enhancing FinMem’s processing
efficiency. It efficiently extracts and summarizes pertinent data and
sentiments for stock investment decisions, demonstrated here using Tesla Inc.
as an example. Subsequently, FinMem directs these insights to an appropriate
layer within its long-term memory architecture, selecting the layer based on
the time sensitivity of the information.

Observation: Triggered the same inquiry, FinMem initiates an observation
operation to gather market facts. The information available to FinMem varies
between the training and testing phases.

During the training phase, FinMem has access to comprehensive stock price data
within the specified period. Upon receiving trading inquiries that specify a
stock ticker and date, FinMem focuses on the daily adjusted closing price
differences, comparing the following day’s price with the current day’s. These
price differences are utilized as market ground labels. Specifically, a
decrease in price suggests a “Sell” action, while an increase or no change in
price indicates a “Buy” action.

During the testing phase, at a specific time point, FinMem loses the ability
to access future price data. Its focus shifts to the analysis of historical
stock price movements, depending on a retrospective evaluation of the
cumulative return from the last $M$ trading days to infer future market
trends. This phase, characterized by the absence of foreseen market grounds,
serves as a critical assessment of FinMem’s development. It tests whether the
system has adequately established logical connections between stock price
trends and various financial information sources, such as news, reports, and
indicators. This stage is key in evaluating FinMem’s capability of
independently evolving its trading strategies for subsequent tasks, leveraging
its analysis and interpretation of historical data patterns.

Reflection: Two types of reflections exist, immediate and extended reflection.
(a) Immediate reflection is activated upon receiving a daily trading inquiry
for a specific ticker. Using LLM and specific prompts exemplified in Figure 3
(2), the agent merges market indications and top-$K$-ranked events from each
long-term memory layer. Market indications are derived from the outcomes of
the observation operation and differ between the training and testing phases.
During testing, this process yields three types of outputs: the trading
direction (“Buy”, “Sell”, or “Hold”), the underlying rationale for this
decision, and the most influential memory events, along with their IDs from
each layer that informed the decision. In the training phase, specifying the
trading direction is unnecessary, as FinMem is already informed of future
stock movement directions. The top-$K$-ranked memory events encapsulate key
insights and sentiments derived from critical investment-related incoming
messages, all distilled by FinMem’s advanced summarization capabilities.

(b) Extended reflection reevaluates immediate reflection outcomes for a ticker
over a specified $M$-day trace period. It encompasses data like stock price
trends, trading returns, and action rationales from multiple immediate
reflections. While immediate reflection enables direct trading execution and
records current feedback, extended reflection summarizes market trends and
reassesses recent Cumulative Return on investment. Extended reflection is
eventually transmitted and stored in the deep processing layer to emphasize
its criticality (detailed introduced in Section 3.2.2) of long-term memory.
$K$ and $M$ are hyperparameters to adjust FinMem’s working memory capacity and
information retrieval ability. FinMem gains the flexibility of integrating
comprehensive information into well-informed decisions by fine-tuning them.

####  3.2.2 Layered long-term memory

Figure 3: (1) The decision-making module workflow of the FinMem trading agent
retrieves critical memory events to inform specific decisions. (2) LLM prompt
template used by FinMem to interact with incoming financial information.

FinMem’s long-term memory organizes hierarchical financial data insights in a
stratified structure, as illustrated in the lower section of Figure 2. Drawing
inspiration from the varying decay speeds in the human cognitive system’s
information processing layers [9], FinMem employs a layered structure to
accommodate the diverse time sensitivities inherent to different types of
financial data. This structure categorizes summarized insights by their
timeliness and decay rates. Insights are derived by the working memory’s
summarization operation. Those directed to deeper layers receive smaller decay
rates, indicating longer retention, while those in shallower layers are
assigned larger decay rates for shorter retention.

$\gamma_{l}^{E}=S_{\text{Recency}_{l}}^{E}+S_{\text{Relevancy}_{l}}^{E}+S_{\text{Importance}_{l}}^{E},$
(1)

where each memory event is only associated with one score and can only belong
to a single layer.

Upon receiving an investment inquiry, FinMem retrieves the top-$K$ pivotal
memory events from each layer and channels them to the immediate reflection
component of the working memory. These events are chosen according to the
descending order of their information retrieval score, denoted as
$\gamma_{l}^{E}$, where $l$ belongs to the set shallow, intermediate, deep, as
specified in Equation 5. $E$ denotes a given memory event. This score, adapted
from Park et al. [35] but with modified recency and importance computations,
especially tailoring to handle data with various timelines. It encapsulates
three metrics: recency, relevancy, and importance. Individual metric scores
exceeding 1.0 are scaled to the [0,1] range before being summed. The
modification is to achieve the layered processing function and represent the
various periodicity of the financial environment.

$\begin{split}&S_{\text{Recency}_{l}}^{E}=e^{-\frac{\delta^{E}}{Q_{l}}},\quad\;\delta^{E}=t_{\text{P}}-t_{E},\\\
\end{split}$ (2)

where $\delta^{E}$ refers to the time difference between the memory event
occurrence and the trading inquiry arrival. $Q_{\text{shallow}}=14$,
$Q_{\text{intermediate}}=90$, and $Q_{\text{deep}}=365$ correspond to day
counts of two weeks, a quarter, and a year for shallow, intermediate, and deep
processing layers, respectively.

Upon a trade inquiry $P$ arrival in processing layer $l$ via LLM prompt, the
agent computes the recency score $S_{\text{Recency}_{l}}^{E}$ per Equation.2.
$S_{\text{Recency}_{l}}^{E}$ inversely correlates with the time gap between
the inquiry and the event’s memory timestamp, mirroring Ebbinghaus’s
forgetting curve [33]. The stability term $Q_{l}$ in Equation.2 partially
controls memory decay rates across layers, indicating longer memory
persistence in the long-term layer with a higher stability value. In the
context of trading, company annual reports, such as Form 10-Ks, are considered
to have more extended timeliness compared to daily financial news. Therefore,
they are assigned a higher stability value and are categorized within the
deeper processing layer. This classification reflects their extended relevance
and impact in financial decision-making scenarios.

$\begin{split}&S_{\text{Relevancy}_{l}}^{E}=\frac{\mathbf{m_{E}}\cdot\mathbf{m_{P}}}{\|\mathbf{m_{E}}\|_{2}\times\|\mathbf{m_{P}}\|_{2}}\end{split}$
(3)

The relevancy score, denoted as $S_{\text{relevancy}_{l}}^{E}$, quantifies the
cosine similarity between the embedding vectors. These vectors are derived
from the textual content of the memory event, $\mathbf{m_{E}}$, and the LLM
prompt query, $\mathbf{m_{P}}$, using OpenAI’s “text-embedding-ada-002” model,
as depicted in Equation 3. The LLM prompt query incorporates inputs related to
trading inquiries and the trading agent’s character setting.

The importance score $S_{\text{Importance}_{l}}^{E}$ is computed using the
value $v_{l}^{E}$ from a uniform piecewise scoring function (Formula 4),
multiplied by a degrading ratio $\theta_{l}$ (Formula 5) as per Equation 6.
The likelihood of higher $v_{l}^{E}$ values increases from shallow to deep
layers. $\theta_{l}$ measures the diminishing importance of an event over
time, which has a close form design of [35]. But our approach tailors
$\theta_{l}$ to the stratified structure of long-term memory. It adopts unique
exponential functions for each layer. The base $\alpha_{l}$ for each layer is
a hyperparameter, set to follow the sequence:
$\alpha_{shallow}<\alpha_{intermediate}<\alpha_{deep}$. These values correlate
with the rate at which their importance degrades after a certain period,
providing another angle to measure importance variances across different
memory types. Through experimentation, we set $\alpha_{shallow}=0.9$,
$\alpha_{intermediate}=0.967$ and $\alpha_{deep}=0.988$. This ensures that
$\theta_{l}$ decreases to a threshold score of $5$ after intervals of $30,90,$
and $365$ days for shallow, intermediate, and deep layers, respectively. The
three-piece-wise functions for $S_{\text{Importance}_{l}}^{E}$ and
$S_{\text{Recency}_{l}}^{E}$ enable FinMem to have layered processing in the
long-term memory component. Memory events are purged when
$S_{\text{Recency}_{l}}^{E}$ is below $0.05$ or
$S_{\text{Importance}_{l}}^{E}$ is under $5$ (pre-scaling).

$\begin{split}&v_{l}^{E}=\begin{cases}40&\text{with probability }p_{1}\\\
60&\text{with probability }p_{2}\\\ 80&\text{with probability
}p_{3}\end{cases}\end{split}$ (4)

$\begin{split}\theta_{l}=(\alpha_{l})^{\delta^{E}},\quad&l=\text{shallow},\text{intermediate},\text{deep},\end{split}$
(5)

where $p_{1}+p_{2}+p_{3}=1$, but their values vary by shallow, intermediate,
and deep processing. when shallow processing
${p_{1},p_{2},p_{3}}=\\{0.8,0.15,0.05\\}$, intermediate processing,
${p_{1},p_{2},p_{3}}=\\{0.05,0.8,0.15\\}$ and deep processing,
${p_{1},p_{2},p_{3}}=\\{0.05,0.15,0.8\\}$.

$S_{\text{Importance}_{l}}^{E}=v_{l}^{E}*\theta_{l},$ (6)

Furthermore, an access counter function oversees the transfer of memory events
among layers, ensuring that significant events influencing trading decisions
ascend from shallower to deeper layers for extended retention and recurrent
access by FinMem. Conversely, less pertinent events gradually diminish. This
process is facilitated by the LLM validation tool Guardrails AI [17], which
monitors critical memory IDs across different layers. An event identified as
pivotal for investment success receives an additional $5$ points in its
importance score $S_{\text{Importance}_{l}}^{E}$. Upon meeting the criteria
for upgrading to a deeper layer, an event’s recency score
$S_{\text{Recency}_{l}}^{E}$ is reset to $1.0$, emphasizing its importance and
preventing rapid decay. By implementing this access counter, FinMem
effectively identifies and prioritizes key events, taking into account their
nature and frequency of retrieval.

###  3.3 Decision-making Module

The decision-making module of FinMem efficiently integrates operational
outcomes from the profiling and memory modules to support well-informed
investment decisions, as depicted in Figure 3 (1). In its daily trading
decisions, FinMem is asked to select from three distinct actions for a single
share of a specific stock by Guardrails AI text validation function: “Buy”,
“Sell”, or “Hold”. Additionally, the inputs and results required by FinMem’s
decision-making module vary between its training and testing phases, with each
phase’s specifics detailed as follows:

During the training phase, FinMem accesses a wide array of multi-source
information relevant to the entire time period. When FinMem is prompted with
trading inquiries containing stock ticker and date, as well as trader
character-related texts, it concurrently initiates observation and
summarization operations in its working memory. FinMem observes the market
ground labels mentioned in the description about the observation operation in
Section 3.2.1, which involve daily adjusted price differences between
consecutive days, indicative of “Buy” or “Sell” actions. Utilizing these price
change signals, FinMem identifies and prioritizes the top-$K$ memories,
ranking them based on retrieval scores from each long-term memory layer. This
procedure enables FinMem to produce comprehensive reflections that provide a
well-founded rationale and in-depth inference of the correlation between
market ground labels and the memories retrieved. Through repeated trading
operations, reflections, and memory events with significant impact, transition
to a deeper memory processing layer, getting preserved for guiding future
investment decisions during the testing phase.

In the testing phase, where FinMem cannot access future price data, it relies
on the Cumulative Return over the previous $M$ trading days to anticipate
future market trends. To compensate for the absence of future market price
information, FinMem utilizes enhanced reflections derived from immediate
reflections spanning an $M$-trading-day period as supplementary references.
When faced with a specific trading inquiry, FinMem integrates insights from
various sources, including historical Cumulative Return, outcomes from
extended reflection, and the Top-$K$ retrieved memories. This comprehensive
approach enables FinMem to execute well-informed trading decisions.

It should be noted that FinMem generates executable actions exclusively in the
immediate reflection operation of the testing phase. Since the trading
direction is guided by the actual price trend, the training phase of FinMem
does not make investment decisions. Instead, this phase is dedicated to
accumulating trading experience through comparing market trends with incoming
multi-source financial messages. Additionally, during this phase, FinMem
develops a memory module enriched with a comprehensive knowledge base, thereby
evolving its capability for independent decision-making in future trading
activities.

##  4 Experiments Setups

We aim to evaluate the trading performance of FinMem. And we further
illustrate its unique advantages of requiring significantly less historical
trading time window to train and take full use of key financial data time
series as well as textual information. Specifically, we conducted several
experiments to study the following research questions (RQs):

  * •

RQ1: Is FinMem capable of outperforming contemporary state-of-the-art
algorithmic trading agents?

  * •

RQ2: Are there tasks that challenge other trading algorithms but are
manageable by FinMem?

  * •

RQ3: Which LLM is best suited to form the backbone framework of FinMem?

  * •

RQ4: Does equipping FinMem with different risk inclination choices truly
differentiate its trading performance?

  * •

RQ5: Can FinMem effectively filter and prioritize information to facilitate
informed trading decisions?

In the rest of the section, we begin by introducing the real-world financial
dataset used in our experiments. We then describe the comparative algorithmic
agents and list several widely used financial metrics. Our experiments fall
into two categories: 1) The comparative experiments of FinMem versus other
algorithmic trading agents, and FinMem using different LLMs as backbone
algorithms. 2) The ablation studies evaluate the effects of FinMem’s
adjustable cognitive span and the role of the trader’s dynamic character
settings, particularly the risk inclinations, on its trading performance.
Through experiments, FinMem demonstrates to outperform other comparative
algorithmic agents. Furthermore, we are able to show that its profiling and
memory modules are sophisticated and tailored to effectively address the
intricacies of the financial landscape, resulting in superior trading
performance.

###  4.1 Datasets And Database Structure:

We assessed FinMem’s performance using multi-source financial data from August
15, 2021, to April 25, 2023, sourced from reputable financial databases and
APIs like Yahoo Finance (via yfinance) and Alpaca News API, detailed explained
in Table 1. The stock tickers used in our comparative experiments are detailed
in Figure 4. These were selected because they are among those with the highest
volumes of accessible news text data, and they are spread across various
trading sectors. This selection provides ample data to evaluate FinMem’s
generalization capabilities. Additionally, Tesla, Inc. (TSLA) was specifically
chosen for ablation studies due to its association with the largest amount of
textual data, offering sufficient information to assess performance
differences for the FinMem’s key features like cognitive spans.

The raw multi-source input data, initially stored in the “Raw Financial Data
Warehouse”, are diverged into FinMem’s “Layered Long-term Memory Data
Warehouse” based on timeliness through working memory’s summarization
operation in Figure 2. The deep processing layer holds annual reports (Form
10K’s) insights, the intermediate layer contains quarterly reports (Form
10Q’s) insights, and the shallow layer accommodates daily financial news
insights.

We leveraged the open-source vector database FAISS [23] for constructing the
memory warehouse of FinMem, benefiting from its rapid querying in high-
dimensional vectors and compatibility with OpenAI for cosine-similarity-based
semantic searches on specific tickers. This setup facilitates efficient top-
ranked event retrieval. Data categorization and memory module workflow are
also illustrated in Figure 2

Raw Financial Data Warehouse
---
News data associated with ticker indexes: News data is sourced from the Alpaca
News API, which utilizes Benzinga as its backend provider.
Corporate quarter filings indexes: Quarterly reports (Form 10-Q) are required
by the U.S. Securities and Exchange Commission (SEC).
Corporate annual filings indexes: Annual reports (Form 10-K) required by the
U.S. Securities and Exchange Commission (SEC).
Stock price records: Daily stock open-high-close-volume (OHLCV) data from
Yahoo Finance.
FinMem’s Layered Long-term Memory Data Warehouse
Shallow Processing: Insights of real-time market news extracted by LLM.
Updated daily.
Intermediate Processing: Insights of 10-Q filings extracted by LLM. Updated
quarterly.
Deep Processing: 10-K fillings, all summarized by LLM. Updated yearly.
Extended reflections for the stock in response to the trading inquiry include
FinMem’s cumulative trading returns, decision-making processes, trade volumes,
and underlying reasons. Updated daily.
Table 1: Raw data and memory warehouses of FinMem Figure 4: The distribution
of news in scraped from Alpaca News API for the five stocks in the experiments

###  4.2 Baseline And Comparative models:

We assess FinMem’s trading performance in comparison to five advanced
algorithmic agents and a commonly accepted baseline trading strategy. Among
these, three models employ Deep Reinforcement Learning (DRL) approaches, while
the remaining two are based on Large Language LLMs. Brief descriptions of each
are provided below:

Buy-and-Hold strategy (B&H):

A passive investment approach, where an investor purchases stocks and holds
onto them for an extended period regardless of market fluctuations, is
commonly used as a baseline for comparison of stock trading strategies.

DRL trading agents:

As the FinMem is practiced and examined on the basis of single stock trading
and discrete trading actions, we choose three advanced DRL algorithms fitting
into the same scenarios according to the previous and shown expressive
performance in the work of Liu et al. [28, 26]. The DRL training agents only
take numeric features as inputs.

  * •

Proximal Policy Optimization (PPO): PPO [42] is employed in stock trading due
to its stability and efficiency. One salient advantage of PPO is that it
maintains a balance between exploration and exploitation by bounding the
policy update, preventing drastic policy changes.

  * •

Deep Q-Network (DQN): DQN [32] is an adaptation of Q-learning, that can be
used to optimize investment strategies. Unlike traditional Q-learning that
relies on a tabular approach for storing Q-values, DQN generalizes Q-value
estimation across states using deep learning, making it more scalable for
complex trading environments.

  * •

Advantage Actor-Critic (A2C): A2C [31] is applied to optimize trading actions
in the financial environment. It operates by simultaneously updating both the
policy (actor) and the value (critic) functions, providing a balance between
exploration and exploitation.

LLM trading agents:

We evaluate FinMem against two LLM agents in the context of stock trading. The
first LLM agent, known for its proficiency in general-purpose tasks, serves as
a baseline. The second agent, a leading-edge LLM in trading, has been
acclaimed for its promising performance in stock market operations.

  * •

General-purpose Generative Agents – GA: The generative AI agent by Park et al.
[36], originally intended to simulate realistic human behavior and make
everyday decisions, has been adapted here for specific stock trading tasks.
This agent’s architecture includes a memory module that employs recency,
relevance, and importance metrics to extract pivotal memory events for
informed decision-making. However, it does not provide a layered memory module
to effectively differentiate the time sensitivities unique to various types of
financial data. Additionally, although it features a profiling module to
define agent attributes like professional background, the model does not
include a mechanism for self-adaptive risk preference. In our experiments, we
modified the original prompt template created by Park et al., which was
intended for general daily tasks, to suit financial investment tasks. The
textual elements of this revised template closely align with those of FinMem,
with the exception of two components that are absent in this version of
general-purpose Generative Agents.

  * •

LLM trading agents – FinGPT: A novel open-source LLM framework specialized for
converting incoming textual and numeric information into informed financial
decision-making, introduced by Yang et al. [55]. It claims superiority over
the traditional buy-and-hold strategy.

###  4.3 Evaluation Metrics:

Figure 5: Cumulative return comparison over time between FinMem and other
algorithmic agents across five stocks. Figure 6: Cumulative Return comparison
over time between FinMem and other algorithmic agents on Coinbase Global, Inc.
(COIN).

We employ five widely-used metrics in finance to compare the investment
rewards of FinMem against other algorithmic trading agents. Here are their
introductions:

  * •

Cumulative Return [21]: Cumulative Return is a key trading performance metric
because it provides a comprehensive insight into investment performance,
especially for strategies that emphasize long-term growth and reinvestment.
The effectiveness of different investment strategies is evaluated based on
their Cumulative Returns, which reflect the total change in value over time.
In this study, we compute Cumulative Returns over the specified period by
summing daily logarithmic returns, as outlined in Equation 7. This method is
widely accepted in the finance area due to its ability to precisely capture
minor price fluctuations and symmetrically address gains and losses. In
essence, a higher Cumulative Return typically indicates a more effective
strategy.

Cumulative Return $\displaystyle=\sum_{t=1}^{n}r_{i}$
$\displaystyle=\sum_{t=1}^{n}\left[\ln\left(\frac{p_{t+1}}{p_{t}}\right)\cdot\text{action}_{t}\right],$
(7)

where $r_{i}$ represents the logarithmic return for day $t+1$, $p_{t}$ is the
closing price on day $t$, $p_{t+1}$ is the closing price on day $t+1$, and
$\text{action}_{t}$ denotes the trading decision made by the model for that
day.

  * •

Sharpe Ratio [43]: Sharpe Ratio is another core metric for evaluating
investment performance and adjusting returns for risk. It is calculated by
dividing the portfolio’s average excess return ($R_{p}$) over the risk-free
rate ($R_{f}$) by its volatility ($\sigma_{p}$), as shown in Equation 8. This
metric adjusts returns for risk, with a higher ratio indicating better risk-
adjusted performance. Essential in comparing different portfolios or
strategies, it contextualizes performance against similar investments.
Although a Sharpe Ratio above 1 is typically considered favorable and above 2
as excellent, these benchmarks can vary depending on the context of
comparison.

$\textbf{Sharpe Ratio}=\frac{R_{p}-R_{f}}{\sigma_{p}}$ (8)

  * •

Annualized Volatility and Daily Volatility[8]: Annualized Volatility (Annum-
Volatility) is calculated as the Daily Volatility (standard deviation of daily
logarithmic returns) multiplied by the square root of the typical number of
trading days in a year (252) as outlined in Equation 9, is vital for assessing
investment risk. This measure reflects the extent of fluctuation in a security
or market index’s returns over a year, indicating potential deviations from
average returns. It’s especially relevant for investors with specific risk
profiles, such as those who are risk-averse, who may prefer portfolios
demonstrating lower annualized volatility.

Annum-Volatility $\displaystyle=\textbf{Daily Volatility}\times\sqrt{252}$ (9)

  * •

Max Drawdown [1]: Max Drawdown is a metric for assessing risk. It represents
the most significant decrease in a portfolio’s value, from its highest
($P_{\text{peak}}$) to its lowest point ($P_{\text{trough}}$) until a new peak
emerges, detailed in Equation 10. Indicative of investment strategy
robustness, a smaller Max Drawdown suggests reduced risk.

$\displaystyle\textbf{Max
Drawdown}=\text{max}(\frac{P_{\text{peak}}-P_{\text{trough}}}{P_{\text{peak}}})$
(10)

In our experiments and ablation studies, we recorded the metric outcomes as an
average from five repeated trials.

Ticker Model Cumulative Return (%) Sharpe Ratio Daily Volatility (%)
Annualized Volatility (%) Max Drawdown (%) TSLA Buy and Hold -18.6312 -0.5410
4.4084 69.9818 55.3208 FinMem 61.7758* 2.6789 2.9522 46.8649 10.7996
Generative Agents 13.4636 0.5990 2.8774 45.6774 24.3177 FinGPT -7.4554 -0.2795
3.4145 54.2027 42.3993 A2C 13.7067 0.3979 4.4096 70.0009 52.3308 PPO 1.2877
0.0374 4.4110 70.0232 54.3264 DQN 33.3393 0.9694 4.4027 69.8900 52.0033 NFLX
Buy and Hold 35.5111 1.4109 3.1964 50.7410 20.9263 FinMem 36.4485* 2.0168
2.2951 36.4342 15.8495 Generative Agents 32.0058 1.5965 2.5460 40.4168 16.9893
FinGPT 9.0090 0.4266 2.6819 42.5732 28.2705 A2C 14.6155 0.5788 3.2071 50.9112
25.0184 PPO 8.4121 0.3330 3.2086 50.9344 25.0184 DQN -12.2067 -0.4833 3.2078
50.9217 28.7017 AMZN Buy and Hold -10.7739 -0.4980 2.7697 43.9674 33.6828
FinMem 4.8850* 0.2327 2.6872 42.6576 22.9294 Generative Agents -13.9271
-0.9981 1.7864 28.3576 27.7334 FinGPT -29.6781 -2.1756 1.7464 27.7225 28.4838
A2C -6.3591 -0.2938 2.7706 43.9819 26.1275 PPO -8.4194 -0.3891 2.7702 43.9761
33.6828 DQN -29.9820 -1.3906 2.7603 43.8177 38.3740 MSFT Buy and Hold 14.6949
0.8359 2.2326 35.4411 15.0097 FinMem 23.2613* 1.4402 2.0512 32.5617 14.9889
Generative Agents -18.1031 -1.6057 1.4318 22.7285 24.2074 FinGPT 5.7356 0.4430
1.6442 26.1008 12.8459 A2C 0.4598 0.0261 2.2357 35.4913 23.6781 PPO 12.8067
0.7282 2.2333 35.4532 19.5355 DQN 14.7397 0.8385 2.2326 35.4408 25.1845 COIN
Buy and Hold -30.0071 -0.5150 6.7517 107.1795 60.5084 FinMem 34.9832* 0.7170
5.6538 89.7515 35.7526 Generative Agents 3.4627 0.0896 4.4783 71.0908 32.0957
FinGPT -88.7805 -1.9507 5.2736 83.7153 73.5774 A2C - - - - - PPO - - - - - DQN
- - - - -

Table 2: Overall trading performance comparison during testing period between
FinMem and other algorithmic agents across five stocks.* indicates that the
result of the Wilcoxon signed-rank test is statistically significant.333The
bold numbers in this and subsequent tables signify the best performance for
the respective metrics. Figure 7: Cumulative Return of FinMem on trading
Tesla, Inc. (TSLA) stock Over an Extended Testing Period.

##  5 Experiments:

###  5.1 Implementation Details:

In the Trading Agents Comparison, FinMem employs GPT-4-Turbo as its backbone
algorithm. The temperature parameter of the model was set at 0.7 to maintain a
balance between response content consistency and model creativity. It was
trained on financial data from August 17, 2021, to October 05, 2022, and
underwent testing with data from October 06, 2022, to April 10, 2023. The
training period was chosen to account for the seasonal nature of corporate
financial reporting and the duration of data retention in FinMem’s memory
module. The selected training duration ensures the inclusion of at least one
publication cycle of either Form 10-Q, classified as intermediate memory, or
Form 10-K, regarded as deep memory, or in some instances, both. This strategy
ensures that the experiences retained in FinMem are still influential during
the testing phase for a significant period. Additionally, the training
duration allowed FinMem sufficient time to establish inferential links between
financial news, market indicators, and stock market trends, thereby
accumulating substantial experience. Furthermore, we set the number of top
memory events retrieved from each layer of long-term memory at 5. We ran
FinMem using each of the three available risk inclination settings. The
reported performance outcomes are based on the setting that achieved the
highest cumulative return during the testing phase.

To maintain consistency in the comparison, the training and testing phases for
the other two LLM-based agents were aligned with those of FinMem. For
parameters of other LLM-based agents that are not encompassed by FinMem’s
configuration, they were kept in accordance with their original settings as
specified in their respective source codes.

Considering that DRL algorithms need extensive training data for stable and
converged results, and given our daily evaluation of trading performance, we
extended the DRL agents’ training period to roughly a 10-year span, from
January 1, 2012, to October 05, 2022, for a fair comparison. The testing
period was kept consistent with the other models. The DRL algorithms were
implemented using Stable Baselines 3 [41].

FinMem’s performance was benchmarked against that of the most effective
comparative model, using Cumulative Return and Sharpe Ratio as the primary
evaluation metrics. The statistical significance of FinMem’s superior
performance was ascertained through the non-parametric Wilcoxon signed-rank
test, which is particularly apt for the non-Gaussian distributed data.

###  5.2 Algorithmic Trading Agents Comparison (RQ1 & RQ2)

In this experiment, we assess the stock trading performance of FinMem against
other models, focusing on stocks from five companies in different trading
sectors: Tesla, Inc. (TSLA), Netflix, Inc. (NFLX), Amazon.com, Inc. (AMZN),
Microsoft Corporation (MSFT), and Coinbase Global, Inc. (COIN). The
performance of all algorithmic trading agents across five key metrics is
consolidated in Table 3. Given the pivotal role of Cumulative Return in
evaluating trading performance over time, we present detailed time series
plots in Figure 5 and Figure 6. It’s important to note that the trading
performance of FinMem for COIN was exclusively compared with LLM trading
agents and the baseline. This is because Coinbase Global, Inc. completed its
IPO in April 2021 and, as a result, had not accumulated enough trading data to
facilitate stable outcomes with Deep Reinforcement Learning (DRL) algorithms.
These plots illustrate the changes in Cumulative Return for each of the five
companies throughout the testing phase, offering an in-depth comparison of
performance.

In response to RQ1, the trading outcomes presented in Table 3 reveal that
FinMem outperforms all other algorithmic trading agents and the B&H baseline
strategy in terms of Cumulative Return and Sharpe Ratio. FinMem’s superiority
is statistically significant when compared to the second-best trading
strategy. Specifically, for TSLA and NFLX, FinMem’s strategy achieves Sharpe
Ratios exceeding $2.0$ and Cumulative Returns surpassing $0.35$ while
maintaining the lowest Volatility and Max Drawdown. These indicators
underscore FinMem’s ability to generate higher returns per unit of risk. In
the case of MSFT and NFLX, FinMem also records a Sharpe Ratio above $1.0$ and
a Cumulative Return over $0.2$, coupled with relatively low Volatility and Max
Drawdown, demonstrating its impressive trading performance. For AMZN and COIN,
FinMem consistently delivers positive Cumulative Returns and superior Sharpe
Ratios, outperforming other strategies that yield negative values for these
metrics. Additionally, its Volatility and Max Drawdown are on the lower end.
Hence, these results collectively demonstrate FinMem’s robust trading
performance across a diverse range of trading sectors. Specifically, FinMem
exhibits superior performance compared to the two other LLM agents in our
study, FinGPT and the general-purpose generative agent developed by Park et
al. This underscores the effectiveness of FinMem’s unique profiling and memory
structure, which are particularly tailored for LLM agents dealing with
financial data, significantly enhancing their investment decision-making
capabilities.

Figure 8: The optimal risk inclination for FinMem when trading different
stocks.

In response to RQ2, the main challenge for DRL trading agents is that they
require training data with a large volume and extended time span, which are
hard to achieve when operating on stocks with limited historical data. As
shown in Table 3, our experiments reveal that FinMem achieves superior trading
performance with a much shorter training duration compared to DRL trading
agents trained on data spanning nearly a decade. This efficiency makes FinMem
particularly useful for newly public companies like Coinbase Global, Inc.,
which have limited trading histories. DRL agents often face convergence issues
due to inadequate training data in such cases. Moreover, even among LLM-based
trading agents suited for shorter training periods, FinMem’s performance
stands out, as illustrated in Figure 6.

To further assess FinMem’s adaptability to limited training data, we narrowed
the training period down to an even shorter period, spanning from August 17,
2021, to February 10, 2022. We then extended the testing phase to cover from
February 11, 2022, to April 25, 2023. This evaluation focused on the stock of
Tesla, Inc., which has the largest volume of news data. The trading
performance of FinMem during this period is depicted in Figure 7. Remarkably,
using less than six months of daily frequency data for training, which
encompassed the publication of one Form 10-K and one Form 10-Q, FinMem
consistently ranked high in gains and attained the highest cumulative return
after the latter half of December 2022.

The consistently strong trading performance of FinMem can be attributed to its
innovative profiling and memory module design. This design enables FinMem to
effectively integrate, comprehend, and prioritize key information from both
textual and numerical data. The flexibility of FinMem’s profiling module in
selecting risk inclinations plays a pivotal role in its ability to both
exploit rising market trends and safeguard assets during downturns. A prime
example is TSLA, which achieved its best trading results under a self-adaptive
risk setting in FinMem. This configuration enables FinMem to pursue a
conservative and cautious strategy when facing negative short-term cumulative
returns. On the other hand, with positive short-term returns, FinMem switches
to an optimistic and assertive approach, thus avoiding excessive passivity.
This self-adaptive risk inclination proved effective for most stocks, apart
from MSFT as shown in Table 8. For MSFT, a risk-seeking inclination was most
beneficial, resonating with its general bullish in the stock market.
Additionally, the memory module’s core features, including varied retention
times for different information types and critical memory transitions, equip
FinMem to capture essential information for well-informed investment
decisions.

##  6 Ablation Studies

We conducted three distinct ablation studies to evaluate key component
alternatives in FinMem. These studies concentrated on the backbone algorithm,
the memory module’s cognitive capacity, and the character setting in the
profiling module, particularly examining the aspect of risk inclination. These
studies were done using the stock of Tesla, Inc., with a more compact training
period from March 14, 2022, to June 15, 2022, and a testing period from June
16, 2022, to December 28, 2022. This shorter duration was chosen for budgetary
efficiency, yet it remains sufficient to differentiate the functionality of
each component.

###  6.1 FinMem Backbone Algorithm Comparison (RQ3)

Metric B&H GPT 3.5-Turbo GPT4 GPT4-Turbo davinci-003 Llama2-70b-chat
Cumulative Return (%) -66.9497 16.1501 62.6180 54.6958 1.6308 -52.7233 Sharpe
Ratio -2.0845 2.1589 2.2251 2.4960 0.8515 -2.8532 Daily Volatility (%) 3.8050
0.8862 3.3339 2.5960 0.2269 2.1891 Annualized Volatility (%) 60.4020 14.0683
52.9237 41.2100 3.6018 34.7503 Max Drawdown (%) 67.3269 1.1073 17.4012 12.5734
0.8408 44.7168

Table 3: Comparison of trading performance during the testing period for
FinMem using different LLMs as backbone algorithms.

In our first study, we evaluated the trading performance of FinMem using
various LLMs as its backbone algorithms. The LLMs under consideration included
davinci-003, GPT 3.5-Turbo, GPT4, GPT4-Turbo, and Llama2-70b-chat. The
parameter settings were consistent with its optimal performance in the
comparative experiment detailed in Section 5, and the risk inclination was
configured to be self-adaptive. All other model settings were maintained as
outlined in Section 5.1. The results of this evaluation are compiled in Table
3.

In response to RQ3, Addressing Research Question 3, the findings demonstrate
that FinMem, powered by GPT-4 and GPT-4 Turbo, delivered superior trading
results during the test phase. Specifically, GPT-4 recorded the highest
cumulative return, while GPT-4-Turbo exhibited the most favorable Sharpe
Ratio. GPT 3.5-Turbo’s performance was also noteworthy, following closely
behind. As depicted in Figure 9, though slightly lower than market baseline
(B&H), FinMem with GPT-4-Turbo led in cumulative returns before October 2022.
This period was characterized by relative stability and a modest upward trend
in TSLA stock. After October 2022, with TSLA undergoing increased volatility
and a notable downward trend, the cumulative return trajectory for FinMem with
GPT-4-Turbo exhibited significantly lower volatility and sustained stable
returns not markedly lower than those of GPT-4. These results indicate that
GPT-4 Turbo is the most suitable backbone algorithm for FinMem.

FinMem configured with davinci-003 and Llama2-70b-chat exhibited the lowest
Annualized Volatility and Max Drawdown, yet their Cumulative Return and Sharpe
Ratio were underwhelming. As illustrated in Figure 9, both models defaulted to
a “Hold” strategy beyond a certain point during periods of intense fluctuation
in TSLA stock. The unsatisfactory performance of davinci-003 may be attributed
to its limited capability, as an earlier generation language model, to capture
and understand nuanced yet decisive information.

We selected Llama2-70b-chat as it was deemed to possess stronger in-context
learning and instruction-following capabilities compared to other Llama family
models with fewer parameters, as noted in Zhao et al. [58]. Nonetheless, in
the context of stock trading, it still demonstrated challenges in adequately
comprehending key messages necessary for effective trading decisions. The
comparatively poorer performance of Llama2-70b-chat can also be attributed to
its shorter context window, especially when compared to the GPT models. When
integrated with FinMem, it needs to simplify prompts and shorten the length of
retrieved memory insights, which could potentially result in some loss of
context. The exceptional trading result demonstrated by GPT-4-Turbo across all
models was a main factor in choosing it as the backbone algorithm for FinMem
in our earlier comparative analysis with other algorithmic trading agents.

Figure 9: Comparison of overall Cumulative Returns over time for FinMem using
different LLMs as backbone algorithms.

###  6.2 Influence about varying the FinMem character design (RQ4)

In our second study, we focused on evaluating the influence of FinMem’s
profiling module on its trading effectiveness. Specifically, our assessment
centered on the effects of customizing trader profiles according to specific
stock trading, with a particular focus on risk inclination. As depicted in
Figure 8, we equipped FinMem with three distinct risk profiles: risk-seeking,
risk-averse, and a self-adaptive character. We executed a comparative analysis
of FinMem’s performance across these risk profiles, maintaining consistency in
all other settings as outlined in Section 5.1.

In response to RQ4, Table 4 delineates the varied trading performance across
different risk profiles. The self-adaptive profile enabled FinMem to achieve
the most favorable trading performance, as it was the only one to secure a
positive Cumulative Return and a Sharpe Ratio exceeding 2.0, along with the
least Max Drawdown by the end of the testing phase. Figure 10 illustrates
FinMem’s capacity to adeptly navigate substantial stock price volatility and
to strategically modulate its trading behavior when necessary. In contrast,
the risk-seeking profile, while beneficial during a stable or bullish market
as evidenced by MSFT’s performance in Figure 5, exhibited increased volatility
and a decline in the face of a market downturn. The risk-averse profile, on
the other hand, maintained a more conservative stance, often opting to hold
positions. This approach resulted in a Cumulative Return trajectory that
generally lagged behind the market baseline, reflecting a degree of
overcaution that limited trading activity and potential gains, particularly in
a bullish market.

Metric | B&H | Self Adaptive | Risk Seeking | Risk Averse
---|---|---|---|---
Cumulative Return (%) | -66.9497 | 54.6958 | -19.4132 | -12.4679
Sharpe Ratio | -2.0845 | 2.4960 | -0.7866 | -1.5783
Daily Volatility (%) | 3.9527 | 2.7419 | 3.2722 | 1.7744
Annualized Volatility (%) | 3.8050 | 2.5960 | 2.9236 | 0.9358
Max-Drawdown (%) | 67.3269 | 12.5734 | 45.0001 | 15.9882
Table 4: Comparison of overall trading performance during the testing period
with different risk inclinations setting in FinMem’s profiling module. Figure
10: Comparison of Cumulative Return during with different risk inclinations
setting in FinMem’s profiling module.

###  6.3 Impact of adjusting the capacity of FinMem working memory (RQ5)

In our third study, we explored whether appropriately tuning the memory
retrieval bandwidth of FinMem could enhance its trading performance. This
bandwidth is tied to the working memory’s capacity within its memory module.
As depicted in Figure 2, FinMem retrieves the top-$K$ memory events from its
long-term memory in response to a trading inquiry. The working memory capacity
is thus set at $3\times K$, mirroring the human cognitive system’s limit of
processing immediate messages upon specific stimuli ($3$ refers to the three
processing layers in long-term memory). By varying the $K$ hyperparameter,
FinMem can expand this capacity far beyond the human cognitive scope. We aimed
to determine whether such flexibility in adjusting memory bandwidth translates
to improvements in FinMem’s performance.

Metric | B&H | Top 1 | Top 3 | Top 5 | Top 10
---|---|---|---|---|---
Cumulative Return (%) | -66.9497 | 52.0936 | 29.4430 | 54.6958 | 79.4448
Sharpe Ratio | -2.0845 | 1.8642 | 1.1214 | 2.4960 | 2.7469
Daily Volatility (%) | 3.8050 | 3.3105 | 3.1105 | 2.5960 | 3.4262
Annualized Volatility (%) | 60.4020 | 52.5529 | 49.3779 | 41.2100 | 54.3891
Max Drawdown (%) | 67.3269 | 25.2355 | 27.0972 | 12.5734 | 17.1360
Table 5: Comparison of overall trading performance during the testing period
with different configurations of working memory capacity.

As demonstrated in Table 5, we adjusted the hyperparameter $K$ to alter the
number of memory events retrieved from shallow, intermediate, and deep long-
term memory layers in FinMem. We tested $K$ values of 1, 3, 5, and 10,
exploring FinMem’s working memory capabilities at levels below, near, and
above the human cognitive limit. For all these $K$ settings, we maintained a
self-adaptive risk inclination, while other settings were consistent with
those described in Section 5.1.

Across all $K$ configurations, FinMem outperformed the Buy & Hold baseline,
indicating the effectiveness of its memory module in processing diverse
information and capturing critical events, which subsequently enhanced its
trading performance, as evidenced by positive Cumulative Returns and Sharpe
Ratios. Notably, higher $K$ values, like 5 and 10, enabled FinMem to achieve
the best Cumulative Returns and Sharpe Ratios exceeding 2.0. With $K$ set to
1, FinMem still performed moderately well by capturing the most critical
memory events of each layer.

An in-depth analysis in Figure 11, which shows the Cumulative Return over time
for various $K$ settings, reveals that a $K$ value of 5 is optimal for trading
TSLA stock, consistently delivering robust performance with the lowest
Volatility and Max-Drawdown. Before mid-October 2022, when the stock market
was relatively stable and slightly upward, FinMem’s trading actions aligned
well with market trends (referring to B&H) and avoided significant losses.
During periods of high volatility and continuous downturns (post-mid-October
2022), it maintained earnings by reducing “Buy” actions and favoring more
“Hold” and “Sell” strategies. However, setting $K$ to 10, while effective
during market volatility, resulted in significant losses in stable market
conditions. The issue may stem from the disproportionately loose capacity
constraints on incoming information relative to the volume of incoming data.
The broad memory retrieval bandwidth might have mixed trivial messages with
critical ones, hampering FinMem’s decision precision. This challenge becomes
especially evident in neutral market conditions, where the influx of
information includes a mix of varying market sentiments and trends.

Addressing RQ5, appropriately tuning the number of memory events (Top-$K$ in
the FinMem memory module can significantly enhance its trading performance.
The aforementioned study illustrates that FinMem can achieve optimal results
by effectively assimilating key signals from a sufficient quantity of filtered
memories across each layer. However, the optimal value for $K$ may vary
depending on the volume and quality of incoming information.

Figure 11: Cumulative Return over time for with different FinMem working
memory capacity.

##  7 Conclusion and future work

In this paper, we introduce FinMem, an innovative automated trading agent
framework featuring an adjustable cognitive memory structure and dynamic
character design. Our research demonstrates its capacity to enhance stock
trading performance substantially using real-world financial datasets.
Additionally, the efficacy of each critical component within FinMem is
thoroughly demonstrated in our ablation studies, highlighting their roles in
optimizing trading outcomes.

Its unique features of human-like cognitive memory modules and dynamic
character design enable it to tackle the complexities of financial
environments and respond aptly to new situations. Compared to other LLM
trading agents, FinMem’s memory module equips it with the capability to better
comprehend financial data featured by various timeliness and organize them as
a self-evolving long-term memory layer. The dynamic character design endows
FinMem with critical professional insights, enabling efficient filtering of
impactful messages from incoming financial data for trading actions.
Additionally, the integration of multiple risk profiles enhances FinMem’s
adaptability to a range of market conditions.

FinMem’s exceptional performance underscores its remarkable ability to
transform diverse financial data into well-informed investment strategies. Its
proficiency in integrating various financial data types is further accentuated
by a notably reduced training duration, which offers advantages for trading
with newly established companies. In our approach, we utilized a limited range
and quality of financial news and reports, employing general-purpose LLMs as
the backbone algorithms. However, FinMem is fully compatible with LLMs
specifically fine-tuned for financial applications. We anticipate that its
trading efficacy will be elevated further with access to a more comprehensive
and higher-quality dataset, coupled with LLMs tailored specifically for
financial contexts.

While primarily designed for financial decision-making, the FinMem framework
boasts a versatility that extends to domains such as IT consulting and
business reporting, where actions are driven by time-sensitive information.
Looking ahead, an intriguing direction for development is the creation of a
multi-agent trading system, rooted in the FinMem platform, aimed at enhancing
investment portfolio optimization. This system, featuring diverse professional
backgrounds in its profiling modules, enables concurrent operations and
trading across a variety of financial products. It dynamically adjust trading
targets based on a sequential assessment of key trading performance
indicators. Simultaneously, it can timely reallocate investment proportions
for each financial product category, leveraging peer-to-peer communication and
systematic performance analysis. Supported by LLMs, this multi-agent trading
system is poised to assemble an optimized investment portfolio through its
strategically planned operations.

## References

  * [1] Andrew Ang and Joseph Chen  “Downside Risk”  In _Journal of Portfolio Management_ 29.4, 2003, pp. 103–112
  * [2] Lisa P Argyle et al.  “Out of one, many: Using language models to simulate human samples”  In _Political Analysis_ 31.3 Cambridge University Press, 2023, pp. 337–351
  * [3] Loukia Avramelou, Paraskevi Nousi, Nikolaos Passalis and Anastasios Tefas  “Deep reinforcement learning for financial trading using multi-modal features”  In _Expert Systems with Applications_ Elsevier, 2023, pp. 121849
  * [4] Surjeet Balhara et al.  “A survey on deep reinforcement learning architectures, applications and emerging trends”  In _IET Communications_ Wiley Online Library, 2022
  * [5] Fischer Black  “Noise”  In _The journal of finance_ 41.3 Wiley Online Library, 1986, pp. 528–543
  * [6] Yu-Fu Chen and Szu-Hao Huang  “Sentiment-influenced trading system based on multimodal deep reinforcement learning”  In _Applied Soft Computing_ 112 Elsevier, 2021, pp. 107788
  * [7] Tai-Liang Chen  “Forecasting the Taiwan stock market with a novel momentum-based fuzzy time-series”  In _Review of Economics & Finance_ 2 Better Advances Press, Canada, 2012, pp. 38–50
  * [8] John H. Cochrane  “Volatility Tests and Efficient Markets: A Review Essay”  In _Journal of Monetary Economics_ 22.3, 1988, pp. 463–485
  * [9] Fergus IM Craik and Robert S Lockhart  “Levels of processing: A framework for memory research”  In _Journal of verbal learning and verbal behavior_ 11.6 Elsevier, 1972, pp. 671–684
  * [10] Quang-Vinh Dang  “Reinforcement learning in stock trading”  In _International conference on computer science, applied mathematics and applications_ , 2019, pp. 311–322  Springer
  * [11] Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova  “Bert: Pre-training of deep bidirectional transformers for language understanding”  In _arXiv preprint arXiv:1810.04805_ , 2018
  * [12] Robert D Edwards, John Magee and WH Charles Bassetti  “Technical analysis of stock trends”  CRC press, 2018
  * [13] Kawin Ethayarajh  “How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings”  In _arXiv preprint arXiv:1909.00512_ , 2019
  * [14] Thomas G Fischer  “Reinforcement learning in financial markets-a survey”, 2018
  * [15] Samuel J Gershman and Bence P Ölveczky  “The neurobiology of deep reinforcement learning”  In _Current Biology_ 30.11 Elsevier, 2020, pp. R629–R632
  * [16] Yoav Goldberg and Omer Levy  “word2vec Explained: deriving Mikolov et al.’s negative-sampling word-embedding method”  In _arXiv preprint arXiv:1402.3722_ , 2014
  * [17] “Guardrails AI” Open source library for interacting with Large Language Models, https://docs.guardrailsai.com
  * [18] Sepp Hochreiter and Jürgen Schmidhuber  “Long short-term memory”  In _Neural computation_ 9.8 MIT press, 1997, pp. 1735–1780
  * [19] Boming Huang et al.  “Automated trading systems statistical and machine learning methods and hardware implementation: a survey”  In _Enterprise Information Systems_ 13.1 Taylor & Francis, 2019, pp. 132–144
  * [20] Ziheng Huang, Sebastian Gutierrez, Hemanth Kamana and Stephen MacNeil  “Memory sandbox: Transparent and interactive memory management for conversational agents”  In _arXiv preprint arXiv:2308.01542_ , 2023
  * [21] John Hull  “Risk Management and Financial Institutions”  John Wiley & Sons, 2007
  * [22] O Jangmin, Jongwoo Lee, Jae Won Lee and Byoung-Tak Zhang  “Adaptive stock trading with dynamic asset allocation using reinforcement learning”  In _Information Sciences_ 176.15 Elsevier, 2006, pp. 2121–2147
  * [23] Jeff Johnson, Matthijs Douze and Hervé Jégou  “Billion-scale similarity search with GPUs”  In _IEEE Transactions on Big Data_ 7.3 IEEE, 2019, pp. 535–547
  * [24] Yifei Li et al.  “Making language models better reasoners with step-aware verifier”  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 2023, pp. 5315–5333
  * [25] Mark Liffiton, Brad Sheese, Jaromir Savelka and Paul Denny  “Codehelp: Using large language models with guardrails for scalable support in programming classes”  In _arXiv preprint arXiv:2308.06921_ , 2023
  * [26] Xiao-Yang Liu et al.  “FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning”  In _Advances in Neural Information Processing Systems_ 35, 2022, pp. 1835–1849
  * [27] Xiao-Yang Liu et al.  “FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance”  In _arXiv preprint arXiv:2011.09607_ , 2020
  * [28] Xiao-Yang Liu, Hongyang Yang, Jiechao Gao and Christina Dan Wang  “FinRL: Deep reinforcement learning framework to automate trading in quantitative finance”  In _ACM International Conference on AI in Finance (ICAIF)_ , 2021
  * [29] Adrian Millea  “Deep reinforcement learning for trading—A critical survey”  In _Data_ 6.11 MDPI, 2021, pp. 119
  * [30] George A Miller  “The magical number seven, plus or minus two: Some limits on our capacity for processing information.”  In _Psychological review_ 63.2 American Psychological Association, 1956, pp. 81
  * [31] Volodymyr Mnih et al.  “Asynchronous methods for deep reinforcement learning”  In _International conference on machine learning_ , 2016, pp. 1928–1937  PMLR
  * [32] Volodymyr Mnih et al.  “Playing atari with deep reinforcement learning”  In _arXiv preprint arXiv:1312.5602_ , 2013
  * [33] Jaap MJ Murre and Joeri Dros  “Replication and analysis of Ebbinghaus’ forgetting curve”  In _PloS one_ 10.7 Public Library of Science San Francisco, CA USA, 2015, pp. e0120644
  * [34] OpenAI  “GPT-4 Technical Report”, 2023  arXiv:2303.08774 [cs.CL]
  * [35] Joon Sung Park et al.  “Generative Agents: Interactive Simulacra of Human Behavior”  In _Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology_ , UIST ’23  San Francisco, CA, USA: Association for Computing Machinery, 2023  DOI: 10.1145/3586183.3606763
  * [36] Joon Sung Park et al.  “Social simulacra: Creating populated prototypes for social computing systems”  In _Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology_ , 2022, pp. 1–18
  * [37] Eero Pätäri and Mika Vilska  “Performance of moving average trading strategies over varying stock market conditions: the Finnish evidence”  In _Applied Economics_ 46.24 Taylor & Francis, 2014, pp. 2851–2872
  * [38] Jeffrey Pennington, Richard Socher and Christopher D Manning  “Glove: Global vectors for word representation”  In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_ , 2014, pp. 1532–1543
  * [39] Tidor-Vlad Pricope  “Deep reinforcement learning in quantitative algorithmic trading: A review”  In _arXiv preprint arXiv:2106.00123_ , 2021
  * [40] Alec Radford, Karthik Narasimhan, Tim Salimans and Ilya Sutskever  “Improving language understanding by generative pre-training”  OpenAI, 2018
  * [41] Antonin Raffin et al.  “Stable-Baselines3: Reliable Reinforcement Learning Implementations”  In _Journal of Machine Learning Research_ 22.268, 2021, pp. 1–8  URL: http://jmlr.org/papers/v22/20-1364.html
  * [42] John Schulman et al.  “Proximal policy optimization algorithms”  In _arXiv preprint arXiv:1707.06347_ , 2017
  * [43] William F. Sharpe  “The Sharpe Ratio”  In _The Journal of Portfolio Management_ 21.1, 1994, pp. 49–58
  * [44] Yong Shi et al.  “Stock trading rule discovery with double deep Q-network”  In _Applied Soft Computing_ 107 Elsevier, 2021, pp. 107320
  * [45] Theodore Sumers, Shunyu Yao, Karthik Narasimhan and Thomas L Griffiths  “Cognitive architectures for language agents”  In _arXiv preprint arXiv:2309.02427_ , 2023
  * [46] Ron Sun  “Desiderata for cognitive architectures”  In _Philosophical psychology_ 17.3 Taylor & Francis, 2004, pp. 341–373
  * [47] John Sweller  “Human cognitive architecture: Why some instructional procedures work and others do not.”  American Psychological Association, 2012
  * [48] Hugo Touvron et al.  “Llama: Open and efficient foundation language models”  In _arXiv preprint arXiv:2302.13971_ , 2023
  * [49] Rashesh Vaidya  “Moving Average Convergence-Divergence (MACD) Trading Rule: An Application in Nepalese Stock Market" NEPSE"”  In _Quantitative Economics and Management Studies_ 1.6, 2020, pp. 366–374
  * [50] Lei Wang et al.  “A survey on large language model-based autonomous agents”  In _arXiv preprint arXiv:2308.11432_ , 2023
  * [51] Lei Wang et al.  “RecAgent: A Novel Simulation Paradigm for Recommender Systems”  In _arXiv preprint arXiv:2306.02552_ , 2023
  * [52] Shijie Wu et al.  “Bloomberggpt: A large language model for finance”  In _arXiv preprint arXiv:2303.17564_ , 2023
  * [53] Xing Wu et al.  “Adaptive stock trading strategies with deep reinforcement learning methods”  In _Information Sciences_ 538 Elsevier, 2020, pp. 142–158
  * [54] Zhuoran Xiong et al.  “Practical deep reinforcement learning approach for stock trading”  In _arXiv preprint arXiv:1811.07522_ , 2018, pp. 1–7
  * [55] Hongyang Yang, Xiao-Yang Liu and Christina Dan Wang  “FinGPT: Open-Source Financial Large Language Models”  In _arXiv preprint arXiv:2306.06031_ , 2023
  * [56] Hongyang Yang, Xiao-Yang Liu, Shan Zhong and Anwar Walid  “Deep reinforcement learning for automated stock trading: An ensemble strategy”  In _Proceedings of the first ACM international conference on AI in finance_ , 2020, pp. 1–8
  * [57] Hongxin Zhang et al.  “Building cooperative embodied agents modularly with large language models”  In _arXiv preprint arXiv:2307.02485_ , 2023
  * [58] Wayne Xin Zhao et al.  “A survey of large language models”  In _arXiv preprint arXiv:2303.18223_ , 2023