11institutetext: University of St. Gallen (HSG), Institute of Computer
Science, St. Gallen, Switzerland, 11email<EMAIL_ADDRESS>https://ics.unisg.ch/chair-ds-nlp-handschuh/

# The Structure of Financial Equity Research Reports – Identification of the
Most Frequently Asked Questions in Financial Analyst Reports to Automate
Equity Research Using Llama 3 and GPT-4

Finance FAQ (FFAQ)

Adria Pop  Jan Spörer  Siegfried Handschuh

###### Abstract

This research dissects financial equity research reports (ERRs) by
systematically mapping their content into categories.

There is insufficient empirical analysis of the questions answered in ERRs. In
particular, it is not understood how frequently certain information appears,
what information is considered essential, and what information requires human
judgment to distill into an ERR.

The study analyzes 72 ERRs sentence-by-sentence, classifying their 4940
sentences into 169 unique question archetypes. We did not predefine the
questions but derived them solely from the statements in the ERRs. This
approach provides an unbiased view of the content of the observed ERRs.
Subsequently, we used public corporate reports to classify the questions’
potential for automation. Answers were labeled “text-extractable” if the
answers to the question were accessible in corporate reports.

78.7% of the questions in ERRs can be automated. Those automatable question
consist of 48.2% text-extractable (suited to processing by large language
models, LLMs) and 30.5% database-extractable questions. Only 21.3% of
questions require human judgment to answer.

We empirically validate, using Llama-3-70B and GPT-4-turbo-2024-04-09 that
recent advances in language generation and information extraction enable the
automation of approximately 80% of the statements in ERRs. Surprisingly, the
models complement each other’s strengths and weaknesses well.

The research confirms that the current writing process of ERRs can likely
benefit from additional automation, improving quality and efficiency. The
research thus allows us to quantify the potential impacts of introducing large
language models in the ERR writing process.

The full question list, including the archetypes and their frequency, will be
made available online after peer review.

###### Keywords:

applied natural language processing text generation question answering
generative AI natural language processing financial text

Working Paper

##  1 Introduction

This empirical study aims to estimate the automation potential when writing
equity research reports (ERRs) by systematically classifying statements of
equity researchers into categories, and subsequently evaluating which sections
and statements in ERRs require human judgment. One of the key contributions is
a question list that analysts answer in ERRs. This question list gives a
holistic overview of the ERR topics. We classify each question’s automation
potential by comparing the data sources to the statements to see whether an
automated system could have written the ERRs. If the share of automatable
questions is high, this study may indicate that the automation of large parts
of ERRs is feasible.

ERRs are written mainly by sell-side banks and specialized research companies.
The purpose of ERRs is ultimately to serve as a buy, hold, or sell
recommendation (Asquith et al.,, 2005, p. 246). Two of the most common
differences among ERRs are their content and intention. The content of ERRs
can be to initiate a company’s coverage, provide an ordinary update to an
already covered company, or provide an extraordinary update about a company.
Subsequent recommendations to buy, hold, or sell may change or stay the same.
One common update cycle for ERRs is quarterly ordinary updates. ERRs thus
often contain recent facts from quarterly company-provided financial reports.
Half of issued new ERRs generally fall into this category, being released
closely after the company published new information (Asquith et al.,, 2005, p.
247). We review the literature on ERRs in section 2.1, discussing their
importance, accuracy, and existing approaches for automation.

We created the question list by manually reading each sentence of 72 ERRs. We
mapped each phrase in the reports to a question. When we encountered a new
question, we added it to the list. The result is a histogram of question
occurrences (figure 2). We provide more detail about the methodology in
section 4.

This approach aligns with prior research (Asquith et al.,, 2005, p. 251), with
the main difference being that we did not assume specific data fields (or
questions) to be present in the reports ex-ante. Instead, we recorded each
statement, derived a question from it, and counted the number of occurrences
of each question. This approach ensures maximum unbiasedness in representing
the landscape of ERRs.

No systematic reviews of ERR automation exist. News feeds for financial news
are already partially generated by AI systems. However, the automation of ERRs
is not yet widely automated, albeit being feasible (Coleman et al.,, 2022).
Some consumer-grade analyst houses such as Zacks.com use template-based
automation to update their company profiles and overview articles. Longer
texts, however, are still written by humans. ERRs are one example of such
longer texts, and they are the focus of this study.

##  2 Literature

###  2.1 Review of the Financial Economics Literature

Considering how strongly stock market prices react after ERRs are published,
their importance for investors and, by extension, stock-listed corporations is
evident in the literature (Bjerring et al.,, 1983; Elton et al.,, 1986; Liu et
al.,, 1990; Beneish,, 1991; Stickel,, 1995).

Womack, (1996) observed that analysts could predict the directionality of
stock returns six months into the future (pp. 139, 163–165), indicating
significant information content in ERRs. At least directionally, not
necessarily with respect to the accuracy of the price targets, ERRs thus
exhibit predictive accuracy higher than what can be expected from random
guessing.

A persistent problem with the reliability of ERRs is the reluctance of
analysts to present negative recommendations, as Barber et al., (2001) and
Mikhail et al., (2004) demonstrated. Asquith et al., (2005) report a rate of
sell recommendations of only 0.5% in an ERR sample from 1997 to 1999 (p. 255),
while Womack, (1996) report a rate of sell recommendations of 14% (p. 164)
between 1989 and 1991. Prior research by Michaely and Womack, (1999) concludes
that the reason for the low number of sell recommendations is likely that the
companies covered by many financial analysts are the banks’ clients.
Therefore, incentives arise to report too positively.

According to a study by Asquith et al., (2005), more than half (54%) of
analyst reports set price targets that are achieved within a year (pp.
278–279). This accuracy rate appears relatively low, considering that the
majority of ERRs tend to offer conservative recommendations, with price
targets slightly exceeding current prices, on average (Asquith et al.,, 2005,
p. 256). The authors did not judge whether the analysts’ success rate was good
or poor. In contrast to this outcome, Bradshaw et al., (2013) found that only
38% of analysts’ price targets are met within a one-year horizon (pp.
953–954). Bonini et al., (2010) provide another critical analysis of financial
analysts’ accuracy (pp. 1177, 1193–1196, 1208). The unequivocal findings
potentially strengthen the argument for automating ERRs to achieve more robust
price targets (Gleason et al.,, 2013, pp. 80–81, for further discussion on
report accuracy).

Research by Coleman et al., (2022) shows that these issues can be mitigated by
using a hybrid machine-human approach. The study presents a computer-aided
approach that better balances the buy, hold, and sell recommendation
frequencies, achieves better portfolio performance, and reduces the time
required for writing reports.

The previously presented studies show that the importance and accuracy of ERRs
are studied extensively. Furthermore, there is extensive research on how stock
performance is predicted by systematic factors, notably factor models by
Eugene Fama and Kenneth French Fama and French, (1992, 1995, 2018, 2015) and
by other studies from the field of risk factors and asset pricing (Carhart,,
1997; Asness et al.,, 2013).

Asquith et al., (2005) performed a similar empirical analysis as our study.
They ex-ante determined 30 variables of interest that were extracted from
ERRs. There are differences in the objective, the data, and the methodology
when comparing Asquith et al., to our study. They examined the accuracy of
ERRs, how much they impact markets, and how independent research providers
compare to sell-side banks. The data is from 1997 to 1999, and they selected
only a subset of high-performing analysts. Also, they derived the 30 data
fields from the objective before reading the ERRs; in this study, however, we
read the ERRs and generated the questions ad-hoc when encountering new
questions.

Figure 1: Histogram With Statement Counts in ERRs by Frequency. ERRs rarely
have less than 30 statements. Most reports have between 30 and 119 statements.

###  2.2 Review of the Technical Literature

Question answering (QA) systems commonly use text retrieval. Text retrievers
get a question or a topic as input and search for fitting segments from a
knowledge base. The retrieved span of text can be used directly as the answer
or can be postprocessed by another system. Text retrieval from financial
statements, even in combination with text from table captions, is already
implemented by Chen et al., (2021). As retrievers are capable of choosing
facts from knowledge bases of practically unlimited size, they are a key
lower-level component needed to enable fact-based automated writing Borgeaud
et al., (2021). It thus remains a core technology for automated ERR writing,
at least until large language models’ context sizes become larger than they
are today (Chen et al.,, 2023).

As the stated goal of this study is to pave the way for automating the writing
of ERRs, we provide a brief overview of the QA literature. QA is a subfield of
natural language processing (NLP). It can be one of the domains that
facilitate the automation of ERR writing.

Dense passage retrieval uses latent representation of the question to search
for an answer in a large corpus of text Karpukhin et al., (2020). Combining
such a retrieval mechanism with a generative language model by including the
retrieval outputs to the language model prompt, one gets a retrieval-augmented
generator (RAG) as presented by Lewis et al., (2020). RAG continues to receive
attention from the research community, as follow-up research on the topic
shows (Izacard et al.,, 2022; Guu et al.,, 2020; Borgeaud et al.,, 2021). Its
effectiveness in writing factually correct and fluent text makes RAG a
technology that may facilitate the automation of ERR writing.

With the emergence of the mentioned RAG methods, generative models are better
capable of performing QA tasks. Furthermore, as models and their training data
scale, their ability to store knowledge in their parameters increases, making
them capable QA models even without external knowledge bases, as GPT-3 Brown
et al., (2020) and the Llama models Touvron et al., (2023) demonstrate. In
addition, advances in the expansion of the context length (Press et al.,,
2022; Chen et al.,, 2023) make it possible to provide more world knowledge
into prompts.

Figure 2: Histogram of Category Frequency, Grouped by Question Category. The
histogram shows how frequently different information is displayed, grouped by
question category. ERRs contain the financial category most frequently and
usually display this category in text-or-tabular format or in text-tabular-or-
graphical or tabular format. Company and market information are commonly in
text-only format. Notably, stock information is rarely in text-only format.
Only 2.82% of questions (weighted by their number of occurrences) were
exclusively displayed in graphical format.

As the general capabilities of language models grow, finance-specific language
models also improve. Wu et al., (2023) develop a language model capable of
understanding the nuances of a financial text. About half of their training
data is finance-specific. The model performs well on financial QA (pp. 31–32).
These generative language model developments increase the capabilities of
state-of-the-art language models to generate factually correct ERRs.

##  3 Data

We downloaded the ERRs dated from 2018 to 2023 from Bloomberg and Refinitiv
Eikon. Each of the 72 reports had an average of seven pages, with the median
being 6.8 pages per report. The shortest report is a one-pager, and the
longest report has 20 pages. See also figure 8 for a histogram of page counts.
We analyzed 493 pages across all reports. These statistics are in line with
findings of previous research by (Asquith et al.,, 2005, p. 252). We sourced
the ERRs from 23 different research providers, each contributing between 1 and
16 reports (figure 3).

The number of statements per report ranges from nine to 115. The average is
68, and the median is 69. See also figure 1 for a histogram of statement
counts. In sum, we analyzed 4940 statements (sentences).

##  4 Methods

Figure 3: Display type of Statements in the ERRs. The Venn diagram shows how
information is conveyed in ERRs. About half of the statements can be made in
either textual or tabular form. 980 statements only appear in text form, and
764 statements always appear in tables. Only relatively few statements, mostly
related to stock price history and other market data, are only displayed in
graphical form.

###  4.1 Question List and Question Categories

The annotation process was bias-free, without presumptions about the space of
questions we would encounter. Sentence-by-sentence, we read the ERRs and
annotated each sentence with a question. When we encountered an answer to a
question that we previously saw, we mapped the statement to the existing
question. When a question was not on the list, we added a new question.

We grouped the resulting 169 questions into five categories. The categories
are: Financials, Company, Product, Stock, Market, Analysis. The questions are
listed in tables 4-7.

###  4.2 Question Classification

To make use of this question list for our purpose of analyzing the automation
potential of ERRs, we classified each unique question in two dimensions:

  1. 1.

Extractability: If the answer to a question is found in a corporate report,
the question is extractable. To validate that our classification of
extractability is correct, we ran the open-source Llama-3-70B model and the
closed-source GPT-4-turbo-2024-04-09 model on 200 example questions, showing
that these language models can indeed extract the answers to the questions
when provided with annual reports as the prompt context. We describe this
validation step in 4.4.

  2. 2.

Display Modality: Refers to how analysts display the statement. The results
are reported in figure 3, showing that most information can be displayed in
text or tabular form.

The aggregated results are in table 2 and the original flags are in tables
4-7. We used company-issued reports to check whether the questions are
extractable.

The annotation process required two iterations of reading through the reports.
In the first reading process, the question list was created. In the second
iteration, we labeled the extractability and the modality columns. The second
iteration involved reading the ERRs again, and finding the source for the
answer to each question. If a direct answer was matched in a single source,
the extractability was marked as “extractable.” If the answer was not found in
a single passage, the extractability was marked as “non-extractable.”

Figure 4: Frequency of Different Modes of Data Representation in ERRs. The two
categories “Tabular or Graphical Data” (53 occurrences) and “Textual or
Graphical Data” (52) were filtered out as only very few questions are
represented in these ways.

###  4.3 Qualitative Validation of the Question List Using Expert Interviews
and Prior Research

We conducted interviews with ten financial analysts. Each interview lasted 45
minutes. The interviews aimed to validate the relevance of our results and
confirm whether the count-based ranking of questions was realistic. In the
interviews, we were reassured that our results align with what would be
expected in practice when considering the most important questions in ERRs. In
some interviews, however, it became clear that financial analysts consider the
management qualifications and subjective impressions about the competence of
managers. This aspect is not captured in this study and will be hard to
capture for automated systems.

In addition to validation through qualitative interviews, prior research by
Asquith et al., (2005, p. 246) confirms that the top questions identified in
our study match their findings about the most frequent ERR contents.

###  4.4 Validation of the Automation Potential by Comparing Human,
Llama-3-70B, and GPT-4-turbo-2024-04-09 Report Generation Performance

We validate the claims made in this paper about the automation potential of
specific questions by automating parts of the report generation. We test which
questions are the hardest questions for language models to answer, which
informs our assessment of the question “extractability.“

We use the open-source Llama-3-70B model by Meta AI and the closed-source
GPT-4-turbo-2024-04-09 model by OpenAI. We set the temperatures of both models
to zero (giving the models the chance to always use their true best guesses)
and do not limit the number of output tokens. In cases where the context
lengths of the models were exceeded, we split the contexts and concatenated
the models’ outputs.

Figure 5: Share of Correct, Incorrect, Database-Extractable, and Non-
Extractable Questions for GPT-4-turbo-2024-04-09 and Llama-3-70B. Stock-
related questions can only be answered using financial market data. Those thus
fall under a separate category that is automatable, yet not by using language
models. Llama 3 has a slight edge over GPT-4 as it answered more questions
correctly than GPT-4 did.

Our results show that Llama-3-70B is able to extract information from annual
reports in 27% of the 200 questions. GPT-4 is able to extract the correct
answer in 26% of the questions. The results are in line with our expectations
and confirm that the models can indeed extract the answers to the questions
when provided with annual reports as the prompt context.

If one adds the database-extractable questions, which can be gathered
automatically from financial data providers, the share of automatable
questions rises to 55% + 26% = 81% for GPT-4, and 82% for Llama 3. If one then
also considers that the models’ performance is highly uncorrelated, one could
use both models at once to achieve an ensemble that can answer 84% of
questions, and makes mistakes only for about 1% of questions (see also figure
6).

Figure 6: Correctness of Answers by GPT-4-turbo-2024-04-09, Llama-3-70B, and
the Best of Both Models. The green parts show correct answers, the red-white-
hatched parts show errors. Interestingly, the errors of GPT-4 and Llama 3 have
almost no overlap. When one model is unable to correctly answer a question,
the other model usually is. There was only one question that both models did
not answer correctly despite the relevant information being present in the
prompt. The fine black lines in the right plot delimit different questions,
and the culmination of errors for certain questions shows that the models have
difficulty with particular questions.

As a qualitative side note, we found that GPT-4 tends to provide longer
responses with more context. In some cases, we thus found that GPT-4 provided
helpful context that Llama-3-70B missed. We performed some follow-up tests
with Llama 3 to see if this difference was a lack of capability or simply a
difference in the default verbosity among the models. We found that, for the
purpose of financial text, Llama 3 is able to provide the same extraction
depth and abstraction capabilities as GPT-4 does. But Llama 3 tends to provide
more direct answers compared to GPT-4 unless prompted to add contextual
flavor. Also, Llama often attempts to calculate growth numbers when asked
about rates. In all attempts, it fails to provide correct absolute or relative
year-on-year changes, but stays in the correct ballpark of plus/minus 10%.

The language models correctly identified information in full-form text and in
tabular format. Tables were simply copy-pasted from the annual reports, so
that the formatting of these tables was not specifically optimized language
model readability. The models showed high robustness in extracting financial
information from tables. For each mistake made by the language models, we
inspected the context to see if a human had been able to answer the question
given the text-only context (no PDF formatting was provided, limiting what the
language models were able to see relative to what a human would be able to
infer from a text’s context in the annual report). We made sure that no
language model answer was marked as incorrect if there was no clear answer in
the context, but no such cases occurred in the sample of 200 questions.

##  5 Results

Figure 7: Share of Question Subcategories. Please note that there are three
“Other” labels on the x-axis. These refer to the “Other” subcategory of their
respective categories: “Financial – Other,” “Company – Other,” and “Analysis –
Other.”

###  5.1 Result Overview

In summary, 78.7% of the 169 questions in ERRs are automatable. 48.2% of the
statements in ERRs are extractable. 30.4% of questions require access to non-
public databases but have potential for automation. Only 21.3% of questions
require judgment that goes beyond extraction.

###  5.2 Analysis of Question Categories and Subcategories

Table 1: Question Frequency List of the Five Most Frequently Answered Questions. The table indicates which types of statements are the most frequent across all ERRs. Across the 72 reports, 66 contained information about the stock price and 65 about the company’s challenges. Company details and market environment statements appear less frequently. # | Question | Count | Subcategory | Numerical |  | Extractable
---
From Text
1 | Stock price | 66 | Stock - Price | Yes | No
2 | Potential challenges | 65 | Company - Other | No | No
3 | Primary industry | 64 | Product | No | Yes
4 | Analyst score | 64 | Financials - Other | No | Yes
5 | Earnings per share | 62 | Financials - Financing | Yes | Yes

A share of 73.4% of statements in the category Product are automatable. The
most critical question category by statement count is the Financials. 70.6% of
statements from this category are extractable. A share of 54.6% is automatable
in the category Company. A share of 16.6% is automatable in the category
Stock. A share of 4.6% is automatable in the category Market. This is because
statements about the market environment usually require access to diverse
sources outside the company’s annual and quarterly reports. None of the
statements from the Analysis category can be automated with extractions from
publicly available corporate reports. This category contains the target price
(forward guidance), recommendation, and risk assessments.

###  5.3 Analysis of Extractable and Non-Extractable Statements

Table 2: Overview of Extractable and Non-extractable Statements. Numeric-
extractable statements have a share of 40%. Numeric-non-extractable statements
have a share of 26%. Almost two-thirds of non-numeric statements are non-
extractable.

Counted by the number of unique questions:

| Numeric | Non-Numeric
---|---|---
Extractable From Text | 67 (40.61%) | 17 (10.3%)
Not Extractable | 42 (25.45%) | 39 (23.64%)

Counted by the total number of statement occurrences:

| Numeric | Non-Numeric
---|---|---
Extractable From Text | 1925 (38.78%) | 437 (8.8%)
Not Extractable | 1425 (28.71%) | 1177 (23.71%)

Table 2 portrays the classifications of questions answered in the examined
ERRs. The x-axis shows whether the questions answered are numeric or non-
numeric. The y-axis exhibits whether the information gained is extractable
from another source of non-written text.

Two-thirds are numeric, and more than half are extractable. While extractable
information is mostly numeric (40%), extractable non-numeric information is
rare (12%). Out of the non-extractable information, slightly more is numeric.

###  5.4 Analysis of Contextualizing and Summarizing Components

The Analysis question category contains mostly summarizing and contextualizing
components. Given the same set of facts, different analysts may weigh, select,
and combine those facts differently, leading to different recommendations.

Related to this, there are numerous questions in the category Market of this
nature. Market developments require simplification and curation to distill
into a few pages of text. Similar to the Analysis category, different
observers judge the same set of facts differently, leading to different
conclusions. Given that the potential inputs to this category are vast, with
many news reports and other sources to choose from, it is unlikely that an
automated system can already handle this task.

###  5.5 Tabular Data

Most statements in ERRs are textual or tabular. Chen et al., (2021) show that
numerical reasoning across tables and text is feasible (pp. 5–7). Zhu et al.,
(2021) confirm that tabular information extraction is possible, particularly
for financial data (pp. 3282–3284).

We confirm these findings: Our validation from section 4.4 has not required
any manual formatting of table data – we copied tables from annual reports
without formatting into the models’ context, and they extracted information
from these ill-formatted strings with high reliability.

##  6 Conclusion

Our results confirm the findings by Coleman et al., (2022) that partly
automating equity research reports (ERRs) is feasible. Only 21.3% of questions
require complex judgment that takes into consideration more information than
would fit in a language models’ context window.

In particular, the Analysis category in ERRs contains summarizing statements
that make recommendations. These statements are not extractable from other
sources. Only 3.5% of statements in ERRs fall under the analysis category.
Given the oversized importance of this category, and given that humans may,
for the time being, still be better at providing high-stake recommendations,
this category should be left to human financial analysts.

Figure 2 shows that, without weighting the questions by their occurrence
frequency, 48.2% of questions answered in ERRs can be answered by extracting
information from public textual sources. An additional 30.4% are available in
proprietary market data sources.

Our counting approach does not weigh the importance of the questions. The most
important questions may appear less frequently. Furthermore, there could be
out-of-distribution questions that we did not capture in this analysis because
they were not present in the ERRs we analyzed.

Only 72 ERRs from 23 research firms were dissected. Other research firms may
include questions not in the space of 169 question archetypes identified in
this study.

Direct information extraction from ERRs is still a largely unexplored field.
Various approaches were presented in section 2.1. Future research can
implement these methods.

## References

  * Asness et al., (2013) Asness, C., Moskowitz, T., and Pedersen, L. 2013\.  Value and momentum everywhere.  The Journal of Finance, 68(3):929–985.
  * Asquith et al., (2005) Asquith, P., Mikhail, M., and Au, A. 2005\.  Information content of equity analyst reports.  Journal of Financial Economics, 75(2):245–282.
  * Barber et al., (2001) Barber, B., Lehavy, R., McNichols, M., and Trueman, B. 2001\.  Can investors profit from the prophets? Security analyst recommendations and stock returns.  The Journal of Finance, 56(2):531–563.
  * Beneish, (1991) Beneish, M. 1991\.  Stock prices and the dissemination of analysts’ recommendation.  Journal of Business, pp. 393–416.
  * Bjerring et al., (1983) Bjerring, J., Lakonishok, J., and Vermaelen, T. 1983\.  Stock prices and financial analysts’ recommendations.  The Journal of Finance, 38(1):187–204.
  * Bonini et al., (2010) Bonini, S., Zanetti, L., Bianchini, R., and Salvi, A. 2010\.  Target price accuracy in equity research.  Journal of Business Finance & Accounting, 37(9-10):1177–1217.
  * Borgeaud et al., (2021) Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., van den Driessche, G., Lespiau, J., Damoc, B., Clark, A., de Las Casas, D., Guy, A., Menick, J., Ring, R., Hennigan, T., Huang, S., Maggiore, L., Jones, C., Cassirer, A., Brock, A., Paganini, M., Irving, G., Vinyals, O., Osindero, S., Simonyan, K., Rae, J., Elsen, E., and Sifre, L. 2021\.  Improving language models by retrieving from trillions of tokens.  International Conference on Machine Learning, Proceedings of Machine Learning Research (PMLR), 162.
  * Bradshaw et al., (2013) Bradshaw, M., Brown, L., and Huang, K. 2013\.  Do sell-side analysts exhibit differential target price forecasting ability?  Review of Accounting Studies, 18(4):930–955.
  * Brown et al., (2020) Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. 2020\.  Language models are few-shot learners.  Advances in Neural Information Processing Systems (NeurIPS), 33:1877–1901.
  * Carhart, (1997) Carhart, M. 1997\.  On persistence in mutual fund performance.  The Journal of Finance, 52(1):57–82.
  * Chen et al., (2023) Chen, S., Wong, S., Chen, L., and Tian, Y. 2023\.  Extending context window of large language models via positional interpolation.  arXiv.
  * Chen et al., (2021) Chen, Z., Chen, W., Smiley, C., Shah, S., Borova, I., Langdon, D., Moussa, R., Beane, M., Huang, T.-H., Routledge, B., and Wang, W. Y. 2021\.  FinQA: A dataset of numerical reasoning over financial data.  Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3697–3711.
  * Coleman et al., (2022) Coleman, B., Merkley, K., and Pacelli, J. 2022\.  Human versus machine: A comparison of robo-analyst and traditional research analyst investment recommendations.  The Accounting Review, 97(5):221–244.
  * Elton et al., (1986) Elton, E., Gruber, M., and Grossman, S. 1986\.  Discrete expectational data and portfolio performance.  The Journal of Finance, 41(3):699–713.
  * Fama and French, (1992) Fama, E. and French, K. 1992\.  The cross-section of expected stock returns.  The Journal of Finance, 47(2):427–465.
  * Fama and French, (1995) Fama, E. and French, K. 1995\.  Size and book-to-market factors in earnings and returns.  The Journal of Finance, 50(1):131–155.
  * Fama and French, (2015) Fama, E. and French, K. 2015\.  A five-factor asset pricing model.  Journal of Financial Economics, 116(1):1–22.
  * Fama and French, (2018) Fama, E. and French, K. 2018\.  Choosing factors.  Journal of Financial Economics, 128(2):234–252.
  * Gleason et al., (2013) Gleason, C., Johnson, B., and Li, H. 2013\.  Valuation model use and the price target performance of sell-side equity analysts.  Contemporary Accounting Research, 30(1):80–115.
  * Guu et al., (2020) Guu, K., Lee, K., Tung, Z., Pasupat, P., and Chang, M.-W. 2020\.  REALM: Retrieval-augmented language model pre-training.  International Conference on Machine Learning, PMLR, pp. 3929–3938.
  * Izacard et al., (2022) Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi-Yu, J., Joulin, A., Riedel, S., and Grave, E. 2022\.  Atlas: Few-shot learning with retrieval augmented language models.  arXiv.
  * Karpukhin et al., (2020) Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. 2020\.  Dense passage retrieval for open-domain question answering.  Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6769–6781.
  * Lewis et al., (2020) Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., Riedel, S., and Kiela, D. 2020\.  Retrieval-augmented generation for knowledge-intensive NLP tasks.  Advances in Neural Information Processing Systems (NeurIPS), 33:9459–9474.
  * Liu et al., (1990) Liu, P., Smith, S., and Syed, A. 1990\.  Stock price reactions to the Wall Street Journal’s securities recommendations.  Journal of Financial and Quantitative Analysis, 25(3):399–410.
  * Michaely and Womack, (1999) Michaely, R. and Womack, K. 1999\.  Conflict of interest and the credibility of underwriter analyst recommendations.  The Review of Financial Studies, 12(4):653–686.
  * Mikhail et al., (2004) Mikhail, M., Walther, B., and Willis, R. 2004\.  Do security analysts exhibit persistent differences in stock picking ability?  Journal of Financial Economics, 74(1):67–91.
  * Press et al., (2022) Press, O., Smith, N., and Lewis, M. 2022\.  Train short, test long: Attention with linear biases enables input length extrapolation.  International Conference of Learning Representations (ICLR).
  * Stickel, (1995) Stickel, S. 1995\.  The anatomy of the performance of buy and sell recommendations.  Financial Analysts Journal, pp. 25–39.
  * Touvron et al., (2023) Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., and others 2023\.  LLaMA: Open and efficient foundation language models.  arXiv.
  * Womack, (1996) Womack, K. 1996\.  Do brokerage analysts’ recommendations have investment value?  The Journal of Finance, 51(1):137–167.
  * Wu et al., (2023) Wu, S., Irsoy, O., Lu, S., Dabravolski, V., Dredze, M., Gehrmann, S., Kambadur, P., Rosenberg, D., and Mann, G. 2023\.  BloombergGPT: a large language model for finance.  arXiv.
  * Zhu et al., (2021) Zhu, F., Lei, W., Huang, Y., Wang, C., Zhang, S., Lv, J., Feng, F., and Chua, T.-S. 2021\.  TAT-QA: A question answering benchmark on a hybrid of tabular and textual content in finance.  In Annual Meeting of the Association for Computational Linguistics and International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 3277–3287.

Figure 8: The histogram shows how frequent certain page counts are among the equity research reports (ERRs). We identified two ERRs with one page and one ERR with two pages. Then, starting with three pages, the number increases significantly, with most ERRs having between three and nine pages. The longest report has 20 pages. Table 3: J.P. Morgan provided the most equity research reports for this analysis with 16 pieces, followed by Deutsche Bank (9), Zacks (8), Barclays (7) and Mizuhu (5). GlobalData and China Renaissance have one report each, with 115 and 106 statements for each of these reports. J.P. Morgan wrote 93, while Zacks wrote 90 and Phillip Securities Research 78 statements per report on average. The average number of statements is 68.6, the median is 70, and the minimum number is a result of Veritas Investment Research’s one-pager with only nine statements. Research Provider |  | Research Report
---
Counts
| Avg. No. Statements
---
per Report
J.P. Morgan | 16 | 93
Deutsche Bank | 9 | 56
Zacks | 8 | 90
Barclays | 7 | 63
Mizuho | 5 | 48
Needham | 5 | 71
KBW | 3 | 55
Refinitiv | 2 | 53
New Constructs | 2 | 40
Phillip Securities Res. | 2 | 78
GlobalData | 1 | 115
China Renaissance | 1 | 106
IBM Res. | 1 | 34
Punto Casa de Bolsa | 1 | 24
Spartan Capital | 1 | 58
Thompson Res. | 1 | 34
Mitsubishi UFJ M.S. | 1 | 31
Oppenheimer | 1 | 44
BPC Res. | 1 | 44
Veritas Investment | 1 | 9
Echelon | 1 | 59
finnCap | 1 | 75
BTIG | 1 | 69
Table 4: Question ranking table 1 of 4 with the top questions by frequency,
ranks 1–50. The table is a question frequency list of the first 50 of the 169
unique questions derived from factual statements and predictive assessments
from equity research reports (ERRs). The table indicates which types of
statements are the most frequent across all equity research reports. Across
the 72 reports, 66 contained information about the stock price and 65 about
the company’s challenges. The less frequent questions refer to more detailed
statements about the company and its market environment. The low frequency of
these questions indicates that the basic questions displayed in the table are
the main focus of most ERRs.

# Question Count Subcategory Numerical Extractable From Text 1 Key financials
122 Financials - Other Yes Yes 2 Analyst rating/buy or sell recommendation 64
Analysis - Recommend. No No 3 Cash flow 64 Financials - P&L Yes Yes 4 Target
price? 62 Analysis - Recommend. Yes No 5 Revenue over time 60 Financials - P&L
Yes Yes 6 EPS forecasts 60 Financials - P&L Yes No 7 Share price/target price
history 60 Stock - Price Yes No 8 EPS growth? 59 Stock - Price Yes No 9 Market
cap? 58 Stock - Price Yes No 10 52-week high (what and when) 56 Stock - Price
Yes No 11 52-week low (what and when) 56 Stock - Price Yes No 12 Current state
of the environment 56 Market - Other No No 13 Revenue growth 55 Financials -
P&L Yes Yes 14 Valuation key figures 55 Stock - Valuation Yes No 15 Day of
market close 54 Stock - Price No No 16 What does the segment consist of? 52
Company - Other No Yes 17 What’s the EBITDA? 52 Financials - P&L Yes No 18
Revenue estimates 52 Financials - P&L Yes No 19 What does the price
performance look like? 52 Stock - Price Yes No 20 What is the company type? 51
Company - Corp. Structure No Yes 21 What is the company’s product/service? 48
Product No Yes 22 How much are the shares outstanding? 48 Financials - Balance
Sh. Yes Yes 23 What is the operating income (+margin)? 47 Financials - P&L Yes
Yes 24 Avg. daily trading volume (ADTV)? 47 Stock - Valuation Yes Yes 25 What
is the net income? 46 Financials - P&L Yes Yes 26 Revenue per segment 46
Financials - P&L Yes Yes 27 Who influences company activities 46 Company -
Other No No 28 What’s the net income after taxes? 45 Financials - P&L Yes Yes
29 What are the dividends per share? 45 Financials - P&L Yes Yes 30 What’s the
price/earnings ratio? 45 Stock - Price Yes No 31 What are potential
opportunities? 45 Market - Other No No 32 What are the sales for the given
year? 44 Financials - P&L Yes Yes 33 What are the income taxes? 44 Financials
- P&L Yes Yes 34 What’s the trailing PE? 44 Stock - Price Yes No 35 Cash flow
from investing 44 Financials - Cash Fl. Yes Yes 36 Gross profit 43 Financials
- P&L Yes Yes 37 SG&A expensive 43 Financials - P&L Yes Yes 38 What is our
recommendation? 43 Analysis - Recommend. No No 39 Regions of operation 42
Company - Channels & Regions No Yes 40 What’s the income before taxes? 42
Financials - P&L Yes Yes 41 How much did the sales grow? 41 Financials - P&L
Yes Yes 42 Net income history 41 Financials - P&L Yes Yes 43 What’s the
forward PE? 41 Stock - Price Yes No 44 Cash flow from financing 41 Financials
- Cash Fl. Yes Yes 45 What’s the I/B/E/S Mean? 40 Stock - Other Analysts Yes
No 46 Investment risks 40 Analysis - Risk No No 47 EV/EBITDA? 40 Stock -
Valuation Yes Yes 48 What’s are the total operations? 39 Financials - P&L Yes
Yes 49 What are the operating expenses? 39 Financials - P&L Yes Yes 50 Who are
the company’s competitors? 39 Market - Competit. No No

Table 5: Continued from the prior table. The table shows the subset of unique
questions ranked by their frequency from 51 to 100.

# Question Count Subcategory Numerical Extractable From Text 51 Whats the
dividend yield/ payout? 38 Stock - Price Yes No 52 What was the FCF and how
did it change? 38 Financials - Cash Fl. Yes No 53 Subsidiaries and partner
companies 37 Company - Corp. Structure No Yes 54 How did cash change over
time? 37 Financials - Cash Fl. No No 55 Whats the correlation? 37 Stock -
Price Yes No 56 How does the company operate? 36 Company - Other No No 57 What
are the CAPEX of these projects? 36 Financials - Cash Fl. No No 58 Consumer
insights 36 Market - Sales & Customers No No 59 Discussion about business
model 35 Market - Other No No 60 Status of ongoing projects 35 Company -
Management No Yes 61 What does the price momentum look like? 35 Stock - Price
Yes No 62 Recent management decisions 35 Company - Management No No 63 What
are the depreciations and amortizations? 34 Financials - P&L Yes Yes 64 What’s
the return on equity? 34 Financials - Other Yes No 65 Is there recent
information, new releases? 34 Company - Other No No 66 Investor composition 33
Company - Ownership Yes Yes 67 How much is the long-term debt? 32 Financials -
Balance Sh. Yes Yes 68 Cash position 32 Financials - Cash Fl. Yes Yes 69 What
are the cost of goods sold? 30 Financials - P&L Yes Yes 70 What does the
global demand look like? 30 Market - Sales & Customers No No 71 What’s the
companies ranking? 29 Stock - Peer Group Yes Yes 72 How much was the Last
Close? 29 Stock - Price Yes No 73 What’s the total equity? 28 Financials -
Balance Sh. Yes Yes 74 What’s the long-term debt/equity? 28 Financials -
Balance Sh. Yes No 75 Dividend growth 28 Financials - Cash Fl. Yes No 76 Net
income growth 27 Financials - P&L Yes No 77 Market cap (floating) 27 Company -
Ownership Yes Yes 78 When does the fiscal year end? 26 Financials - Other Yes
Yes 79 What are the total assets? 26 Financials - Balance Sh. Yes No 80 How
much are the total liabilities? 26 Financials - Balance Sh. Yes Yes 81 Cash at
the year-start 26 Financials - Balance Sh. Yes Yes 82 Operating cash flow 26
Financials - Cash Fl. Yes Yes 83 What are some industry trends? 26 Market -
Other No No 84 How much are the total current liabilities? 25 Financials -
Balance Sh. Yes Yes 85 What’s the price/book ratio? 25 Stock - Price Yes No 86
Key numbers competitive landscape 25 Market - Competit. No No 87 What’s the
volatility? 24 Stock - Price Yes No 88 Differentiators of the company 23
Product No No 89 What companies did the company acquire? 23 Company - M&A No
No 90 Cash relative to total assets 23 Financials - Balance Sh. Yes Yes 91
Current assets relative to total assets 23 Financials - Balance Sh. Yes Yes 92
Short-term debt 23 Financials - Balance Sh. Yes Yes 93 What does the global
supply look like? 23 Market - Supplier & Inputs No No 94 Unique selling
proposition 22 Product No No 95 What do the assets consist of? 22 Financials -
Balance Sh. No No 96 Other current assets relative to total assets 22
Financials - Balance Sh. Yes Yes 97 How much of the assets are in net fixed
assets? 22 Financials - Balance Sh. Yes Yes 98 What does the company’s
distribution look like? 22 Company - Channels & Regions No No 99 Which is the
stock exchange? 22 Company - Base Data No Yes 100 What do the liabilities
consist of? 21 Financials - Balance Sh. No Yes

Table 6: Continued from the prior table. The table shows the subset of unique
questions ranked by their frequency from 101 to 150.

# Question Count Subcategory Numerical Extractable From Text 101 How does the
company reach customers? 20 Market - Sales & Customers No Yes 102 How much of
the assets are in other noncurrent? 20 Financials - Balance Sh. Yes Yes 103
What’s the return on assets? 20 Financials - Other Yes No 104 What are the
competitor’s valuation key figures? 20 Market - Competit. No No 105 What’s the
seasonality? 20 Financials - P&L Yes No 106 What’s the nonoperating income? 19
Financials - P&L Yes Yes 107 How much are the other current liabilities? 19
Financials - Balance Sh. Yes Yes 108 How much are the other noncurrent? 19
Financials - Balance Sh. Yes Yes 109 Competitors’ profitability figures 19
Market - Competit. No No 110 What are the competitor’s growth key figures? 19
Market - Competit. No No 111 What do the company’s subsidiaries do? 19 Company
- Corp. Structure No No 112 What deals were made? 18 Company - M&A No No 113
How much cash was used for acquisitions? 18 Financials - Cash Fl. Yes Yes 114
How much of the assets are in net receivables? 18 Financials - Balance Sh. Yes
Yes 115 What was the Beta? 18 Stock - Price Yes No 116 Country’s global
production share 18 Company - Capabilities Yes Yes 117 Who are the key
executives and what are their titles? 17 Company - Management No Yes 118 How
much of the assets are in inventories? 17 Financials - Balance Sh. Yes Yes 119
What are the competitor’s operations key figures? 17 Market - Competit. No No
120 What are the nonoperating expenses? 16 Financials - P&L Yes Yes 121
Competitors’ per share figures 16 Market - Competit. No No 122 What’s the
score overall? 16 Analysis - Recommend. Yes No 123 What is the facility size?
15 Financials - Balance Sh. Yes No 124 What’s the price/cash flow ratio? 15
Stock - Price Yes No 125 What’s the current ratio? 15 Financials - Balance Sh.
Yes No 126 What’s the LTG Forecast? 15 Financials - P&L Yes No 127 What’s the
dividend rate? 14 Stock - Price Yes No 128 What was the date of the initiated
coverage? 14 Analysis - Base Data No No 129 What are the competitor’s
financial key figures? 13 Market - Competit. No No 130 What’s the PEG? 13
Financials - P&L Yes No 131 What’s the relative strength? 13 Stock - Price Yes
No 132 What’s the company’s headquarters/address? 12 Company - Base Data No
Yes 133 Prescreen score 12 Financials - Other No No 134 How much is the common
stock equity? 12 Financials - Balance Sh. Yes Yes 135 What’s the price/sales
ratio? 12 Stock - Price Yes No 136 How fragmented is the production nation-
wide? 12 Company - Capabilities No No 137 What percentage is owned by
institutions? 11 Company - Ownership Yes No 138 Who is the founder and
chairman? 10 Company - Management No Yes 139 What companies are worth
investing in? 10 Stock - Peer Group No No 140 Company history 9 Company -
Other No No 141 How much was spent on advertising? 9 Company - Management Yes
Yes 142 How much is the preferred stock equity? 9 Financials - Balance Sh. Yes
Yes 143 ROIC? 8 Stock - Valuation Yes Yes 144 R&D costs? 8 Financials -
Balance Sh. Yes Yes 145 What are the continuing operations? 7 Product Yes Yes
146 What are the discontinued operations? 5 Product Yes Yes 147 When did the
company go public? 5 Company - Base Data Yes Yes 148 What’s the employee
growth in percent? 4 Company - Employees Yes Yes 149 What’s the year of
founding? 4 Company - Base Data Yes Yes 150 What bonus did the
managers/executives get? 4 Company - Management Yes Yes

Table 7: Continued from the prior table. The table shows the subset of unique
questions ranked by their frequency from 151 to 169.

# Question Count Subcategory Numerical Extractable From Text 151 Executives’
CVs 4 Company - Management Yes Yes 152 What firm is the accountant of the
company? 3 Company - Bus. Partners No Yes 153 What salary did the
managers/executives get? 3 Company - Management Yes Yes 154 What’s the year of
change in control? 2 Company - Ownership Yes Yes 155 What’s the DUNS Number? 1
Company - Base Data No Yes 156 Employee count total 1 Company - Employees Yes
Yes 157 Employee count at specific location 1 Company - Employees Yes No 158
What’s the primary SIC Code? 1 Company - Base Data No Yes 159 What’s the
primary NAICS Code? 1 Company - Base Data No Yes 160 EVA? 1 Stock - Valuation
Yes Yes 161 Stock price 66 Stock - Price Yes No 162 Potential challenges 65
Company - Other No No 163 Primary industry 64 Product No Yes 164 Earnings per
share 62 Financials - Financing Yes Yes 165 Changes in key figures 62
Financials - Other No No