Identification of the Most Frequently Asked Questions in Financial Analyst Reports to Automate Equity Research Using Llama 3 and GPT-4

This is the first systematic analysis of the structure of equity research reports (ERRs). The paper uncovers the automation potential of ERRs, and validates their automation potential with the state-of-the-art language models GPT-4 and Llama-3-70B.

The study examines 72 ERRs, categorizing 4,964 sentences into 169 unique question archetypes across five main categories: Financials, Company, Product, Stock, and Market. The researchers classify each question based on its potential for automation, distinguishing between text-extractable, database-extractable, and non-extractable information.

Key findings include:

– 78.7% of the questions in ERRs are potentially automatable.
– 48.2% of questions are text-extractable (suited for processing by large language models).
– 30.5% of questions are database-extractable.
– Only 21.3% of questions require human judgment to answer.

The study validates these findings using two large language models: Llama-3-70B and GPT-4-turbo-2024-04-09. The results show that these models can extract relevant information from annual reports for a significant portion of the questions, with potential for even higher accuracy when used in combination.