Deep Representation Learning for Financial Document Analytics

Abstract

In this thesis, we leverage machine learning (ML) methods primarily based on deep neural networks to drastically reduce the manual work of financial analysts, investors, auditors, and other stakeholders by automating key steps in their analysis of financial disclosure documents. A core challenge in this context is transforming highly unstructured and inherently discrete textual data into meaningful numerical representations that ML models can effectively interpret. We refer to this automated conversion process as representation learning and remark that the learned representations must encode the text's syntactic, semantic, and contextual structure. <br /> We develop novel methodologies utilizing deep representation learning to improve the efficiency and quality of several financial document analysis tasks. First, we introduce an approach for the joint extraction, linking, and consistency checking of Key Performance Indicators (KPIs) from corporate disclosure reports. By fine-tuning a bidirectional text encoder neural network with classification heads for named entity recognition and relation extraction, we efficiently extract KPIs and predict their interrelationships. Building upon this, we enhance the detection of numerical inconsistencies between semantically equivalent KPIs using contrastive learning techniques. This includes joint sentence and table encoding and a contrastive autoencoder classification module, along with a filtering mechanism employing cross-attention to handle data imbalance from numerous unrelated KPI pairs. <br /> To assist auditors in aligning regulatory requirements with relevant sections of financial reports, we introduce a context-aware recommender system designed to retrieve the most pertinent text passages in sustainability reports. The system utilizes a Transformer-based encoding module with a non-linear multi-label classification head, trained end-to-end. Recognizing the limitations of processing paragraphs in isolation, we propose a novel pre-training methodology called Pointer-Guided Segment Ordering, which enhances the model's ability to generate contextually rich paragraph embeddings by understanding narrative flow and inter-paragraph relationships. <br /> Addressing the dynamic nature of accounting standards, we propose a flexible compliance check methodology using Large Language Models (LLMs). We combine a fine-tuned semantic text matching model with an LLM-based re-ranking module, enabling zero-shot matching between financial reports and potentially unseen legal requirements. We further integrate a compliance verification component that employs zero- and few-shot learning with prompting techniques like chain-of-thought to assess compliance with disclosure requirements from international accounting standards. <br /> Lastly, we develop a specialized LLM-powered chatbot with an optimized Retrieval-Augmented Generation pipeline to support compliance with Risk Management and Quality (R&Q) standards. By integrating hybrid search techniques and relevance boosting, the system enhances retrieval accuracy and provides precise and contextually appropriate answers to queries related to R&Q standards, aiding employees in accessing and interpreting complex regulatory information.