Context-aware Deep Learning in Medical Image Analysis

Abstract

Deep Learning (DL) has demonstrated outstanding performance in the area of medical image analysis, with models achieving levels of accuracy that may exceed those of human experts across various applications. This highlights the significant potential of DL-based diagnostic decision support systems to optimize clinical workflows and improve patient outcomes. However, challenges remain that limit the clinical impact and advancements of DL-based systems, such as the need of interpretability or extensive labeled data sets. Since annotation typically requires expert knowledge, generating large-scale annotated data sets presents significant time- and cost constraints. In the following thesis, we address these challenges by incorporating prior contextual information into the DL process, a method known as context-aware DL. Specifically, we focus on two types of contextual information: expert knowledge and prior insights regarding the label quality. The following open research questions are addressed to support the advancement of DL-based decision support systems: 1) Can expert knowledge be leveraged to mitigate current challenges in medical image analysis, and if so, how? 2) Is it possible to utilize contextual information to attain good performance in the multi-label classification of medical image data despite a substantial proportion of missing labels, and if so, how? 3) Is it feasible to integrate contextual information about the label quality to enhance performance when handling data sets with label noise, and if so, how? <br /> Firstly, we integrate expert knowledge regarding the elements of bilateral symmetry of the lung fields into the DL method to automatically detect lung diseases in chest X-ray data. The symmetry-aware architectures and loss function surpass state-of-the-art data-driven DL baselines, enhancing interpretability and data efficiency. To further investigate the potential of expert knowledge as contextual insight, we examine the context-aware analysis of lumbar spine magnetic resonance imaging scans. We demonstrate that contextual information can be integrated through a two-step process. The given work presents an expert knowledge system that utilizes data-driven segmentation masks of the most crucial entities, enabling interpretable diagnostic decision support. These methods emphasize the benefits of context-aware DL based on expert knowledge to address crucial challenges in the medical image analysis field. <br /> Moreover, we focus on the contextual information based on prior insights regarding the label quality to address missing labels or label noise. Proposed novel loss functions achieve high classification performance, surpassing state-of-the-art methods when handling single positive multi-label training for medical images. Additionally, we present a context-aware pre-training strategy to efficiently utilize automatically generated labels, significantly reducing the annotation costs. The context-aware method surpasses purely data-driven training (e.g. not distinguishing between manually and automatically generated labels), highlighting the potential of context-aware training pipelines. <br /> Finally, we introduce a novel context-aware loss function, that achieves remarkable performance even in the presence of label noise. This given context-aware loss enables the abstaining of potentially noisy samples by integrating insights about the expected label noise as a form of regularization. The presented work underscores the significant potential of context-aware DL to mitigate the adverse effects of missing labels or label noise. <br /> In addition to the introduction of novel context-aware DL methods that address the relevant research questions, we tackle further significant shortcomings in the domain of medical image analysis. We evaluate both proposed and state-of-the-art DL methods within this field, with a particular emphasis on German patient cohorts. This work bridges state-of-the-art DL research and the crucial area of medical image analysis, presenting initial investigation of relevant DL methods for the domain. Consequently, our work enhances the assessment of applicable DL methods for real-life medical image use cases, ultimately improving their potential clinical impact. <br /> Overall, this thesis contributes to the advancement of context-aware DL-based diagnosis decision support systems, aiming to alleviate the workload of medical professionals in their daily practice while enhancing patient outcomes.