Linking Local and Global Explanations for a Human-Centered Model Understanding in AI Assessments

Abstract

This thesis explores the challenges of connecting local and global explanations and develops solutions for realizing this link to enable a human-centered understanding of AI models. To validate the trustworthy use of AI models in an AI assessment, it is essential to understand the behavior of these complex and opaque models. Most existing transparency and explainable AI methods therefore attempt to explain the opaque inner workings of AI models by providing either local or global insights. However, local methods can overwhelm analysts with numerous single input explanations without context, leading to misinterpretation, while global methods provide simpler explanations through aggregation, but at the cost of important details. Consequently, achieving a detailed understanding of model behavior while maintaining scalability of the highly manual assessment process is challenging. <br/>

To address this gap, this research takes the approach of establishing the linkage between local and global explanations from both directions. The goal is to provide detailed, <em>i</em>.<em>e</em>, local, insights into model behavior while connecting to global properties to handle large datasets. The contributions of this thesis are as follows. <br/> First, for the "global to local" perspective, the visual analytics tool ScrutinAI is developed. Its iterative workflow is designed to empower analysts to identify and interpret patterns in model behavior by utilizing their domain knowledge. For scalability of the analyses, the tool initially offers a global view of data, metadata and model predictions, which can be drilled down to data point groups or individual data points as required by the analyst. Along two case-studies from the automotive domain, this work shows that ScrutinAI has helped analysts to first efficiently formulate hypotheses on a data-global level. In the subsequent detailed analysis, they were then able to uncover systematic weaknesses, such as poor model performance for certain object characteristics such as size. <br/> The second perspective, "local to global", is addressed by the introduction of the Concept Explanation Clusters (CEC) approach. CEC utilizes individual local explanations to derive global explanations in terms of human-understandable feature combinations. To this end, CEC identifies regional clusters of similar cases, where similarities are based on patterns of significant features and input data. The effectiveness and applicability of CEC across various models and data types are demonstrated using standard textual and tabular datasets widely recognized in the research community. The results confirm that CEC allows for efficient identification of regional class-wise concepts (<em>e</em>.<em>g</em>., focus on a specific product) as well as dedicated sub-concepts per cluster (<em>e</em>.<em>g</em>., each focus on different aspects of that product). These regional concepts provided important context that facilitated the understanding of the model behavior. Additionally, CEC enabled the analysis of outlier clusters and misclassifications, which provided valuable insights into the weaknesses of the investigated models. <br/>

By systematically focusing on enhancing transparency in the context of AI assessments, this thesis establishes a link between local and global explanations from both directions. The contributions not only support comprehensive evaluations of AI models but also foster a deeper understanding of their behavior.