Context-aware Human Motion Anticipation

Abstract

This thesis addresses the challenges of human motion anticipation and evaluation in complex contexts such as social interactions and human intention, focusing on motions lasting beyond one second. Central to our approach is the understanding of human motion in relation to context, categorized into historical motion data, underlying intentions, social interactions, and scene constraints. We introduce novel methodologies and a comprehensive dataset to facilitate advancements in this field. Our contributions include two generative models: Intention RNN and Social Diffusion. Intention RNN is an adversarially trained recurrent neural network which first forecasts discrete intention signals and then forecasts human motion based on the intention signal. Social Diffusion adopts a diffusion-based approach to predicting social motion dynamics - the motion of multiple humans socially interacting - and introduces a simple yet effective summarization function to model an arbitrary number of persons. Both models account for the complexity and stochasticity inherent in human motion and allow for forecasting horizons way beyond one second. Furthermore, we present a multi-person human motion dataset, "Humans in Kitchens", featuring natural interactions in kitchen environments. This dataset is instrumental in providing accurate 3D human motion tracking and annotated scene geometry, offering a rich resource for understanding complex human activities. Additionally, we propose two novel evaluation metrics: Normalized Directional Motion Similarity (NDMS) for assessing the quality of individual human motions and Symbolic Social Cues Protocol (SSCP) for evaluating social interactions. These metrics address the limitations of existing evaluation methods and provide a more nuanced understanding of motion quality and realism.