Learning Discriminative Representations and Generative Approaches for Outdoor 3D LiDAR Data

Abstract

Mobile robots have become part of our everyday lives nowadays to automate tasks that are monotonous, repetitive, or dangerous to humans, such as autonomous driving. The automation of the driving task is complex to achieve due to the highly dynamic and constantly changing scenarios. In such a scenario, the robot's perception system must be robust to deal with the different agents in the scene, such as pedestrians, cyclists, and other vehicles, constantly interacting. At the same time, wrong decisions can have catastrophic impacts. Therefore, a key limitation for achieving autonomous driving comes from the limitations of current perception systems in interpreting the raw sensor data, such as 3D LiDAR sensors. The training of these perception systems requires the annotated dataset to be large in quantity and variability, which is challenging to acquire when considering 3D LiDAR data. <br/>

This thesis divides the challenge of improving perception systems into three main challenges. The first challenge is related to the amount of labeled data available to train perception models, accounting for quantity and variability. The second challenge is related to the classes defined in the annotated data. In unstructured environments, it is impossible to define and label all potentially occurring classes of objects. The behavior of the perception system towards those unusual objects becomes unpredictable, potentially leading to dangerous situations. Finally, the third challenge relates to the domain gap between data collected with different sensors. Different LiDARs often have different laser beam patterns or sensor resolutions, leading to drastic changes in the appearance of the collected scans, limiting the use of annotated data between different LiDARs. <br/>

To address the first challenge, this thesis proposes self-supervised learning strategies to optimize a network for learning a latent representation that can distinguish between coarse segments of objects in the scene without labels. Regarding the second challenge, we propose a new class-agnostic instance segmentation method and a benchmark for evaluating instance segmentation methods in this context. To address the third challenge, this thesis proposes a method to predict a dense and complete point cloud from a single LiDAR scan and an approach to generate novel dense point clouds with semantic annotations to be used as training data. In sum, this thesis tackles those three different challenges to improve perception systems and reduce their dependency on manually annotated datasets.