Spatio-Temporal Registration Techniques for Agricultural Robots

Abstract

A critical challenge that we face today is to meet the rising demand for food, feed, fiber, and fuel from an ever-growing world population. We must meet this demand within the limited arable land available to us and do so in the aggravated situation caused by climate change. Moreover, present-day levels of agro-chemical usage is unsustainable. They lead to large scale environmental pollution and adverse effects on the biodiversity of our planet. A promising way to meet this challenge is through intensifying production sustainably using existing resources and novel technology in combination. Robotic systems deployed in agricultural fields are seen as a potential solution to achieve this goal. These systems can increase productivity by providing high-quality site-specific treatment at the level of an individual plant through continuous monitoring and timely intervention in the field, while drastically reducing or eliminating the use of agro-chemicals. The development of such automated robotic systems is envisioned to play an essential role in the future of agricultural plant production. <br /> Agricultural robots are ideal platforms to monitor the plants in the field with a high spatial and temporal frequency and provide intervention capability whenever an action is required. In this thesis, we focus on the fundamental task of registration, which would form the core of such robotic systems. The goal of registration is to bring two sets of measurements into a common coordinate frame, which forms the basis for associating data separated in space and time. It is a core building block for solving several state estimation problems in robotics, geodesy, and photogrammetry. As a result, registration of sensor data has been extensively studied in the literature from multiple disciplines. However, existing techniques fail to perform reliably in the agricultural domain due to a unique set of challenges. These challenges vary from the large change in the visual appearance of the field over time to the structural change of individual plants as they grow over the crop season and to the vastly differing viewpoints where data is captured from multiple platforms in an aerial-ground robotics system. <br /> Our main contribution in the thesis is a set of novel registration techniques that explicitly considers the challenges brought forward by the spatio-temporal nature of the task in agricultural application. We show that our registration techniques perform reliably in challenging conditions and demonstrate their advantages over state-of-the-art registration approaches. We use these registration techniques to demonstrate their application for long-term monitoring of crops in the field, for accurate localization of ground robots for navigation in crop fields, and for performing automated phenotyping to analyze the growth of individual plant parts from high-fidelity point cloud data. We also study the effect of outliers in data for registration and state estimation problems and propose a general solution for robust state estimation in the presence of different outlier distributions that occur in these tasks. The registration techniques developed in this thesis contribute to the robust operation of autonomous robots in crop fields over long periods of time and form the backbone of applications interested in tracking spatio-temporal traits of plants. <br /> In sum, this thesis makes several contributions in the context of spatio-temporal registration in the agricultural domain of plant production. Compared to the current state-of-the-art, the approaches presented in this thesis allow for a more robust and longer-term registration of data captured by robots in the fields and effectively handle the challenges resulting from plant growth. All approaches described in this thesis have been published in peer-reviewed conference papers and journal articles. In addition to that, we have released most of the techniques developed in this thesis as open-source software and also published three challenging datasets for long-term spatio-temporal registration tasks.