Robotics in Horticulture: From Sensor Data to Autonomous Harvesting

Abstract

Our agricultural production system has to cope with an increased demand for food, feed, fuel, and fiber in a range between 30% and 60% caused by a rapidly growing world population, set to reach 10 billion in 2050, and changing life style. At the same time, we need to overcome a set of challenges caused by climate change and our unsustainable farming practices. First, unsustainable agricultural practices, such as the intensive use of agrochemicals, led to salinization, desertification, and soil erosion, resulting in a substantial loss of arable land. Second, developed countries have been increasingly concerned about labor scarcity due to an ageing workforce paired with a low number of new entrants in the sector. As a result, our farming system would need to increase the yield per area unit relying on seasonal workers, often largely represented by migrant labor. Within such a frame, the need to rethink our entire production systems in terms of sustainability and automation stands out clearly. <br /> Robotic systems offer an alternative perspective to standard agricultural practices along the whole production line. Robots can take over labor-intensive tasks for which finding workers is getting harder and harder, such as pruning and harvesting; robots can reduce agrochemical inputs by means of targeted weeding instead of uniformly spraying the entire field; robots can continuously monitor fields at a large scale to provide measurements about the state of plants that breeders can use to develop more resilient crop varieties. Finally, robots can facilitate the implementation of different growing techniques, such as vertical farming, to increase yield per unit area and reduce transportation from farmers to customers. <br /> Labor shortage is one of the main concerns in horticultural production systems for high-quality crops, which strongly rely on human workers. To date, human workers are still more productive and efficient of robotic solutions in tasks such as pruning, thinning, and harvesting due to the challenging environment and the dexterity needed to handle plants without harming them. <br /> To be effective across such a diverse list of tasks, robots need robust and accurate perception systems able to cope with domain-specific challenges to improve the robot's awareness of its surroundings. To name a few, agricultural environments are naturally cluttered, meaning obtaining a complete observation of the robot's surroundings is challenging. Additionally, agricultural environments present a large number of variations caused by plant growth, light conditions, soil state, and growing techniques used by the farmers. Such challenges represent a limitation for the widespread adoption of robotic technologies as their efficacy differs based on the ability of the robot to perceive its working environment. <br /> The main contributions of this thesis are novel perception systems for agricultural robots across different tasks with a strong focus on horticulture. First, we introduce computer vision algorithms for estimating the 3D shape of plants and fruits when only a partial and often noisy observation is available. Second, we develop techniques to better understand and monitor plants and fruits by finding data associations in point clouds obtained at different points in time. Third, we propose an approach to adapt image segmentation models that distinguish pixels belonging to different semantic classes to new, unseen environments. Finally, we integrate these novel vision techniques to complete 3D shapes in a robotic harvesting pipeline implemented on a vertical farm. <br /> In summary, this thesis contributes to different research directions, including semantic, geometric, and temporal interpretation of sensor data in agricultural robotic systems. Such contributions culminate with the integration of advanced perception techniques into an autonomous robotic harvesting pipeline that proved more general and more robust than the current state-of-the-art. The computer vision approaches presented in this work allow for a more precise estimation of 3D plants and fruits geometry, as well as improving spatio-temporal understanding of their development while showing how to adapt such vision systems to new environments. The techniques presented in this thesis contribute to robotic systems that can more robustly deal with the challenges posed by agricultural environments. Thus, they make a concrete step toward a food production system that relies less on human workers and more on autonomous systems.