Efficient Learning and Optimization for Robotic Manipulator Motion Generation

Abstract

A central goal of robotics research is to develop autonomous systems capable of achieving and ultimately surpassing human-level task performance in unstructured environments. Robotic manipulation plays a critical role in this aspiration. In this thesis, we address the problem of robotic manipulator motion generation with data-driven and optimization-based approaches. The high-dimensional state spaces with complex underlying dynamics and real-time operational constraints pose major challenges. Our methods address these problems and provide solutions to a sequence of interconnected tasks. These include planning and tracking manipulator trajectories followed by dexterous object manipulation. <br/> First, a feed-forward open-loop reference correction policy improves the joint trajectory tracking accuracy. The policy is learned offline in a supervised manner on a small real-world dataset. We propose to incorporate a hardwired one-step future prediction into the model to facilitate planning behavior. Next, we introduce a methodology for learning a closed-loop policy with deep reinforcement learning directly on the real robot. Our policy leverages the advantages of online feedback to significantly improve trajectory tracking accuracy. <br/> Second, we address dual-arm trajectory optimization with multiple constraints. We propose an obstacle cost function based on the estimation of the worst-case overlap volume. Additionally, we handle the closed kinematic chain constraint by subdividing the system into active and passive sub-chains, with an implicit redundancy resolution for the passive sub-chain. These components significantly decrease the method's runtime when optimizing high-dimensional dual-arm trajectories. <br/> Third, we propose a method for learning dexterous pre-grasp manipulation for functional grasping using a human-like hand. The policy is trained with deep reinforcement learning. Our dense multi-component reward function and curriculum avoid the need for expert demonstrations and other costly data collection processes. We propose two target grasp representations and analyze their effects on the behavior of the policy. The policy quickly learns to dexterously manipulate novel object instances of known categories and achieve provided functional grasps that enable object use, such as operating a drill. <br/> We showcase the effectiveness of our methods in simulation and real-world experiments. Our approaches significantly improve trajectory tracking accuracy, quickly generate high-dimensional trajectories that satisfy multiple constraints, and dexterously manipulate complex objects using a human-like hand.