Title: A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learning,
Abstract: Insertion is a challenging haptic and visual control problem with significant
practical value for manufacturing. Existing approaches in the model-based
robotics community can be highly effective when task geometry is known, but are
complex and cumbersome to implement, and must be tailored to each individual
problem by a qualified engineer. Within the learning community there is a long
history of insertion research, but existing approaches are typically either too
sample-inefficient to run on real robots, or assume access to high-level object
features, e.g. socket pose. In this paper we show that relatively minor
modifications to an off-the-shelf Deep-RL algorithm (DDPG), combined with a
small number of human demonstrations, allows the robot to quickly learn to
solve these tasks efficiently and robustly. Our approach requires no modeling
or simulation, no parameterized search or alignment behaviors, no vision system
aside from raw images, and no reward shaping. We evaluate our approach on a
narrow-clearance peg-insertion task and a deformable clip-insertion task, both
of which include variability in the socket position. Our results show that
these tasks can be solved reliably on the real robot in less than 10 minutes of
interaction time, and that the resulting policies are robust to variance in the
socket position and orientation.