Title: 3D Move to See: Multi-perspective visual servoing for improving object views with semantic segmentation,
Abstract: In this paper, we present a new approach to visual servoing for robotics,
referred to as 3D Move to See (3DMTS), based on the principle of finding the
next best view using a 3D camera array and a robotic manipulator to obtain
multiple samples of the scene from different perspectives. The method uses
semantic vision and an objective function applied to each perspective to sample
a gradient representing the direction of the next best view. The method is
demonstrated within simulation and on a real robotic platform containing a
custom 3D camera array for the challenging scenario of robotic harvesting in a
highly occluded and unstructured environment. It was shown on a real robotic
platform that by moving the end effector using the gradient of an objective
function leads to a locally optimal view of the object of interest, even
amongst occlusions. The overall performance of the 3DMTS method obtained a mean
increase in target size by 29.3% compared to a baseline method using a single
RGB-D camera, which obtained 9.17%. The results demonstrate qualitatively and
quantitatively that the 3DMTS method performed better in most scenarios, and
yielded three times the target size compared to the baseline method. The
increased target size in the final view will improve the detection of key
features of the object of interest for further manipulation, such as grasping
and harvesting.