In this paper, an automated twistlock handling robot system is proposed with the aim to perform tedious and danger works with safe efficient operations. The proposed robot system is composed of an unmanned ground vehicle, a human friendly lightweight robot manipulator, a robot gripper with three fingers, and a 3D Kinect sensor for object recognition. A segmentation algorithm has been developed and 3D models have been elaborated for different twistlocks. Then, a library has been built for automatic object recognition and registration purposes. An architecture design of the software module is introduced and a full cycle of twistlock handling operation is illustrated. Simulations are reported in order to evaluate feasibility and operation performances of the system.