1. An apparatus of controlling a robot, the apparatus comprising: an image obtaining unit configured to obtain a three-dimensional image of a user; a driving unit configured to drive an arm of the robot that is composed of a plurality of segments; and a control unit configured to generate a user model that corresponds to a motion of the joint of the user based on the three-dimensional image, to generate a target model having a length of the segment that varies based on the user model, and to allow the arm of the robot to be driven based on the target model.