1. A computer-implemented method for performing object recognition comprising: receiving image data; extracting a depth image and a color image from the image data; creating a mask image by segmenting the image data into a plurality of components; identifying objects within the plurality of components of the mask image; determining a first likelihood score from the depth image and the mask image using a layered classifier; determining a second likelihood score from the color image and the mask image by generating an object image by copying pixels from a first image of the components in the mask image and classifying the object image using a deep convolutional neural network (CNN); and determining a class for at least a portion of the image data based on the first likelihood score and the second likelihood score.