1. A method for similarity metric learning for multimodal medical image data, the method comprising: receiving a first set of image data of a volume, wherein the first set of image data is captured with a first imaging modality; receiving a second set of image data of the volume, wherein the second set of image data is captured with a second imaging modality; aligning the first set of image data and the second set of image data; training a first set of parameters with a multimodal stacked denoising auto encoder to generate a shared feature representation of the first set of image data and the second set of image data; training a second set of parameters with a denoising auto encoder to generate a transformation of the shared feature representation; initializing, using the first set of parameters and the second set of parameters, a neural network classifier; and training, using training data from the aligned first set of image data and the second set of image data, the neural network classifier to generate a similarity metric for the first and second imaging modalities.