1. A method for determining a hemodynamic metric to assess a stenosis of interest for a patient, comprising: receiving a medical image of the patient including the stenosis of interest; detecting image patches corresponding to the stenosis of interest and a coronary tree of the patient; and determining a value of at least one hemodynamic metric for the stenosis of interest using a trained deep neural network regressor applied directly to the detected image patches without first extracting features from the medical image, wherein the trained deep neural network regressor is trained using a first set of training image patches without corresponding values of the at least one hemodynamic metric to train weights of layers other than a final layer of the deep neural network regressor and a second set of training image patches including identified stenosis image patches and corresponding values of the at least one hemodynamic metric to refine the weights of all layers including the final layer of the deep neural network regressor.