Inception/ResNet doing worse than SIFT in feature extraction
After examining the dataset, we found that the problem was in NUSWIDE dataset itself. Almost half of the dataset didn't have supervised labels (81 labels entered by humans). Also we weren't training on the whole dataset, rather, we were taking random samples because some methods required that the whole dataset was loaded in memory which wasn't feasible. 
After knowing that, we deleted the tuples that didn't have supervised labels, extracted features and done PCA and got a comparable results to SIFT (improved, in some cases). 
So my advice if you face a similar problem is:


Make sure that your data labels are valid.
Use multiple layers for feature extraction, not just the deeper ones in the network.
Using PCA instead of random sampling dataset worked very well for us.