How to identify statistically distinct features of different sets?
This looks like a standard machine learning problem.  You could use any machine learning technique.  You might start with Naive Bayes.

If you want to evaluate a single feature, you could use information gain or BIC.

For the combination of all features, you can use a machine learning algorithm.  As I mentioned, I would suggest trying Naive Bayes first.  If you need something more powerful, there are many other classifiers: random forests, SVM's, k-nearest neighbors.  Read a textbook on machine learning to learn more about the subject.