How to implement multi class classifier for a set of sentences? I'm currently involved in an aspect level sentiment analysis project and using Stanford CoreNLP toolkit to implement the system. My knowledge on these concepts is very limited and I'm seeking your help to clarify some things related to both machine learning and classification.

I have a set of sentences, which are travel reviews that needs to be categorized under different labels (AMBIANCE, SCENERY, COST, ENTERTAINMENT etc). This is done by checking if certain aspect terms related to the aforementioned categories are found in the sentences.

Now, I want to train a classifier (I'm using a Stanford classifier), to classify these sentences to respective categories and I have a train data set containing around 3000+ sentences.

My problem is, a sentence may contain aspect terms that belong to not just one, but multiple aspect categories. In that case, I want the classifier to classify them all at once.

For example:

Review Sentence:

  On the upside it was very **calm** there and good for **swimming** 

Categories

  AMBIANCE, ENTERTAINMENT 


I have tried to pre-process the training dataset in this manner, and trained a classifier. But when I tried to make it classify a sentence containing aspect terms belonging to two different categories, it only identified one category.

Training dataset was of the following format

sentence1    [tab]   category1
sentence2    [tab]   category1, category2
sentence3    [tab]   category2, category3


Can someone please tell me if my approach is wrong? How can I achieve the desired output? I would greatly appreciate any help on this matter, as I'm currently stuck at this phase of my project.
I was thinking if Apriori might be more suited for your purpose.

For your consideration:
1) Tokenize the training sentences into bag of words:
Review Sentence | Upside | Calm | Swimming | Cat

2) Tag the correct consequents for bag of words.

3) Apriori should produce 1 rule for ambiance and 1 rule for Entertainment.

Hope this helps.