Increase accuracy of classification problem I am trying to build a classifier that predicts the compiler given some operations of assembly code. Here is the pandas dataframe:



What I do is using a TfidfVectorizer and select the features that have most predictive power by doing:

tfidf_vectorizer=TfidfVectorizer(max_features=500)


so using max_features=500 to select the 500 features with the highest idf. The problem is that the accuracy is still low, infact it is around 0.69. I would like to arrive at least at 0.9, but I dont't know what else do.

I am using support vector machines and this gives me the accuracy of 0.69. I also tried random fores and I was around 0.75.

My code is the following:

from sklearn.feature_extraction.text import TfidfVectorizer 

tfidf_vectorizer=TfidfVectorizer(max_features=1000)
df_x = df['opcodes']


X_all = tfidf_vectorizer_vectors=tfidf_vectorizer.fit_transform(df_x)
y_all = df['compiler']


X_train, X_test, y_train, y_test = train_test_split(X_all, y_all, 
      test_size=0.2, random_state=15)

from sklearn import svm

model = svm.SVC(kernel='linear', C=1).fit(X_train,y_train)

y_pred = model.predict(X_test)

acc = model.score(X_test, y_test)    
print("Accuracy %.3f" %acc)


moreover the dataset is balanced, infact I have 3 compilers with 1000 samples each.

I don't know what other startegy to try to increase accuracy and get at 0.9. 

Can somebody please help me?  Thank's in advance.
I'm not familiar with scikit but I'm assuming that TfidfVectorizer represents bag of words features right? By this I mean that it treats all the instructions in an instance as a set, i.e. doesn't take into account their sequential order.

I'm also not familiar with compilers but I'm guessing that the order of the instructions could be a relevant indication? I.e. a compiler may generate particular sequences of instructions.

Based on these remarks I would try to represent instances with n-grams of instructions rather than individual instructions. Then you can still use some kind of bag-of-ngrams representation, possibly with TFIDF, but I would start with simple binary or frequency features. A simple feature selection step with something like information gain might be useful.

[edit] N-grams take order into account locally. In a bag of words model, words (or instructions in your case) are considered individually of each other: for instance the sequence push, push, mov is the same as push, mov, push. With bigrams this sequence would be represented as (push,push), (push,mov) whereas the second one is (push,mov), (mov,push). This means two things:


Higher level of detail about the instance, which can help the model capture the relevant indications
More features so higher risk of overfitting (the model taking some random details as indication, which lead to errors on the test set).