I am currently experimenting with the idea of including the class of a feature vector as a separate feature. My work is about preposition selection in learner language use. I want to train a classifier that, given a certain position of a sentence, uses relevant context features around that position to build vectors. The classifier detects if there should be a preposition or if it should remain empty. Some features include:Neighbouring N-grams, where N={2,3}, at Part-Of-Speech and token-levelNeighbouring nouns and verbsThe class expresses a binary distribution: 1 for preposition presence and 0 for preposition absence.Consider that if a language learner opted to select a preposition in a given sentence, but the classifier thinks there should NOT be one, it can be interpreted as the classifier marking a language error: The learner inserted a preposition in a prohibited context. Conversely, if a learner did NOT use a preposition at a given position in a sentence, but the classifier does expect one, it is also an error.The model is trained on correct negative (no preposition) and positive (yes preposition) events (context features around a certain position: a whitespace or a preposition). This means that the assigned labels are the gold standard, similarly for the test data, because they are trained on *native *language use which is assumed to be error-free. The same assumption cannot be made when running the classifier on learner's language - after all, learners make mistakes, which is what I am trying to detect. I want to include the decision (yes/no) of the learner into the classifier rather than just as the assigned label. Because in practice, the chance on either error is very small, so the classifier should be very confident before marking it as such. But when I do this, the classifier always selects the value of that decision feature as the class. After all, it is trained so that there is a 100% correlation between said feature and the assigned class. How do I turn this learner's decision - which is NOT an absolute indication of the actual answer - into an indication instead?I'm experimenting with Linear Support Vector Classifier and Logistic/Linear Regression.Thanks a bunch, if more details are needed, please ask.
How to include class as a feature
machine learning;feature selection