Classification of Conversations in Text
There are multiple techniques that help with the problem you sketch, the applicability of which usually depends on the classification technique and the corpus. But I get the idea you would be helped by some practical examples. So let's go over some of them. Feel free to comment if I tread over familiar grounds, or you want me to elaborate on some of them. Or where to start experimenting with them


Stopping A simple technique is applying a stoplist: A list of common words that should be removed. There are pre-packaged lists, but most packages allow you to provide your own.
TF/IDF A technique that transforms your features by weighing words by their term frequency (how often do they occur in the document) divided by the document frequency (how often does the word occur in other documents. This way frequent words are made less relevant to the document
POS Many packages offer you a part-of-sentence-tagger, that will tag the words by their grammatical function (Verb for instance). You can leverage that in the tokenization step to filter out words (usually you'll look for verbs and nouns). Some vectorizers can do this straight out. (This could also be done with NER)
Stemming transforms inflections of words (eg: train/trains) to a stem. This might make some of your words a little more relevant by upping the chance a pair of them collides 
Restricting your vectorizer: Most packages sport a vectorizer that you can instruct to either look for a minimum/maximum document count (ignore words that either occur in to many different documents, or to little different documents), or to cap the amount of features (words). Capping the amount of words usually selects for most frequently used words.
Encoding word/token based features to more semantic features: Word2Vec, but also older techniques such as LDA/LSI.
Picking the classification algorithm: Some algorithms are very capable of handeling large feature spaces (Naive Bayes for instance), some algorithms learn to transform the feature space to find better ways to weigh the features.


Packages such as Sklearn, NLTK and Gensim offer most of these techniques.

Let me know if this was helpful