How to classify whether text answer is relevant to an initial text question
Both message and answer are your input, so your feature vector should contain information about both.

Here's a simple structure of a possible solution using scikit-learn:

import numpy as np
from sklearn.svm import SVC
from sklearn.feature_extraction import DictVectorizer

dataset = (("Hey @foobar, have you been hacked?",
            "@barfoo it seems so, thx for suggesting",
            True), # True for relevant, False for not relevant
           ("Hey @foobar, have you been hacked?",
            "Lose 20 pounds quickly! http://blabla.com",
            False))

def extractMessageFeatures(message):
    # here comes your real feature extraction algorithm
    return { 'message_predicted_spam': False,
             'message_contains_valid_username': True }

def extractAnswerFeatures(answer):
    # here comes your real feature extraction algorithm
    return { 'answer_predicted_spam': False,
             'answer_contains_valid_username': True }

def extractFeatures(data):
    features = []
    for instance in data:
        instanceFeatures = extractMessageFeatures(data[0])
        instanceFeatures.update(extractAnswerFeatures(data[1]))
        features.append(instanceFeatures)
    return features

def trainClassifier(data):
    features = extractFeatures(data)
    vec = DictVectorizer()
    featureVector = vec.fit_transform(features)

    print vec.get_feature_names()
    print featureVector.toarray()

    svc = SVC()
    svc.fit(featureVector, np.array([i[2] for i in data]))
    return svc

clf = trainClassifier(dataset)

# now, you can clf.predict(...)


Now, the hardest part is to decide which features to extract from both messages and answers. It's up to you.

One of the simplest solutions would be to use n-gram features.

Other approach would be to use some spam detection to decide whether answer is spam or not and treat this information as a feature.

You can also use Twitter-specific information (for example, whether users are mentioning each other in their tweets, using the same hashtags etc.).

You can combine these features in whatever fashion you like, of course.

Except of creating feature extraction functionality you need a labeled dataset of messages, answers and relevant/non-relevant labels.

But once you have both (feature extraction functionality and a proper dataset), you're good to go with a task which clearly matches standard machine learning approaches.