Multi threading using Python and pymongo
Question: Hi im looking to make a program that will class tweets positively and
negatively classifies tweets about a company already saved in a mongodb and
once classified, to update a integer based on then result.

I have code made to make this possible but i would like to multi-thread the
program but I have had no experience of this in python whatsoever and have
been trying to follow tutorials with no luck as the program just starts and
exits without going through any of the code.

If anyone could help me with this it would be much appreciated. The code for
the program and the intended multi-threading is below.

    
    
    from textblob.classifiers import NaiveBayesClassifier
    import pymongo
    import datetime
    from threading import Thread
    
    train = [
    ('I love this sandwich.', 'pos'),
    ('This is an amazing place!', 'pos'),
    ('I feel very good about these beers.', 'pos'),
    ('This is my best work.', 'pos'),
    ("What an awesome view", 'pos'),
    ('I do not like this restaurant', 'neg'),
    ('I am tired of this stuff.', 'neg'),
    ("I can't deal with this", 'neg'),
    ('He is my sworn enemy!', 'neg'),
    ('My boss is horrible.', 'neg'),
    (':)', 'pos'),
    (':(', 'neg'),
    ('gr8', 'pos'),
    ('gr8t', 'pos'),
    ('lol', 'pos'),
    ('bff', 'neg'),
    ]
    
    test = [
    'The beer was good.',
    'I do not enjoy my job',
    "I ain't feeling dandy today.",
    "I feel amazing!",
    'Gary is a friend of mine.',
    "I can't believe I'm doing this.",
    ]
    
    filterKeywords = ['IBM', 'Microsoft', 'Facebook', 'Yahoo', 'Apple',   'Google', 'Amazon', 'EBay', 'Diageo',
                  'General Motors', 'General Electric', 'Telefonica', 'Rolls Royce', 'Walmart', 'HSBC', 'BP',
                  'Investec', 'WWE', 'Time Warner', 'Santander Group']
    
    # Create pos/neg counter variables for each company using dicts
    vars = {}
    for word in filterKeywords:
    vars[word + "SentimentOverall"] = 0
    
    
    # Initialising the classifier
    cl = NaiveBayesClassifier(train)
    
    
    class TrainingClassification():
        def __init__(self):
            #creating the mongodb connection
            try:
                conn = pymongo.MongoClient('localhost', 27017)
                print "Connected successfully!!!"
                global db
                db = conn.TwitterDB
            except pymongo.errors.ConnectionFailure, e:
                print "Could not connect to MongoDB: %s" % e
    
            thread1 = Thread(target=self.apple_thread, args=())
            thread1.start()
            thread1.join()
            print "thread finished...exiting"
    
        def apple_thread(self):
            appleSentimentText = []
            for record in db.Apple.find():
                if record.get('created_at'):
                    created_at = record.get('created_at')
                    dt = datetime.strptime(created_at, '%a %b %d %H:%M:%S +0000 %Y')
                    if record.get('text') and dt > datetime.today():
                        appleSentimentText.append(record.get("text"))
            for targetText in appleSentimentText:
                classificationApple = cl.classify(targetText)
                if classificationApple == "pos":
                    vars["AppleSentimentOverall"] = vars["AppleSentimentOverall"] + 1
                elif classificationApple == "neg":
                    vars["AppleSentimentOverall"] = vars["AppleSentimentOverall"] - 1
    


Answer: The main issue with your code is here:

    
    
    thread1.start()
    thread1.join()
    

when you call join on a thread, it has the effect of making the current
running thread (in your case, the main thread) wait until completion of the
thread (here, thread1). So you can see that your code will actually not be
faster. It just launches one thread and waits for it. It will actually be
marginally slower because of the thread creation.

Here's a proper way to do multithreading:

    
    
    thread1.start()
    thread2.start()
    thread1.join()
    thread2.join()
    

In this code, the threads 1 and 2 will both run in parallel.

IMPORTANT: Note that in Python, it is a "simulated" parallelisation. Because
Python's core is not thread safe (mainly because of the way it does garbage
collection), it uses the GIL (Global Interpreter Lock), and therefore all
threads in a process run on only 1 core. If you are keen on using real
parallelisation (for example if your 2 threads are CPU bounds rather than I/O
bounds), then look at the multiprocessing module.

