Why additional memory allocation makes multithread Python application work few times faster?
Question: I'm writing python module which one of the functions is to check multiple IP
addresses if they're active and write this information to database. As those
are I/O bound operations I decided to work on multiple threads:

  * 20 threads for pinging host and checking if it's active (function `check_if_active`)
  * 5 threads for updating data in database (function `check_if_active_callback`)

Program works as followed:

  1. Main thread takes IPs from database and puts them to queue `pending_ip`
  2. One of 20 threads takes record from `pending_ip` queue, pings host and puts answer to `done_ip` queue
  3. One of 5 threads takes record from `done_ip` queue and does update in database if needed

What I've observed (during timing tests to get answer how many threads would
suit the best in my situation) is that program works aprox. 7-8 times faster
if in 5 loops I first declare and start 20+5 threads, delete those objects and
then in 6th loop run the program, than if I run program without those
additional 5 loops.

I suppose this could be somehow related to memory management. Not really sure
though if deleting objects makes any sense in python. My questions are:

  * why is that happening?
  * how I can achieve this time boost without additional code (and additional memory allocation)?

Code:

    
    
    import time, os
    from threading import Thread
    from Queue import Queue
    from readconfig import read_db_config
    from mysql.connector import Error, MySQLConnection
    pending_ip = Queue()
    done_ip = Queue()
    
    class Database:
        connection = MySQLConnection()
        def connect(self):
            db_config = read_db_config("mysql")
            try:
                self.connection = MySQLConnection(**db_config)
            except Error as e:
                print(e)
        def close_connection(self):
            if self.connection.is_connected() is True:
                self.connection.close()
        def query(self, sqlquery):
            if self.connection.is_connected() is False:
                self.connect()
            try:
                cursor = self.connection.cursor()
                cursor.execute(sqlquery)
                rows = cursor.fetchall()
            except Error as e:
                print(e)
            finally:
                cursor.close()
                return rows
        def update(self,sqlquery, var):
            if self.connection.is_connected() is False:
                self.connect()
            try:
                cursor = self.connection.cursor()
                cursor.execute(sqlquery, var)
                self.connection.commit()
            except Error as e:
                self.connection.rollback()
                print(e)
            finally:
                cursor.close()
    
    db=Database()
    
    def check_if_active(q):
        while True:
            host = q.get()
            response = os.system("ping -c 1 -W 2 %s > /dev/null 2>&1" % (host))
            if response == 0:
                ret = 1
            else:
                ret = 0
            done_ip.put((host, ret))
            q.task_done()
    
    
    def check_if_active_callback(q, db2):
        while True:
            record = q.get()
            sql = "select active from table where address_ip='%s'" % record[0]
            rowIP = db2.query(sql)
            if(rowIP[0][0] != record[1]):
                sqlupdq = "update table set active=%s where address_ip=%s"
                updv = (record[1], record[0])
                db2.update(sqlupdq, updv)
            q.task_done()
    
    
    def calculator():
        #some irrelevant code
        rows = db.query("select ip_address from table limit 1000")
        for row in rows:
            pending_ip.put(row[0])
        #some irrelevant code
    
    if __name__ == '__main__':
        num_threads_pinger = 20
        num_threads_pinger_callback = 5
        db = Database()
        for i in range(6):
            db_pinger_callback =[]
            worker_p = []
            worker_cb = []
            #additional memory allocation here in 5 loops for 20 threads
            for z in range(num_threads_pinger):
                worker = Thread(target=check_if_active, args=(pending_ip))
                worker.setDaemon(True)
                worker.start()
                worker_p.append(worker)
            #additional memory allocation here in 5 loops for 5 threads
            for z in range(num_threads_pinger_callback):
                db_pinger_callback.append(Database())
                worker = Thread(target=check_if_active_callback, args=(done_ip, db_pinger_callback[z]))
                worker.setDaemon(True)
                worker.start()
                worker_cb.append(worker)
    
            if i == 5:
                start_time = time.time()
                calculator()
                pending_ip.join()
                done_ip.join()
                print "%s sec" % (time.time() - start_time)
    
            #freeing (?) that additional memory
            for z in range(num_threads_pinger - 1, 0, -1):
                del worker_p[z]
            #freeing (?) that additional memory
            for z in range(num_threads_pinger_callback - 1, 0, -1):
                db_pinger_callback[z].close_connection()
                del db_pinger_callback[z]
                del worker_cb[z]
    
            db.close_connection()
    


Answer: In order to give you an exact explanation it would help to know what version
of Python you're using. For instance if you're using PyPy then what you've
observed is the JIT kicking in after you call your loop 5 times and it just
returns a pre-calculated answer. If you're using a standard version of Python
then this speed up is due to the interpreter using the compiled byte code from
the .pyc files. How it works is basically Python will first create an in
memory representation of your code and run it from there. During repeated
calls the interpreter will convert some of the more often used code into byte
code and store it on disk in .pyc files (this is python byte code similar to
java byte code not to be confused with native machine code). Every time you
call the same function the interpreter will go to your .pyc files and execute
the corresponding byte code this makes the execution much faster as the code
you're running is precompiled compared to when you call the function once and
python has to parse and interpret your code.

