Multiple python threads writing to different records in same list simultaneously - is this ok?
Question: I am trying to fix a bug where multiple threads are writing to a list in
memory. Right now I have a thread lock and am occasionally running into
problems that are related to the work being done in the threads.

I was hoping to simply make an hash of lists, one for each thread, and remove
the thread lock. It seems like each thread could write to its own record
without worrying about the others, but perhaps the fact that they are all
using the same owning hash would itself be a problem.

Does anyone happen to know if this will work or not? If not, could I, for
example, dynamically add a list to a package for each thread? Is that
essentially the same thing?

I am far from a threading expert so any advice welcome.

Thanks,


Answer: 
    import threading
    
    def job(root_folder,my_list):
        for current,files,dirs in os.walk(root):
            my_list.extend(files)
            time.sleep(1)
    
    my_lists = [[],[],[]]
    my_folders = ["C:\\Windows","C:\\Users","C:\\Temp"]
    my_threads = []
    for folder,a_list in zip(my_folders,my_lists):
        my_threads.append(threading.Thread(target=job,args=(folder,a_list)
    for thread in my_threads:
       thread.start()
    for thread in my_threads:
       thread.join()
    
    my_full_list = my_lists[0] + my_lists[1] + my_lists[2]
    

this way each thread just modifies its own list and at the end combines all
the individual lists

also as pointed out this gives zero performance gain (actually probably slower
than not threading it... ) you may get performance gains using multiprocessing
instead ...

