Title: Multithreading a for loop in Python
Tags: python;multithreading
Question: I have a for loop that looks like this in Python:
def layerOneBackprop(l, a):
for x in range(len(l.weights)):
for y in range(len(l.weights[x])):
l.weightDelta[x][y] = a.weightDelta[0][x] * a.weights[0][x] * dSigmoid(l.layerOut[x])
dW = l.inVals[y] * l.weightDelta[x][y]
l.weights[x][y] = l.weights[x][y] - (learningRate * dW)
My current problem is that this loop (the entire function actually) is quite slow as it takes nearly 30-40 seconds to run per call. I am hoping that parallelizing this function will help it run faster on my server. How would this be done in Python?
I have seen thread implementations but I have also read that the GIL for Python may cause my code to become essentially single threaded. How could this be accomplished without such problems?
Comment: Thanks for the tip! I am using `numpy` right now, this isn't my most optimized code for `numpy` as well. I wanted to see if it was possible in my old implementation. As `theano` goes, I can't use that because of the environment this code will run in.

Comment: Dude, take a look to PyCuda or PyOpenCl, nothing would be faster than that, hope you a have a cool graphic cards.

Comment: Multiprocessing is the answer you're looking for, but I agree with John it is not the *right* answer. Unless you have a 16 core machine, multiprocessing really isn't going to help a whole lot.
+1 to `numpy`, +1 to `pypy`, +1 to `cython`

Comment: While blocking IO operations release Python's GIL, intensive computations don't, so multithreading in Python simply doesn't work. If you look at the provided link (your question is effectively a duplicate of it) you will find multiprocessing proposed for that reason.

Comment: It looks like you're writing some functionality for deep neural networks. Honestly you'll probably gain more performance by switching to `numpy` or `theano` and using their native matrix operations instead of using python's built-in lists.

Comment: This sounds a lot like:
http://stackoverflow.com/questions/20548628/how-to-do-parallel-programming-in-python



{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 0, 'CommentCount': 6, 'ContentLicense': 'CC BY-SA 3.0', 'CreationDate': '2016:00:04/27/16', 'FavoriteCount': 1.0, 'PostTypeId': 1, 'Tags': '<python><multithreading>', 'Title': 'Multithreading a for loop in Python', 'ViewCount': 565, 'id': 36880474, 'user_AboutMe': '<p>Currently at Clear Street in NYC as a Software Engineer</p>\n<p>OS Design, Distributed Systems, Infrastructure, things that go fast</p>\n', 'user_AccountId': 3167144.0, 'user_DisplayName': 'Tanishq dubey', 'user_Reputation': 1438.0, 'user_Views': 134.0, 'user_id': 2676875.0}}