title: Parallel for loop over numpy matrix
body: <p>I am looking at the <a href="https://pythonhosted.org/joblib/parallel.html" rel="nofollow noreferrer"><code>joblib</code></a> examples but I can't figure out how to do a parallel for loop over a matrix. I am computing a pairwise distance metric between the rows of a matrix. So I was doing:</p>
  
  <pre><code>N, _ = data.shape
  upper_triangle = [(i, j) for i in range(N) for j in range(i + 1, N)]
  dist_mat = np.zeros((N,N))  
  
  for (i, j) in upper_triangle:
      dist_mat[i,j] = dist_fun(data[i], data[j])
      dist_mat[j,i] = dist_mat[i,j]
  </code></pre>
  
  <p>where <code>dist_fun</code> takes two vectors and computes a distance. How can I make this loop parallel since calls to <code>dist_fun</code> can be made independent of each other. </p>
  
  <hr>
  
  <p>EDIT: The distance function I am using is <a href="https://github.com/slaypni/fastdtw" rel="nofollow noreferrer"><code>fastdtw</code></a> which is <strong>not</strong> so fast. So I think <strong><em>really do</em></strong> want to parallelize this. Using:</p>
  
  <pre><code>dist_mat = pdist(data, lambda x,y : fastdtw(x,y, dist=euclidean)[0])
  </code></pre>
  
  <p>I get an execution time of 58.1084 secs, and using:</p>
  
  <pre><code>dist_mat = np.zeros((N,N))
  for (i,j), _ in np.ndenumerate(dist_mat):
      dist_mat[i,j], _ = fastdtw(data[i,:], timeseries[j,:], dist=euclidean)
  </code></pre>
  
  <p>I get 116.36 seconds and using:</p>
  
  <pre><code>upper_triangle = [(i,j) for i in range(N) for j in range(i+1, N)]
  dist_mat = np.zeros((N,N))
  
  for (i,j) in upper_triangle:
      dist_mat[i,j], _ = fastdtw(data[i,:], data[j,:], dist=euclidean)
      dist_mat[j,i] = dist_mat[i,j]
  </code></pre>
  
  <p>I get 55.62 secs. Here <code>N=33</code>. Does <code>scipy</code> automatically makes use of all available cores?</p>
  
  <hr>
  
  <p>EDIT: I think I have found a work around using the <a href="https://docs.python.org/2/library/multiprocessing.html" rel="nofollow noreferrer"><code>multiprocessing</code></a> package, but I will leave the question un-answered for the <strong><em>joblib</em></strong> folks to respond before I post what I think works. </p>