title: Scipy - how to further optimize sparse matrix code for stochastic gradient descent
body: <p>I'm working on implementing the stochastic gradient descent algorithm for recommender systems using sparse matrices with Scipy.</p>
  
  <p>This is how a first basic implementation looks like:</p>
  
  <pre><code>    N = self.model.shape[0] #no of users
      M = self.model.shape[1] #no of items
      self.p = np.random.rand(N, K)
      self.q = np.random.rand(M, K)
      rows,cols = self.model.nonzero()        
      for step in xrange(steps):
          for u, i in zip(rows,cols):
              e=self.model-np.dot(self.p,self.q.T) #calculate error for gradient
              p_temp = learning_rate * ( e[u,i] * self.q[i,:] - regularization * self.p[u,:])
              self.q[i,:]+= learning_rate * ( e[u,i] * self.p[u,:] - regularization * self.q[i,:])
              self.p[u,:] += p_temp
  </code></pre>
  
  <p>Unfortunately, my code is still pretty slow, even for a small 4x5 ratings matrix. I was thinking that this is probably due to the sparse matrix for loop. I've tried expressing the q and p changes using fancy indexing but since I'm still pretty new at scipy and numpy, I couldn't figure a better way to do it. </p>
  
  <p>Do you have any pointers on how i could avoid iterating over the rows and columns of the sparse matrix explicitly?</p>