Broadcast error when adding ndarray and sparse matrix converted to dense in python
Question: I am converting from a scipy sparse matrix to a dense matrix and adding that
to an ndarray using a += operator and I am getting a broadcast error. The
ndarray has a shape (M,) while the dense matrix has a shape (M,1). When I try
and convert the numpy.matrix to an ndarray the shape becomes (1,M). I also
tried using reshape(M,) after the conversion and the shape becomes (M,M).
Could someone please explain where am I messing up.

    
    
    M = 100
    N = 1000
    K = 4
    a= np.zeros((M,K))
    
    b = sp.csr_matrix(np.ones((N,1)))
    d = sp.csr_matrix(np.ones((M,N)))
    
    c = np.ones(())
    for k in range(K): 
        a[:,k] += d.dot(b).todense()
    

P.S: I am a newbie to both python and stackoverflow. Apologies if this is a
wrongly framed question or if this question was already asked earlier (I
couldn't find it)


Answer: Side note to future googlers, OP's code has the following package imported:

    
    
    > import numpy as np
    > import scipy.sparse as sp
    

Anyway, let's begin with our explanation.

* * *

You can check your matrix size using:

    
    
    > np.shape(your_matrix_here) #equals to MATLAB: > size(your_matrix_here)
    

As you call todense(), you got:

    
    
    matrix([[ 1000.],  #shape: (1, 100)
            [ 1000.],
            [ 1000.],
            ...
    

while a[:,1] looks like this:

    
    
    array([ 0.,  0., ... ,  0.,  0.]) #shape: (100,)
    

So you may want to transpose your matrix using transpose().

* * *

But transposing the matrix returns matrix containing matrix:

    
    
    matrix([[ 1000.,  1000., ... , 1000.,  1000.]]) #shape: (1, 100)
    

and for some reason `d.dot(b).todense().transpose()[0]` doesn't return the
first element the matrix:

    
    
    matrix([[ 1000.,  1000., ... , 1000.,  1000.]]) #still the same!
    

* * *

This can be fixed using:

    
    
    > np.array(d.dot(b).todense().transpose())[0]
    

thus returning:

    
    
    array([ 1000.,  1000., ... 1000.,  1000.])
    

* * *

Now two of them got the same shape, allowing them to perform matrix operation:

    
    
    > np.shape(np.array(d.dot(b).todense().transpose())[0]) #(100,)
    
    > np.shape(a[:,1])                                      #(100,)
    

* * *

**In conclusion,** you want to change this line:

    
    
    a[:,k] += d.dot(b).todense()
    

to:

    
    
    a[:,k] += np.array(d.dot(b).todense().transpose())[0]
    

