title: Scipy large sparse array dimensions MemoryError
body: <p>I'm working with relatively large high-dimensional sparse arrays using <code>scipy.sparse</code>.
  The actual data and row/column indices are no issue to store.</p>
  <p>The problem is I end up up with things like</p>
  <pre class="lang-py prettyprint-override"><code>sp.csr_matrix(([1], ([0], [0])), shape=(int(1e14), 1)).shape
  </code></pre>
  <p>which gives</p>
  <blockquote>
  <p><code>MemoryError: Unable to allocate 728. TiB for an array with shape (100000000000001,) and data type int64</code></p>
  </blockquote>
  <p>since it looks like <code>scipy</code> tries to allocate row/column masks (or something?)</p>
  <p>Is there a good workaround for this? Would using <code>coo_matrix</code> fix it?</p>
  <h3>Update</h3>
  <p>It turns out I'm just an idiot and should have paid better attention to whether I was using a CSC matrix or CSR matrix.</p>
  <p>CSC will compress the rows. CSR will compress the columns. For data stored like this (in what I am sure is a terrible format for making use of sparsity), CSC will work way better.</p>
  <p>In any case, both of these work fine</p>
  <pre class="lang-py prettyprint-override"><code>wat = sp.csc_matrix(([1], ([0], [0])), shape=(int(1e14), 1))
  wat2 = sp.csr_matrix(([1], ([0], [0])), shape=(1, int(1e14)))
  </code></pre>
  <p>and this is just a misunderstanding of what CSC and CSR do for us</p>