title: why is converting a long 2D list to numpy array so slow?
body: <p>I have a long list of xy coordinates, and would like to convert it into numpy array.</p>
  
  <pre><code>&gt;&gt;&gt; import numpy as np
  &gt;&gt;&gt; xy = np.random.rand(1000000, 2).tolist()
  </code></pre>
  
  <p>The obvious way would be:</p>
  
  <pre><code>&gt;&gt;&gt; a = np.array(xy) # Very slow...
  </code></pre>
  
  <p>However, the above code is unreasonably slow. Interestingly, to transpose the long list first, convert it into numpy array, and then transpose back would be much faster (20x on my laptop).</p>
  
  <pre><code>&gt;&gt;&gt; def longlist2array(longlist):
  ...     wide = [[row[c] for row in longlist] for c in range(len(longlist[0]))]
  ...     return np.array(wide).T
  &gt;&gt;&gt; a = longlist2array(xy) # 20x faster!
  </code></pre>
  
  <p>Is this a bug of numpy?</p>
  
  <p>EDIT:</p>
  
  <p>This is a list of points (with xy coordinates) generated on-the-fly, so instead of preallocating an array and enlarging it when necessary, or maintaining two 1D lists for x and y, I think current representation is most natural.</p>
  
  <p>Why is looping through 2nd index faster than 1st index, given that we are iterating through a python list in both directions?</p>
  
  <p>EDIT 2:</p>
  
  <p>Based on @tiago's answer and <a href="https://stackoverflow.com/questions/367565/how-do-i-build-a-numpy-array-from-a-generator">this question</a>, I found the following code twice as fast as my original version:</p>
  
  <pre><code>&gt;&gt;&gt; from itertools import chain
  &gt;&gt;&gt; def longlist2array(longlist):
  ...     flat = np.fromiter(chain.from_iterable(longlist), np.array(longlist[0][0]).dtype, -1) # Without intermediate list:)
  ...     return flat.reshape((len(longlist), -1))
  </code></pre>