title: efficiently convert uneven list of lists to minimal containing array padded with nan
body: <p>consider the list of lists <code>l</code></p>
  
  <pre><code>l = [[1, 2, 3], [1, 2]]
  </code></pre>
  
  <p>if I convert this to a <code>np.array</code> I'll get a one dimensional object array with <code>[1, 2, 3]</code> in the first position and <code>[1, 2]</code> in the second position.</p>
  
  <pre><code>print(np.array(l))
  
  [[1, 2, 3] [1, 2]]
  </code></pre>
  
  <p>I want this instead</p>
  
  <pre><code>print(np.array([[1, 2, 3], [1, 2, np.nan]]))
  
  [[  1.   2.   3.]
   [  1.   2.  nan]]
  </code></pre>
  
  <hr>
  
  <p>I can do this with a loop, but we all know how unpopular loops are</p>
  
  <pre><code>def box_pir(l):
      lengths = [i for i in map(len, l)]
      shape = (len(l), max(lengths))
      a = np.full(shape, np.nan)
      for i, r in enumerate(l):
          a[i, :lengths[i]] = r
      return a
  
  print(box_pir(l))
  
  [[  1.   2.   3.]
   [  1.   2.  nan]]
  </code></pre>
  
  <hr>
  
  <p><strong><em>how do I do this in a fast, vectorized way?</em></strong></p>
  
  <hr>
  
  <p><strong><em>timing</em></strong>  </p>
  
  <p><a href="https://i.stack.imgur.com/Tx98Q.png" rel="noreferrer"><img src="https://i.stack.imgur.com/Tx98Q.png" alt="enter image description here"></a></p>
  
  <p><a href="https://i.stack.imgur.com/8tR1O.png" rel="noreferrer"><img src="https://i.stack.imgur.com/8tR1O.png" alt="enter image description here"></a></p>
  
  <p><strong><em>setup functions</em></strong>  </p>
  
  <pre><code>%%cython
  import numpy as np
  
  def box_pir_cython(l):
      lengths = [len(item) for item in l]
      shape = (len(l), max(lengths))
      a = np.full(shape, np.nan)
      for i, r in enumerate(l):
          a[i, :lengths[i]] = r
      return a
  </code></pre>
  
  <hr>
  
  <pre><code>def box_divikar(v):
      lens = np.array([len(item) for item in v])
      mask = lens[:,None] &gt; np.arange(lens.max())
      out = np.full(mask.shape, np.nan)
      out[mask] = np.concatenate(v)
      return out
  
  def box_hpaulj(LoL):
      return np.array(list(zip_longest(*LoL, fillvalue=np.nan))).T
  
  def box_simon(LoL):
      max_len = len(max(LoL, key=len))
      return np.array([x + [np.nan]*(max_len-len(x)) for x in LoL])
  
  def box_dawg(LoL):
      cols=len(max(LoL, key=len))
      rows=len(LoL)
      AoA=np.empty((rows,cols, ))
      AoA.fill(np.nan)
      for idx in range(rows):
          AoA[idx,0:len(LoL[idx])]=LoL[idx]
      return AoA
  
  def box_pir(l):
      lengths = [len(item) for item in l]
      shape = (len(l), max(lengths))
      a = np.full(shape, np.nan)
      for i, r in enumerate(l):
          a[i, :lengths[i]] = r
      return a
  
  def box_pandas(l):
      return pd.DataFrame(l).values
  </code></pre>