title: Processing a list of lists in Cython, with nogil
body: <p>In Python I have a list of lists as input:</p>
  
  <pre><code>input = [[0,1,2],[0,3,4,5],[0,6]]
  </code></pre>
  
  <p>In reality, the number of sub-lists is tens of thousands. The length of each sub-list could vary greatly, from zero or one value to many hundreds.</p>
  
  <p>I want to pass the input data as some 2D structure to a Cython module that will process it. I wish to process the data on multiple cores, thus I use <code>prange</code> with <code>nogil=True</code>:</p>
  
  <pre><code>from cython.parallel import prange
  
  cpdef int my_func(long[:,:] arr):
      cdef int i,j
      for i in prange(arr.shape[0], nogil=True):
          for j in range(arr.shape[1]):
                      # Do something
              pass
      return 42
  </code></pre>
  
  <p>I see the following solutions:</p>
  
  <ol>
  <li>Put the list of lists into a 2D ndarray. But as the length of each sub-list vary greatly, a ndarray is not an ideal data structure</li>
  <li>Modify <code>my_func</code> to accept a list of lists. The problem is that part of the code is executed without the GIL and therefore not able to access python objects.</li>
  </ol>
  
  <p>Does anyone have suggestions, preferably with code, on how to solve this problem?</p>