class_docstr: Generates and executes code to produce a large number of variable-size
  lists, simply.
  
  .. note:: This functionality is provided as a preview. Its interface
      is subject to change until this notice is removed.
  
  .. versionadded:: 2013.1
  
  Here's a usage example::
  
      from pyopencl.algorithm import ListOfListsBuilder
      builder = ListOfListsBuilder(context, [("mylist", np.int32)], """
              void generate(LIST_ARG_DECL USER_ARG_DECL index_type i)
              {
                  int count = i % 4;
                  for (int j = 0; j < count; ++j)
                  {
                      APPEND_mylist(count);
                  }
              }
              """, arg_decls=[])
  
      result, event = builder(queue, 2000)
  
      inf = result["mylist"]
      assert inf.count == 3000
      assert (inf.list.get()[-6:] == [1, 2, 2, 3, 3, 3]).all()
  
  The function ``generate`` above is called once for each "input object".
  Each input object can then generate zero or more list entries.
  The number of these input objects is given to :meth:`__call__` as *n_objects*.
  List entries are generated by calls to ``APPEND_<list name>(value)``.
  Multiple lists may be generated at once.
  
  .. automethod:: __init__
  .. automethod:: __call__
docstr: :arg args: arguments corresponding to ``arg_decls`` in the constructor.
      Array-like arguments must be either 1D :class:`pyopencl.array.Array`
      objects or :class:`pyopencl.MemoryObject` objects, of which the latter
      can be obtained from a :class:`pyopencl.array.Array` using the
      :attr:`pyopencl.array.Array.data` attribute.
  :arg allocator: optionally, the allocator to use to allocate new
      arrays.
  :arg omit_lists: an iterable of list names that should *not* be built
      with this invocation. The kernel code may *not* call ``APPEND_name``
      for these omitted lists. If it does, undefined behavior will result.
      The returned *lists* dictionary will not contain an entry for names
      in *omit_lists*.
  :arg wait_for: |explain-waitfor|
  :returns: a tuple ``(lists, event)``, where ``lists`` is a mapping from
      (built) list names to objects which have attributes
  
      * ``count`` for the total number of entries in all lists combined
      * ``lists`` for the array containing all lists.
      * ``starts`` for the array of starting indices in ``lists``.
        ``starts`` is built so that it has n+1 entries, so that
        the *i*'th entry is the start of the *i*'th list, and the
        *i*'th entry is the index one past the *i*'th list's end,
        even for the last list.
  
        This implies that all lists are contiguous.
  
      If the list name is specified in *eliminate_empty_output_lists*
      constructor argument, *lists* has two additional attributes
      ``num_nonempty_lists`` and ``nonempty_indices``
  
      * ``num_nonempty_lists`` for the number of nonempty lists.
      * ``nonempty_indices`` for the index of nonempty list in input objects.
  
      In this case, ``starts`` has ``num_nonempty_lists + 1`` entries.
      The *i*'s entry is the start of the *i*'th nonempty list, which is
      generated by the object with index ``nonempty_indices[i]``.
  
      *event* is a :class:`pyopencl.Event` for dependency management.
  
  .. versionchanged:: 2016.2
  
      Added omit_lists.
code: def __call__(self, queue, n_objects, *args, **kwargs):
      """
      :arg args: arguments corresponding to ``arg_decls`` in the constructor.
          Array-like arguments must be either 1D :class:`pyopencl.array.Array`
          objects or :class:`pyopencl.MemoryObject` objects, of which the latter
          can be obtained from a :class:`pyopencl.array.Array` using the
          :attr:`pyopencl.array.Array.data` attribute.
      :arg allocator: optionally, the allocator to use to allocate new
          arrays.
      :arg omit_lists: an iterable of list names that should *not* be built
          with this invocation. The kernel code may *not* call ``APPEND_name``
          for these omitted lists. If it does, undefined behavior will result.
          The returned *lists* dictionary will not contain an entry for names
          in *omit_lists*.
      :arg wait_for: |explain-waitfor|
      :returns: a tuple ``(lists, event)``, where ``lists`` is a mapping from
          (built) list names to objects which have attributes
  
          * ``count`` for the total number of entries in all lists combined
          * ``lists`` for the array containing all lists.
          * ``starts`` for the array of starting indices in ``lists``.
            ``starts`` is built so that it has n+1 entries, so that
            the *i*'th entry is the start of the *i*'th list, and the
            *i*'th entry is the index one past the *i*'th list's end,
            even for the last list.
  
            This implies that all lists are contiguous.
  
          If the list name is specified in *eliminate_empty_output_lists*
          constructor argument, *lists* has two additional attributes
          ``num_nonempty_lists`` and ``nonempty_indices``
  
          * ``num_nonempty_lists`` for the number of nonempty lists.
          * ``nonempty_indices`` for the index of nonempty list in input objects.
  
          In this case, ``starts`` has ``num_nonempty_lists + 1`` entries.
          The *i*'s entry is the start of the *i*'th nonempty list, which is
          generated by the object with index ``nonempty_indices[i]``.
  
          *event* is a :class:`pyopencl.Event` for dependency management.
  
      .. versionchanged:: 2016.2
  
          Added omit_lists.
      """
      if n_objects >= int(np.iinfo(np.int32).max):
          index_dtype = np.int64
      else:
          index_dtype = np.int32
      index_dtype = np.dtype(index_dtype)
  
      allocator = kwargs.pop("allocator", None)
      omit_lists = kwargs.pop("omit_lists", [])
      wait_for = kwargs.pop("wait_for", None)
      if kwargs:
          raise TypeError(
              "invalid keyword arguments: '%s'" % ", ".join(kwargs)
          )
  
      for oml in omit_lists:
          if not any(
              oml == name for name, _ in self.list_names_and_dtypes
          ):
              raise ValueError("invalid list name '%s' in omit_lists")
  
      result = {}
      count_list_args = []
  
      if wait_for is None:
          wait_for = []
      else:
          # We'll be modifying it below.
          wait_for = list(wait_for)
  
      count_kernel = self.get_count_kernel(index_dtype)
      write_kernel = self.get_write_kernel(index_dtype)
      scan_kernel = self.get_scan_kernel(index_dtype)
      if self.eliminate_empty_output_lists:
          compress_kernel = self.get_compress_kernel(index_dtype)
  
      data_args = []
      for i, (arg_descr, arg_val) in enumerate(
          zip(self.arg_decls, args)
      ):
          from pyopencl.tools import VectorArg
  
          if isinstance(arg_descr, VectorArg):
              from pyopencl import MemoryObject
  
              if arg_val is None:
                  data_args.append(arg_val)
                  if arg_descr.with_offset:
                      data_args.append(0)
                  continue
  
              if isinstance(arg_val, MemoryObject):
                  data_args.append(arg_val)
                  if arg_descr.with_offset:
                      raise ValueError(
                          "with_offset=True specified for argument %d "
                          "but the argument is not an array" % i
                      )
                  continue
  
              if arg_val.ndim != 1:
                  raise ValueError(
                      "argument %d is a multidimensional array" % i
                  )
  
              data_args.append(arg_val.base_data)
              if arg_descr.with_offset:
                  data_args.append(arg_val.offset)
              wait_for.extend(arg_val.events)
          else:
              data_args.append(arg_val)
  
      del args
      data_args = tuple(data_args)
  
      # {{{ allocate memory for counts
  
      for name, _dtype in self.list_names_and_dtypes:
          if name in self.count_sharing:
              continue
          if name in omit_lists:
              count_list_args.append(None)
              continue
  
          counts = cl.array.empty(
              queue, (n_objects + 1), index_dtype, allocator=allocator
          )
          counts[-1] = 0
          wait_for = wait_for + counts.events
  
          # The scan will turn the "counts" array into the "starts" array
          # in-place.
          if name in self.eliminate_empty_output_lists:
              result[name] = BuiltList(
                  count=None,
                  starts=counts,
                  lists=None,
                  num_nonempty_lists=None,
                  nonempty_indices=None,
              )
          else:
              result[name] = BuiltList(
                  count=None, starts=counts, lists=None
              )
          count_list_args.append(counts.data)
  
      # }}}
  
      if self.debug:
          gsize = (1,)
          lsize = (1,)
      elif self.do_not_vectorize():
          gsize = (4 * queue.device.max_compute_units,)
          lsize = (1,)
      else:
          from pyopencl.array import _splay
  
          gsize, lsize = _splay(queue.device, n_objects)
  
      count_event = count_kernel(
          queue,
          gsize,
          lsize,
          *(tuple(count_list_args) + data_args + (n_objects,)),
          wait_for=wait_for,
      )
  
      compress_events = {}
      for name, _dtype in self.list_names_and_dtypes:
          if name in omit_lists:
              continue
          if name in self.count_sharing:
              continue
          if name not in self.eliminate_empty_output_lists:
              continue
  
          compressed_counts = cl.array.empty(
              queue, (n_objects + 1,), index_dtype, allocator=allocator
          )
          info_record = result[name]
          info_record.nonempty_indices = cl.array.empty(
              queue, (n_objects + 1,), index_dtype, allocator=allocator
          )
          info_record.num_nonempty_lists = cl.array.empty(
              queue, (1,), index_dtype, allocator=allocator
          )
          info_record.compressed_indices = cl.array.empty(
              queue, (n_objects + 1,), index_dtype, allocator=allocator
          )
          info_record.compressed_indices[0] = 0
          compress_events[name] = compress_kernel(
              info_record.starts,
              compressed_counts,
              info_record.nonempty_indices,
              info_record.compressed_indices,
              info_record.num_nonempty_lists,
              wait_for=[count_event]
              + info_record.compressed_indices.events,
          )
  
          info_record.starts = compressed_counts
  
      # {{{ run scans
  
      scan_events = []
  
      for name, _dtype in self.list_names_and_dtypes:
          if name in self.count_sharing:
              continue
          if name in omit_lists:
              continue
  
          info_record = result[name]
          if name in self.eliminate_empty_output_lists:
              compress_events[name].wait()
              num_nonempty_lists = info_record.num_nonempty_lists.get()[
                  0
              ]
              info_record.num_nonempty_lists = num_nonempty_lists
              info_record.starts = info_record.starts[
                  : num_nonempty_lists + 1
              ]
              info_record.nonempty_indices = (
                  info_record.nonempty_indices[:num_nonempty_lists]
              )
              info_record.starts[-1] = 0
  
          starts_ary = info_record.starts
          if name in self.eliminate_empty_output_lists:
              evt = scan_kernel(
                  starts_ary,
                  size=info_record.num_nonempty_lists,
                  wait_for=starts_ary.events,
              )
          else:
              evt = scan_kernel(
                  starts_ary, wait_for=[count_event], size=n_objects
              )
  
          starts_ary.setitem(0, 0, queue=queue, wait_for=[evt])
          scan_events.extend(starts_ary.events)
  
          # retrieve count
          info_record.count = int(starts_ary[-1].get())
  
      # }}}
  
      # {{{ deal with count-sharing lists, allocate memory for lists
  
      write_list_args = []
      for name, dtype in self.list_names_and_dtypes:
          if name in omit_lists:
              write_list_args.append(None)
              if name not in self.count_sharing:
                  write_list_args.append(None)
              if name in self.eliminate_empty_output_lists:
                  write_list_args.append(None)
              continue
  
          if name in self.count_sharing:
              sharing_from = self.count_sharing[name]
  
              info_record = result[name] = BuiltList(
                  count=result[sharing_from].count,
                  starts=result[sharing_from].starts,
              )
  
          else:
              info_record = result[name]
  
          info_record.lists = cl.array.empty(
              queue, info_record.count, dtype, allocator=allocator
          )
          write_list_args.append(info_record.lists.data)
  
          if name not in self.count_sharing:
              write_list_args.append(info_record.starts.data)
  
          if name in self.eliminate_empty_output_lists:
              write_list_args.append(
                  info_record.compressed_indices.data
              )
  
      # }}}
  
      evt = write_kernel(
          queue,
          gsize,
          lsize,
          *(tuple(write_list_args) + data_args + (n_objects,)),
          wait_for=scan_events,
      )
  
      return result, evt
  
imports: from dataclasses import dataclass
  from typing import Optional
  import numpy
  import pyopencl
  import pyopencl.array
  from pyopencl.elementwise import ElementwiseKernel
  from pyopencl.scan import ScanTemplate, GenericScanKernel
  from pyopencl.tools import dtype_to_ctype, get_arg_offset_adjuster_code
  from pytools import memoize, memoize_method
  from mako.template import Template
label_desc: Use image node_id 7 for calling the ListOfListsBuilder obj's underlying member method code with example usage: obj.__call__(queue, n_objects) and returns: result,  evt