title: JobLib | PicklingError: Could not pickle the task to send it to the workers
body: <p>I am wanting to run many invocations of Neural Network <em>GPT-2</em> by <em>HappyFace</em>.</p>
  <p>It works without stand-alone:</p>
  <pre><code>%%time
  for invoc in invoc_list:
      val_list = query(invoc, entries)
      val_list
  &gt;&gt;&gt; Wall time: 32.8 s
  &gt;&gt;&gt; ['&quot;I think we all knew the first part of The First World War.&quot;\n\nThe question of how the people got to where they are in the United States isn\'t entirely clear, but it involves a series of complex and',
  'from a lowly high school English teacher to the highest political executive in Israel when the Nazis captured that state in 1941, she wrote in her 1945 biography the &quot;secret&quot; book on the Nazi regime:\n\n&quot;Among the Nazi',
  'It is said that the woman of the North Sea was one of the most powerful of the Seven Sisters; and many of them were able to overcome other men. She is described as a &quot;Great Great Woman from the',
  'In a series of letters of his heredity to a classmate in a college near his home, she was asked by him to take her down to her home in Florida where she lives with her boyfriend, while she tried to']
  </code></pre>
  <p><strong>My aim is to have the above output yielded, but running in parallel via. JobLib.</strong></p>
  <p>However, when <em>&quot;converting&quot;</em> my code to <code>JobLib</code> code for <code>parallelisation</code>; I get the following error:</p>
  <pre><code>%%time
  my_list = Parallel(n_jobs=8)(delayed(query)(invoc, entries) for invoc in invoc_list)
  my_list
  &gt;&gt;&gt; PicklingError: Could not pickle the task to send it to the workers.
  </code></pre>
  <hr />
  <p>Notebook:</p>
  <pre><code>import time
   
  # GPT-2
  from transformers import pipeline, set_seed
  generator = pipeline('text-generation', model='gpt2')
  set_seed(42)
   
  #import dill as pickle
   
   
  def query(payload, entries):
      list_dict = generator(payload, min_length=25, max_length=50, num_return_sequences=entries)
      output = [d['generated_text'].split(payload)[1].strip() for d in list_dict]
      return output
   
  invoc_list = ['his her criminal charges include but are not limited to:', 'the defendant was sentenced to:', 'his her education history:']
  entries = 4
   
  %%time
  %%time
  for invoc in invoc_list:
      val_list = query(invoc, entries)
      val_list
   
  %%time
  my_list = Parallel(n_jobs=8)(delayed(query)(invoc, entries) for invoc in invoc_list)
  my_list
   
  </code></pre>
  <p>Note: you may need to install:</p>
  <pre><code>pip3 install joblib
  </code></pre>
  <pre><code>pip3 install dill
  </code></pre>