title: Tensflow Keras: TypeError: can't pickle _thread.RLock objects when using multiprocessing
body: <p>I have raised this issue in GitHub: <a href="https://github.com/tensorflow/tensorflow/issues/46917" rel="nofollow noreferrer">https://github.com/tensorflow/tensorflow/issues/46917</a></p>
  <p>I am trying to use multiprocessing threads to speedup the some of my code. In which I have to send a Keras model to each thread and use it to predict on some inputs and do some following computations. However, I end up with the following error</p>
  <pre><code>Tensflow Keras: TypeError: can't pickle _thread.RLock objects
  </code></pre>
  <p>I tried,</p>
  <ol>
  <li>using <code>partial</code> to fix the model argument and use the resulting partial-function.</li>
  <li>cloning the model and using a clone for each thread</li>
  <li>saving and reloading a model for each thread</li>
  <li>tried using <code>pathos.multiprocessing</code>
  but none of them worked.</li>
  </ol>
  <p>The following is the MWE</p>
  <pre><code>import tensorflow as tf
  from tensorflow import keras
  import numpy as np
  
  
  from multiprocessing import Pool
  # from multiprocessing.dummy import Pool as ThreadPool
  # from pathos.multiprocessing import ProcessingPool as Pool
  from functools import partial
  
  
  def simple_model():
      model = keras.models.Sequential([
          keras.layers.Dense(units = 10, input_shape = [1]),
          keras.layers.Dense(units = 1, activation = 'sigmoid')
      ])
      model.compile(optimizer = 'sgd', loss = 'mean_squared_error')
      return model
  
  def clone_model(model):
      model_clone = tf.keras.models.clone_model(model)
      model_clone.set_weights(model.get_weights())
      model_clone.build((None, 1))
      model_clone.compile(optimizer = 'sgd', loss = 'mean_squared_error')
      return model_clone
  
  def work(model, seq):
      return model.predict(seq)
  
  def load_model(model_savepath):
      return tf.keras.models.load_model(model_savepath)
  
  def worker(model, n = 4):
      seqences = np.arange(0,100).reshape(n, -1)
      pool = Pool()
      model_savepath = './simple_model.h5'
      model.save(model_savepath)
      model_list = [load_model(model_savepath) for _ in range(n)]
      # model_list = [clone_model(model) for _ in range(n)]
      results = pool.map(work, zip(model_list,seqences))
      # partial_work = partial(work, model=model)
      # results = pool.map(partial_work, seqences)
      pool.close()
      pool.join()
      
      return np.reshape(results, (-1, ))
  
  
  
  if __name__ == '__main__':
  
      model = simple_model()
      out = worker(model, n=4)
      print(out)
  </code></pre>
  <p>This results in the following error trace:</p>
  <pre><code>File &quot;c:/Users/***/Documents/GitHub/COVID-NSF/test4.py&quot;, line 42, in &lt;module&gt;
    out = worker(model, n=4)
  File &quot;c:/Users/****/Documents/GitHub/COVID-NSF/test4.py&quot;, line 30, in worker
    results = pool.map(work, zip(model_list,seqences))
  File &quot;C:\Users\****\anaconda3\envs\tf-gpu\lib\multiprocessing\pool.py&quot;, line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File &quot;C:\Users\****\anaconda3\envs\tf-gpu\lib\multiprocessing\pool.py&quot;, line 657, in get
    raise self._value
  File &quot;C:\Users\***\anaconda3\envs\tf-gpu\lib\multiprocessing\pool.py&quot;, line 431, in _handle_tasks
    put(task)
  File &quot;C:\Users\***\anaconda3\envs\tf-gpu\lib\multiprocessing\connection.py&quot;, line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File &quot;C:\Users\***\anaconda3\envs\tf-gpu\lib\multiprocessing\reduction.py&quot;, line 51, in dumps
    cls(buf, protocol).dump(obj)
  TypeError: can't pickle _thread.RLock objects
  </code></pre>