title: Linear data processing to simultaneous processing
body: <p>I'm trying to understand multiprocessing. Let's say that I have a fairly large number of values between 0 and 1 stored in a list.</p>
  
  <pre><code>a = [rand() for _ in range(1e9)]
  </code></pre>
  
  <p>And I want to do something if the value is higher than 0.5.</p>
  
  <pre><code>for i in a:
      if i &gt; 0.5:
          do_something()
  </code></pre>
  
  <p>Each value will thus be evaluated linearly. But can I evaluate several values simultaneously?</p>
  
  <pre><code>is_above = (i for i in a if i &gt; 0.5)
  
  import multiprocessing as mp
  pool = mp.Pool()
  task = pool.imap(do_something, is_above, chunksize=1e9/4) # 1e9 elements divided by 4 cores?
  pool.close()
  pool.join()
  </code></pre>
  
  <p>Will this piece of code work as expected?</p>
  
  <p>Edit:
  Here is a concrete example, with prime numbers :</p>
  
  <pre><code>import multiprocessing as mp
  
  def is_prime(num):
      if num &gt; 2:
          for i in range(2, num // 2):
              if (num % i) == 0:
                  break
          else:
              print(num)
  
  
  start = 1_000_000_000
  stop = 1_000_000_500
  a = (i for i in range(start, stop))
  
  if __name__ == '__main__':
      p = mp.Pool()
      t = p.imap(is_prime, a, chunksize=int((stop - start) / 6)) #6 cores
      p.close()
      p.join()
  
  </code></pre>
  
  <p>I think anyone interested in this should read this before :
  <a href="https://medium.com/contentsquare-engineering-blog/multithreading-vs-multiprocessing-in-python-ece023ad55a" rel="nofollow noreferrer">https://medium.com/contentsquare-engineering-blog/multithreading-vs-multiprocessing-in-python-ece023ad55a</a></p>