title: Keep track of the state of event in multiprocessing by writing to file python
body: <p>I am working on a task that requires multiprocessing in python and I need to keep track of the state by writing already processed document IDs to a file (a single file shared among processes).</p>
  <p>I have implemented a simple version using the code snippet below. In the code, I have some Ids stored in a variable called <code>question</code>, the shared file <code>f</code> and in the main method, I split the <code>question</code> into possible chunks that can be processed parallel.</p>
  <p>Is this the right way to do such?</p>
  <pre><code>from multiprocessing import Pool
  from multiprocessing import Queue
  
  def reader(val):
    pqueue.put(val)
  
  def writer():
    a = pqueue.get()
    f = open(&quot;num.txt&quot;,'a')
    f.write(str(a))
    f.write(&quot;\n&quot;)
    f.close()
    
  
  def main():
    
    global question
    global pqueue
    pqueue = Queue() # writer() writes to pqueue from _this_ process
  
    
    processes = []
    question = [16,0,1,2,3,4,5,6,7,8,10,11,12,13,14,15]
    cores=5
    loww=0
    chunksize = int((len(question)-loww)/cores)
    splits = []
    for i in range(cores):
      splits.append(loww+1+((i)*chunksize))
    splits.append(len(question)+1)
    print(splits)
    args = []
    for i in range(cores):
      a=[]
      arguments = (i, splits[i], splits[i+1])
      a.append(arguments)
      args.append(a)
    print(args)
    p = Pool(cores)
    p.map(call_process, args)
    p.close()
    p.join
  
  def call_process(args):
    lower=args[0][1]
    upper=args[0][2]
    for x in range(lower,upper):
      a = question[x-1]
      try:
        pass
      except:
        continue
      #write item to file
      print(f,'a = ',a)
      reader(a)
      writer()
      
  
  
  main()
  </code></pre>
  <p>Note: the code seems not to be working.</p>