Something strange happen with python multiprocess
Question: I've just tested python multiprocessing for reading file or a global variable,
but there is something strange happen.

for expample:

    
    
    import multiprocessing
    
    a = 0
    
    def test(lock, name):    
        global a
        with lock:        
            for i in range(10):
                a = a + 1
            print "in process %d : %d" % (name, a)
    
    def main():    
        lock = multiprocessing.Lock()            
    
        p1 = multiprocessing.Process(target=test, args=(lock, 1))
        p2 = multiprocessing.Process(target=test, args=(lock, 2))
    
        p1.start()
        p2.start()
    
        p1.join()
        p2.join()
    
        print "in main process : %d" % a
    
    if __name__=='__main__':
        main()   
    

The program read a global variable, but the output is:

    
    
    in process 1 : 10
    in process 2 : 10
    in main process : 0
    

It seems that the sub-process cannot get and edit the global variable
properly. Also, if I change the program to read the file, each sub-process
will read the file completely, ignoring the lock.

So how does these happen? And how to solve this problem?


Answer: Global variables are not shared between processes. When you create and start a
new `Process()`, that process runs inside a separate "cloned" copy of the
current Python interpreter. Updating the variable from within a `Process()`
will only update the variable locally to the particular process it is updated
in.

To share data between Python processes, we need a
[`multiprocessing.Pipe()`](https://docs.python.org/2/library/multiprocessing.html#multiprocessing.Pipe),
a
[`multiprocessing.Queue()`](https://docs.python.org/2/library/multiprocessing.html#multiprocessing.Queue),
a
[`multiprocessing.Value()`](https://docs.python.org/2/library/multiprocessing.html#multiprocessing.Value),
a
[`multiprocessing.Array()`](https://docs.python.org/2/library/multiprocessing.html#multiprocessing.Array)
or one of the other multiprocessing-safe containers.

Here's an example based on your code:

    
    
    import multiprocessing
    
    def worker(lock, counter, name):
        with lock:        
            for i in range(10):
                counter.value += 1
    
            print "In process {}: {}".format(name, counter.value)
    
    def main():
        lock = multiprocessing.Lock()    
        counter = multiprocessing.Value('i', 0)         
    
        p1 = multiprocessing.Process(target=worker, args=(lock, counter, 1))
        p2 = multiprocessing.Process(target=worker, args=(lock, counter, 2))
    
        p1.start()
        p2.start()
    
        p1.join()
        p2.join()
    
        print "In main process: {}".format(counter.value)
    
    if __name__=='__main__':
        main()   
    

This gives me:

    
    
    In process 1: 10
    In process 2: 20
    In main process: 20
    

Now, if you really want to use a global variable, you can use a
[`multiprocessing.Manager()`](https://docs.python.org/2/library/multiprocessing.html#multiprocessing.Manager),
but I think the first method is preferable, and this is a "heavier" solution.
Here's an example:

    
    
    import multiprocessing
    
    manager = multiprocessing.Manager()
    counter = manager.Value('i', 0);
    
    def worker(lock, name):
        global counter
    
        with lock:        
            for i in range(10):
                counter.value += 1
    
            print "In process {}: {}".format(name, counter.value)
    
    def main():
        global counter
    
        lock = multiprocessing.Lock()       
    
        p1 = multiprocessing.Process(target=worker, args=(lock, 1))
        p2 = multiprocessing.Process(target=worker, args=(lock, 2))
    
        p1.start()
        p2.start()
    
        p1.join()
        p2.join()
    
        print "In main process: {}".format(counter.value)
    
    if __name__=='__main__':
        main()   
    

