Nesting mpi calls with mpi4py
Question: I am trying to use mpi4py to call a second instance of an mpi executable.

I am getting the error:

    
    
    Open MPI does not support recursive calls of mpirun
    

But I was under the impression that is exactly what Spawn is supposed to be
able to handle - i.e. setting up a new communicator within which another mpi
command could be launched.

The test code:

parent.py:

    
    
    #!/usr/bin/env python
    from mpi4py import MPI
    import numpy
    import sys
    
    
    rank = MPI.COMM_WORLD.Get_rank()
    new_comm = MPI.COMM_WORLD.Split(color=rank, key=rank)
    print(new_comm.Get_rank())
    new_comm.Spawn(sys.executable,
                               args=['test.py'],
                               maxprocs=4)
    

which calls test.py:

    
    
    #!/usr/bin/env python
    from mpi4py import MPI
    import numpy
    import os
    import sys
    
    comm = MPI.Comm.Get_parent()
    rank = comm.Get_rank()
    
    cwd=os.getcwd()
    directory=os.path.join(cwd,str(rank))
    os.chdir(directory)
    
    
    os.system('{}'.format('mpirun -np 4 SOME_MPI_EXECUTABLE_HERE'))
    
    
    
    print("Finished in "+directory)
    os.chdir(cwd)
    
    comm.Disconnect()
    

I'm running with:

    
    
    mpirun --oversubscribe -np 1 parent.py
    

Using openmpi 2.0.0 with gcc, and python/3.4.2

Anyone have any bright ideas as to why this is happening.....

Thanks!


Answer: The following code seems to perform the way I wanted.

    
    
    #!/usr/bin/env python
    from mpi4py import MPI
    import numpy
    import sys
    import os
    
    rank = MPI.COMM_WORLD.Get_rank()
    new_comm = MPI.COMM_WORLD.Split(color=rank, key=rank)
    print(new_comm.Get_rank())
    
    cwd=os.getcwd()
    os.mkdir(str(rank))
    directory=os.path.join(cwd,str(rank))
    print(rank,directory)
    os.chdir(directory)
    
    
    new_comm.Spawn("SOME_MPI_EXECUTABLE_HERE",
                      args=[""],
                               maxprocs=4)
    

run with:

mpirun --oversubscribe -np 4 parent.py

Seems to start 4 instances of SOME_MPI_EXECUTABLE each running on 4 cores.

(Thanks to Zulan)

