Geek asks and answers

Interacting with MPI binary using (non MPI) python script

I would like to somehow call the execution of certain functions of the MPI program (written in C ++) through, for example, the (serial) python script . This python script should start the mpi program at the beginning, for example,

subprocess.call(['mpirun','-np', '4', 'mpibinary', 'args' ])

I need to call the function of this MPI program several times, and I want to avoid restarting the program for different inputs, since I need to reinitialize all my data structures, which are expensive. So I was thinking of running the function externally when the MPI program is idle. I think this can be done using an IO file, i.e. The root rank of the MPI program monitors a specific file for (1) a cycle, and as soon as its contents change, it analyzes the new contents, notifies the other ranks and calls the function. Is there a more elegant solution to my problem?

A better solution would be to have a python class that wraps the important functions of a C ++ MPI program so that I can call them from python using

mpiprogram.superfunction(a,b)
+3
source share
1 answer

, Python MPI. ( MPI) MPI, . :

1) Python 0 MPI. mpibinary, , 0 mpibinary. - . Open MPI :

mpirun --hostfile hosts -np 1 pythonbinary args : -np 32 mpibinary args

MPMD ( ), pythonbinary, 0, 32 mpibinary, 1, 2- ... 32 ( 33 ). MPI MPMD. MPI_Comm_split(), , Python. - . Python, ++. MPI_Comm_split() "" . . , , :

Python:

python_comm = mpi.mpi_comm_split(mpi.MPI_COMM_WORLD, 0, 0)

++:

int rank;
MPI_Comm c_comm;

MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_split(MPI_COMM_WORLD, 1, rank, &c_comm);

rank , , c_comm , , 1 MPI_COMM_WORLD 0 c_comm, 2 1 ..

++ c_comm , . Python ++ MPI_COMM_WORLD, Python - 0.

2) MPI-2. MPI-, Python:

mpirun --hostfile hosts -np 1 pythonbinary args

Python MPI , MPI_Comm_spawn() . MPI_COMM_WORLD, MPI_Comm_split(). spawn , Python MPI.

hosts , MPI. Python MPI-.

, MPI- Python script, MPI_Init, MPI_Finalize, MPI_Comm_split MPI_Send/MPI_Recv. . MPI , , . , Python MPI, .

( MQ).

+5

More articles:


All Articles