[gmx-users] How to assign PME ranks to particular nodes?

[Marcin Mielniczuk]

Hi,

I'm running Gromacs on a heterogenous cluster, with one node
significantly faster than the other. Therefore, I'd like to achieve the
following setup:
* run 2 or 3 PP processes and 1 PME process on the faster node (with a
lower number of OpenMP threads)
* run 2 PP processes on the slower node (with a lower number of OpenMP
threads)

Setting the number of threads is easy: one may just create a wrapper
script on every node setting OMP_NUM_THREADS and GMX_PME_NUM_THREADS and
use it instead of gmx_mpi. The rank assignment is more difficult.

It's possible to use the OpenMPI --map-by option to control how ranks
are mapped to nodes and use -ddorder pp_pme in Gromacs, so that the last
rank is the PME rank, but this may possibly negatively affect the
topology of the PP ranks. An alternative is to use an OpenMPI rankfile,
which requires me to specify precisely how all the ranks are mapped, not
just the rank counts.

Is there any better way to set the number of PP/PME ranks on particular
nodes?

Thanks,
Marcin