[gmx-users] GPU and MPI

[Da-Wei Li]

Dear users,

I recently try to run Gromacs on two nodes, each of them has 12 cores and 2
GPUs. The nodes are connected with infiniband and scaling is pretty good
when no GPU is evolved.

My command is like this:

mpiexec  -npernode 2 -np 4 mdrun_mpi -ntomp 6


However, it looks like Gromacs only detected 2 GPUs on node 0, then skip
node 1. Part of the output looks like:


************************

Using 4 MPI processes

Using 6 OpenMP threads per MPI process

2 GPUs detected on host n0316.ten:

  #0: NVIDIA Tesla M2070, compute cap.: 2.0, ECC: yes, stat: compatible

  #1: NVIDIA Tesla M2070, compute cap.: 2.0, ECC: yes, stat: compatible

2 GPUs user-selected for this run.

Mapping of GPUs to the 2 PP ranks in this node: #0, #1

****************************


The performance is about only 40% of the run, where I use only 1 node (12
cores+2GPUs).


Does I miss something?


thanks.


dawei