[gmx-users] Running job on GPUs

[Nidhi Katyal]

Hello all

I am trying to run my job on 2 nodes by utilizing all available cores. On
each node of the cluster, we have two GPUs and two sockets with 8 cores
each.
Every time I am submitting the job, we find that it is running on one node.
How to make use of the other node?

Till now, I have used following trial commands as suggested in
http://www.gromacs.org/Documentation/Acceleration_and_parallelization

1)  mpirun -n 2 mdrun_mpi -v -deffnm nvt -ntomp 16

output:

Using 2 MPI processes
Using 16 OpenMP threads per MPI process

WARNING: Oversubscribing the available 16 logical CPU cores with 32 threads.
         This will cause considerable performance loss!

2)  mpirun -n 4 mdrun_mpi -v -deffnm nvt -ntomp 8

output:

Incorrect launch configuration: mismatching number of PP MPI processes and
GPUs per node.
mdrun_mpi was started with 4 PP MPI processes per node, but only 2 GPUs
were detected.

I understand that the above error comes when number of MPI ranks is not a
multiple of number of GPUs intended to be used. But in my case 4 is a
multiple of 2.

3) mpirun -n 4 -npernode 2 mdrun_mpi -v -deffnm nvt

The job still runs on 1 node.

How can I run my job on 2 nodes utilizing all cores and GPUs?

Thanks
Nidhi