[gmx-users] Stampede multiple gpu nodes simulation

[devapriyachem at gmail.com]

Hello,

I have been trying to run a MD simulation on Stampede cluster using the gpu 
queue. Using only 1 node, I get roughly 42 ns/day performance. I have around 
42,000 atoms in my system. 

However, I am unable to get this system to run using 2 or more gpu nodes. I 
have encountered errors from GPU devices not found to oversubscribing logical 
cores on the first node. 

#SBATCH -p gpu               # Queue name
#SBATCH -N 2                   # Total number of nodes requested (16 
cores/node)

One of my attempts was as follows:

ibrun -n 2 -N 1 mdrun_mpi_gpu -s md-01.tpr -x md-01.xtc -e md-01.edr -g 
md-01.log

If anyone has experience setting up multi-node runs on gpu queue on Stampede 
and could share their insight, I would be very grateful.

Thanks,
Deva