[gmx-users] Performance on multiple GPUs per node

[Jens Krüger]

Dear all,

we are currently planning a new cluster at our universities compute 
centre. The big question on our side is, how many and which GPUs we 
should put into the nodes.

We have access to a test system with four Tesla K80s per Node. Using one 
GPU node we can reach something like 23 ns/day for the ADH system (PME, 
cubic) which is pretty much in line with e.g., 
http://exxactcorp.com/index.php/solution/solu_list/84

When trying to use 2 or more GPUs on one node, the performance plunges 
to below 10 ns/day no matter how we split the MPI/OMP threads. Has 
anybody of you guys access to a comparable hardware setup? We would be 
interested in benchmark data answering the question: Does GROMACS-5.1 
scales on more than one GPU per node?

Thanks and best wishes,

Jens