[gmx-users] GROMACS performance with an NVidia Tesla k40c

[Michail Palaiokostas Avramidis]

Dear GMX users,


I have recently installed an Nvidia Tesla K40c in my workstation (already had a quadro k2000) and I am currently trying to optimize its usage with GROMACS. I used two compilations of GROMACS, one is the standard one as suggested in the beginning of the installation documentation and one where I added some more flags to see what will happen. The latter compilation used:


cmake .. -DGMX_BUILD_OWN_FFTW=ON -DREGRESSIONTEST_DOWNLOAD=ON -DGMX_GPU=on -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-7.5 -DNVML_INCLUDE_DIR=/usr/include/nvidia/gdk -DNVML_LIBRARY=/usr/lib/nvidia-352/libnvidia-ml.so


So far I used 4 different combinations to test a water-membrane system of ~30500 atoms for 5000 steps:

1) CPU only,

2) CPU+2GPUs (the default),

3) CPU+Quadro and

4) CPU+Tesla.

Obviously the fastest is the Tesla one with 31ns/day. This is 3.6 times faster than the CPU-only setup.


While this is good, I am not entirely satisfied with the speed-up. Do you think is normal? Would you expect more?


One thing I noticed is that there was absolutely no difference with using the custom, GPU-oriented compilation of GROMACS. Did I miss something there?


The second thing I noticed is that even by increasing nstlist the performance remained the same (despite the suggestion in the documentation).


Finally, in my log file I got the message (the actual log is attached to the message):

Force evaluation time GPU/CPU: 2.232 ms/3.189 ms = 0.700

For optimal performance this ratio should be close to 1!

NOTE: The GPU has >25% less load than the CPU. This imbalance causes performance loss.


Can you please help me solve this imbalance? At the moment I am executing gromacs with: gmx mdrun -v -deffnm npt-ini -gpu_id 0


Thank you in advance for your help.


Best Regards,

Michail


-------------------------------------------------------------------
Michail (Michalis) Palaiokostas
PhD Student
School of Engineering and Materials Science
Queen Mary University of London
-------------------------------------------------------------------