[gmx-users] gmx 2019 performance issues

[Tamas Hegedus]

Hi,

I do not really see an increased performance with gmx 2019 using -bonded 
gpu. I do not see what I miss or misunderstand.
The only thing I see that all cpu run at ~100% with gmx2018, while some 
of the cpus run only at ~60% with gmx2019.

There are: 196382 Atoms
Speeds comes from 500 ps runs.

 From one of the log files:
Mapping of GPU IDs to the 4 GPU tasks in the 4 ranks on this node:
   PP:0,PP:0,PP:2,PME:2
PP tasks will do (non-perturbed) short-ranged and most bonded 
interactions on the GPU
PME tasks will do all aspects on the GPU

------------------------------
16 cores 4 GPUs
gmx 2018 48ns/day
gmx 2019 54ns/day

gmx mdrun -nt 16 -ntmpi 4 -pin on -v -deffnm md_test -nb gpu -pme gpu 
-npme 1 -gputasks 0123

gmx mdrun -nt 16 -ntmpi 4 -pin on -v -deffnm md_test -nb gpu -bonded gpu 
-pme gpu -npme 1 -gputasks 0123

Since the GPUs are not utilized well (some of them are below 50%), my 
objective is run 2 jobs/node with 8 CPUs and 2 GPUs with higher usage.

------------------------------
8 cores 2 GPUs
gmx 2018 33 ns/day
gmx 2019 35 ns/day

gmx mdrun -nt 8 -ntmpi 4 -pin on -v -deffnm md_test -nb gpu -pme gpu 
-npme 1 -gputasks 0033

gmx mdrun -nt 8 -ntmpi 4 -pin on -v -deffnm md_test -nb gpu -bonded gpu 
-pme gpu -npme 1 -gputasks 0022

gmx mdrun -ntomp 2 -ntmpi 4 -pin on -v -deffnm md_test -nb gpu -bonded 
gpu -pme gpu -npme 1 -gputasks 0022
Changing -nt to -ntomp did not help to increase performance.

And the GPUs are not utilized much better. 1080Ti runs max 60-75%

------------------------------
The main question:
* I use 16 core AMD 2950X with 4 high end GPUs (1080Ti, 2080Ti).
* GPUs does not run at 100%, so I would like load more on them and 
possibly run 2 gmx jobs on the same node.

I see two options:
* cheaper: decrease the cores from 16 to 8 and push bonded calculations 
to gpu using gmx 2019
* expensive: replace the 16core 2950X to 32core 2990WX

2950X 16 cores 2 GPUs
gmx 2018 43 ns/day
gmx 2019 43 ns/day

33 ns/day (8core/2GPUs) <<<< 54 (16core/4GPUS)
43 ns/day << 54 (16core/4GPUS)

So this could be a compromise if 16/32 cores works similarly as 16/16 
cores. E.g. 2990 has slower memory access compared to 2950; I do not 
expect this to influence gmx runs too much. However, if it decreases by 
10-15 percentage then most likely it does not worth to invest into the 
32 core processor.

Thanks for your feedbacks.
Tamas

-- 
Tamas Hegedus, PhD
Senior Research Fellow
Department of Biophysics and Radiation Biology
Semmelweis University     | phone: (36) 1-459 1500/60233
Tuzolto utca 37-47        | mailto:tamas at hegelab.org
Budapest, 1094, Hungary   | http://www.hegelab.org