[gmx-users] new i9 processor

[Harry Mark Greenblatt]

BS”D

Dear Bernhard and Szilárd,

Since your replies to me are related, I’ll combine my replies to one letter.

you are right for the dual xeon 16 core setup you (and we) use, but for the single core i9 setup we made the experience, that we only gained between 5-10% of performance by adding a second gpu. This was what I was hinting here…

Bernhard, just so I understand the test you performed, you took the 10-core i9 machine and added a second 1080ti card?  So now instead of 10 cores paired with one GPU, the simulation ran using domain decomposition with two ranks, each rank running on 5 cores paired to each GPU, correct?  I would guess that those runs had a large performance imbalance, with a large amount of time spent waiting for the 5 cores to finish their part of the calculation.  To take advantage of the second GPU, one would need another ~10 cores, as Szilárd mentioned in his email, which can’t be done on a Core motherboard.



I hope I did say exactly that because in general it is not true: both
because we're accelerating (for now) a single task and because offload
to multiple GPU requires domain-decomposition.
The switch from no DD (only multi-threading) to DD has an initial hit
and going from 1 to 2 GPU you can therefore often see fairly moderate
scaling. From e.g. 2 to 4-way DD you can get near-linear scaling, but
it will depend on the system size.

Scaling to multiple GPUs, especially if by that you mean using the
same number of cores and increasing the GPU count will certainly not
give linear scaling in performance (time-to-solution) as only part of
the computation is reduced — if that scales at all.

I guess my use of the word “multiple” was not ideal.  I am not really thinking beyond two GPU’s.  I was discussing going from 6/8/10 cores paired with one GPU, to 12/16/20 cores paired with 2 GPU’s, using 2 MPI ranks, so that you still maintain the ratio of 6/8/10 cores per GPU.  In that situation, we have observed close to double the performance, and that situation was (I believe) what you commented on in some past email.

Threadripper is also an option; if anybody has successfully install Linux and run GROMACS on a that architecture I would be happy to hear about it.




So generally: you'll need a well-balanced CPU cores to GPU ratio and
increasing the GPUs count only will only help if the run in GPU-bound
(and even than, only by a limited amount).


Thanks


Harry




--------------------------------------------------------------------
Harry M. Greenblatt
Associate Staff Scientist
Dept of Structural Biology           harry.greenblatt at weizmann.ac.il<../../owa/redir.aspx?C=QQgUExlE8Ueu2zs5OGxuL5gubHf97c8IyXxHOfOIqyzCgIQtXppXx1YBYaN5yrHbaDn2xAb8moU.&URL=mailto%3aharry.greenblatt%40weizmann.ac.il>
Weizmann Institute of Science        Phone:  972-8-934-6340
234 Herzl St.                        Facsimile:   972-8-934-3361
Rehovot, 7610001
Israel