[gmx-users] HPC performance in Gromacs ?

[Szilárd Páll]

Hi,

Only one of the files has the right permissions, but based on that the
TL;DR conclusion is that you seem to be oversubscribing the nodes and
missing thread pinning.
You're also running at a fairly high parallelization (17700 atoms system on
72 cores across 3 nodes) and to get good performance you'll need to make
sure that rank / thread count and assignment to cores, PP-PME balance are
correct/well-tuned.


A few things to look into:

* Don't oversubscribe the hardware: you're using 3 nodes with 24 cores
each, but you're starting a lot more threads than cores:

"On 54 MPI ranks doing PP, each using 3 OpenMP threads, and
on 18 MPI ranks doing PME"


* Make sure you're pinning threads either with the job scheduler of by
passing "-pin on"; at the moment neither is happening (at least not correct
because):

"Number of logical cores detected (24) does not match the number reported
by OpenMP (1).
Consider setting the launch configuration manually!:"

"Non-default thread affinity set probably by the OpenMP library,
disabling internal thread affinity"

These indicate that your job scheduler is not doing correct affinity
settings and therefore mdrun's threads are quite likely unpinned which will
most often lead to performance loss.
Either correct your job scheduler settings (which will not be easy if you
use heterogeneous thread counts across ranks, but I'm not even sure you
really want that) or use mdrun thread pinning with  "-pin on".


The load imbalance is just a symptom of the above issues, and should
improve when you fix these; later you might have to adjust the PP-PME
resource split to something closer to 4:1 or more give the long cutoff.

Cheers,


--
Szilárd

On Tue, Sep 5, 2017 at 8:54 PM, Li, J. <jing.li at rug.nl> wrote:

> Hello Nikhil,
>
> Thanks for your suggestions. here is the links for two log files. 1 is for
> normal set, 2 is for tunning set.
>
> https://drive.google.com/a/rug.nl/file/d/0B-VoMAn-
> UrjYbEY2dUFvS3pLUWs/view?usp=sharing
> https://drive.google.com/a/rug.nl/file/d/0B-VoMAn-
> UrjYcjctSHBOckRKeEU/view?usp=sharing
>
> Jing
>
> --
> Jing Li
> PhD student
> FMNS, University of Groningen
> Room 5115.0309, Nijenborg 4, Groningen
> The Netherlands
> --
> Gromacs Users mailing list
>
> * Please search the archive at http://www.gromacs.org/
> Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>