[gmx-users] Output pinning for mdrun

[Mark Abraham]

On Thu, Oct 24, 2013 at 4:52 PM, Carsten Kutzner <ckutzne at gwdg.de> wrote:

> On Oct 24, 2013, at 4:25 PM, Mark Abraham <mark.j.abraham at gmail.com>
> wrote:
>
> > Hi,
> >
> > No. mdrun reports the stride with which it moves over the logical cores
> > reported by the OS, setting the affinity of GROMACS threads to logical
> > cores, and warnings are written for various wrong-looking cases, but we
> > haven't taken the time to write a sane report of how GROMACS logical
> > threads and ranks are actually mapped to CPU cores. Where supported by
> the
> > processor, the CPUID information is available and used in
> > gmx_thread_affinity.c. It's just not much fun to try to report that in a
> > way that will make sense on all possible hardware that supports CPUID -
> and
> > then people will ask why it doesn't map to what their mpirun reports, get
> > confused by hyper-threading, etc.
> Yes, I see.
> >
> > What question were you seeking to answer?
> Well, I just wanted to check whether my process placement is correct and
> that
> I am not getting decreased performance due to a suboptimal placement. In
> many cases the performance is really bad (like 50% of the expected values)
> if the pinning is wrong or does not work, but you never know.
>

GROMACS does report if its attempt to set affinities fail (and the reason),
which covers some of the problem cases. Keeping MPI ranks closely
associated with the hardware granularity (nodes, sockets, GPUs) will be
important, but that's something to configure at the mpirun level.
(Thread-MPI, being a single-node solution, has more assumptions it can make
safely.) Keeping OpenMP threads within their MPI ranks pinned within
hardware regions is important, but severity and solutions vary a lot with
the hardware and software context (e.g. you might as well get out an
abacus, as run GROMACS with OpenMP spread over a whole AMD processor, but
with a single GPU then that can be the best you can do, at the moment).

Key to interpreting performance results is to measure the (pinned)
single-core performance, so that there is a minimum-overhead reference
number for comparison.

Mark

On some clusters there are of course tools that check and output the process
> placement for a dummy parallel job, or environment variables like
> MP_INFOLEVEL for
> loadleveler.
>
> Thanks!
>   Carsten
>
>
> > Mark
> >
> >
> >
> > On Thu, Oct 24, 2013 at 11:44 AM, Carsten Kutzner <ckutzne at gwdg.de>
> wrote:
> >
> >> Hi,
> >>
> >> can one output how mdrun threads are pinned to CPU cores?
> >>
> >> Thanks,
> >>  Carsten
> >> --
> >> gmx-users mailing list    gmx-users at gromacs.org
> >> http://lists.gromacs.org/mailman/listinfo/gmx-users
> >> * Please search the archive at
> >> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> >> * Please don't post (un)subscribe requests to the list. Use the
> >> www interface or send it to gmx-users-request at gromacs.org.
> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>
> > --
> > gmx-users mailing list    gmx-users at gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-request at gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>