[gmx-users] Output pinning for mdrun

[Szilárd Páll]

Hi Carsten,

On Thu, Oct 24, 2013 at 4:52 PM, Carsten Kutzner <ckutzne at gwdg.de> wrote:
> On Oct 24, 2013, at 4:25 PM, Mark Abraham <mark.j.abraham at gmail.com> wrote:
>
>> Hi,
>>
>> No. mdrun reports the stride with which it moves over the logical cores
>> reported by the OS, setting the affinity of GROMACS threads to logical
>> cores, and warnings are written for various wrong-looking cases, but we
>> haven't taken the time to write a sane report of how GROMACS logical
>> threads and ranks are actually mapped to CPU cores. Where supported by the
>> processor, the CPUID information is available and used in
>> gmx_thread_affinity.c. It's just not much fun to try to report that in a
>> way that will make sense on all possible hardware that supports CPUID - and
>> then people will ask why it doesn't map to what their mpirun reports, get
>> confused by hyper-threading, etc.
> Yes, I see.
>>
>> What question were you seeking to answer?
> Well, I just wanted to check whether my process placement is correct and that
> I am not getting decreased performance due to a suboptimal placement. In
> many cases the performance is really bad (like 50% of the expected values)
> if the pinning is wrong or does not work, but you never know.

That could very well be the result of two software threads pinned to
that same hardware thread.

You can use hwloc-ps to display the mapping of thread and process ID-s
to hardware entities (core, socket, NUMA domain, etc.). However, this
will still not give you the full picture, that is the mdrun thread ID
-> OS thread ID -> hardware thread/core mapping, as you can't know
which OS thread corresponds to which software thread. I have some code
in a pending (and for now blocked) change:
https://gerrit.gromacs.org/#/c/2633/
in src/gmxlib/gmx_thread_affinity.c:print_current_proc_thread_binding().

Let me know if that worked.

Cheers,
--
Szilárd

> On some clusters there are of course tools that check and output the process
> placement for a dummy parallel job, or environment variables like MP_INFOLEVEL for
> loadleveler.
>
> Thanks!
>   Carsten
>
>
>> Mark
>>
>>
>>
>> On Thu, Oct 24, 2013 at 11:44 AM, Carsten Kutzner <ckutzne at gwdg.de> wrote:
>>
>>> Hi,
>>>
>>> can one output how mdrun threads are pinned to CPU cores?
>>>
>>> Thanks,
>>>  Carsten
>>> --
>>> gmx-users mailing list    gmx-users at gromacs.org
>>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>> * Please don't post (un)subscribe requests to the list. Use the
>>> www interface or send it to gmx-users-request at gromacs.org.
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>> --
>> gmx-users mailing list    gmx-users at gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> * Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-request at gromacs.org.
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists