[gmx-developers] KNL usage and mcdram

[Szilárd Páll]

On Fri, Sep 22, 2017 at 6:07 PM, Schulz, Roland <roland.schulz at intel.com> wrote:
> Usually it is best to run with all in mcdram closely followed by using cache mode.
> To run with all in mcdram use: "numactl -m1". For a few more tips about how to run on Xeon Phi you might want to look at:
> https://software.intel.com/en-us/articles/recipe-building-and-running-gromacs-on-intel-processors

Do you have any data on the the differences between clustering modes?
That guide suggests quadrant mode but I do wonder if there is any
benefit to that as long as ranks don't span across cluster-boundaries.

What about huge pages?

> -----Original Message-----
> From: gromacs.org_gmx-developers-bounces at maillist.sys.kth.se [mailto:gromacs.org_gmx-developers-bounces at maillist.sys.kth.se] On Behalf Of Åke Sandgren
> Sent: Friday, September 22, 2017 6:27 AM
> To: gmx-developers at gromacs.org
> Subject: Re: [gmx-developers] KNL usage and mcdram
>
> Yeah, I'm running a longer benchmark series of the ion_channel case on all our different hw types. Just wanted to make sure I hadn't missed something when building the KNL version.
>
> On 09/22/2017 02:36 PM, Szilárd Páll wrote:
>> No, it does not. IIRC, as Berk said, there were small differences when
>> using cache mode (as long as everything fits!), but you can also
>> configure it in flat mode and bind mdrun to the first NUMA region. For
>> all intents and purposes that should be as good as it gets.
>>
>> You might want to also try huge pages and quadrant mode (I don't have
>> detailed data comment on the impact of these, but both of those have
>> been claimed to be beneficial).
>>
>> --
>> Szilárd
>>
>>
>> On Fri, Sep 22, 2017 at 12:22 PM, Åke Sandgren
>> <ake.sandgren at hpc2n.umu.se> wrote:
>>> Ok, so gromacs doesn't explicitly use mcdram at least.
>>>
>>> On 09/22/2017 11:44 AM, Berk Hess wrote:
>>>> Hi,
>>>>
>>>> GROMACS simulations will usually completely fit in MCDRAM. So cache
>>>> configuration should be ok. I don't recall if there is some small
>>>> overhead to cache mode, in which case it would be better to use
>>>> MCDRAM as (the only) memory.
>>>>
>>>> Cheers,
>>>>
>>>> Berk
>>>>
>>>> On 22/09/17 10:28 , Åke Sandgren wrote:
>>>>> Hi!
>>>>>
>>>>> Quick question, does gromacs currently take advantage of explicit
>>>>> use of mcdram on KNL nodes?
>>>>> I.e., when it's not configure as cache that is.
>>>>>
>>>>> Didn't see anything obvious during confiugure.
>>>>>
>>>>
>>>
>>> --
>>> Ake Sandgren, HPC2N, Umea University, S-90187 Umea, Sweden
>>> Internet: ake at hpc2n.umu.se   Phone: +46 90 7866134 Fax: +46 90-580 14
>>> Mobile: +46 70 7716134 WWW: http://www.hpc2n.umu.se
>>> --
>>> Gromacs Developers mailing list
>>>
>>> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before posting!
>>>
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>> * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or send a mail to gmx-developers-request at gromacs.org.
>
> --
> Ake Sandgren, HPC2N, Umea University, S-90187 Umea, Sweden
> Internet: ake at hpc2n.umu.se   Phone: +46 90 7866134 Fax: +46 90-580 14
> Mobile: +46 70 7716134 WWW: http://www.hpc2n.umu.se
> --
> Gromacs Developers mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or send a mail to gmx-developers-request at gromacs.org.
> --
> Gromacs Developers mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or send a mail to gmx-developers-request at gromacs.org.