[gmx-developers] Disabling Pthreads
Gilles Gouaillardet
gilles at rist.or.jp
Wed Nov 11 08:27:38 CET 2020
Berk,
There is a total of 52 cores, and my observation is that each MPI task
does spawn 52 threads.
So the worst case scenario is 48 MPI tasks each spawning 52 threads, so
a total of 48 * 52 threads on a single node
Cheers,
Gilles
On 11/11/2020 4:23 PM, Berk Hess wrote:
> On 2020-11-11 03:37, Gilles Gouaillardet wrote:
>> Erik and all,
>>
>>
>> I am kind of facing the exact opposite issue on an other ARM processor:
>>
>> High end A64fx (Fugaku/FX1000) have 48 cores plus 2 or 4 assistant
>> cores.
>>
>> A job is put in a cgroup of 48 cores (e.g. no assistant cores)
>>
>> Worst case scenario, a flat mpi run (48 tasks) will spawn 48 * 52
>> cores to spin up all the cores.
> What do you mean with 48 * 52 cores? There are only 52 cores. Gromacs
> will by default not spawn more threads in total then there are cores.
> If you ask for 48 MPI ranks with a real MPI library Gromacs will not
> spawn any additional threads. With thread-mpi it will spawn 48 or 52
> total.
>
> Cheers,
>
> Berk
>>
>> 1) GROMACS is not cgroup aware and hence consider there are 52 (or
>> 50) cores per node (this is a very minor issue)
>>
>> 2) spawning such a high number of threads caused some crashes (weird
>> stack traces, I did not spend much time investigating)
>>
>> 3) in the case of A64fx, all cores are up and running, ready to
>> crunch, and do not require any special tricks.
>>
>>
>> At this stage, I think the easiest path to address this on A64fx is
>> to add yet an other cmake option to
>>
>> unconditionally skip the spin up phase.
>>
>> This could be improved by adding a command line option, or an
>> environment variable, to change the default behavior
>>
>> (default behavior should be a cmake option imho)
>>
>>
>> Any thoughts on how to best move forward?
>>
>>
>> Cheers,
>>
>>
>> Gilles
>>
>> On 11/11/2020 6:08 AM, Erik Lindahl wrote:
>>> We might be able to work around the last aspect, but it will likely
>>> take a couple of weeks until I can lay hands on a new ARM-based
>>> Macbook.
>>>
>>> Long story made short: The advanced power-saving features on ARM
>>> mean some cores are not visible until they are used, so we created a
>>> small hack where we "spin up" the CPU by exercising all cores.
>>>
>>> We might anyway need to do something different with the new type of
>>> big.LITTLE cores where we have 4+4 or 8+4 cores, but I can't even
>>> start to work on that until I have suitable hardware. The good news
>>> is that such hardware was announced a couple of hours ago, with
>>> availability next week ;-)
>>>
>>> Cheers,
>>>
>>> Erik
>>>
>>> On Tue, Nov 10, 2020 at 9:45 PM Mark Abraham
>>> <mark.j.abraham at gmail.com <mailto:mark.j.abraham at gmail.com>> wrote:
>>>
>>> No, the use of std::thread in eg hardware detection also requires
>>> a lower level threading implementation.
>>>
>>> Mark
>>>
>>> On Tue, Nov 10, 2020, 20:41 Berk Hess <hess at kth.se
>>> <mailto:hess at kth.se>> wrote:
>>>
>>> Hi,
>>>
>>> Turning off GMX_THREAD_MPI in cmake should remove the
>>> dependency on pthreads.
>>>
>>> Cheers,
>>>
>>> Berk
>>>
>>> On 2020-11-10 18:06, Guido Giuntoli wrote:
>>>>
>>>> Hi,
>>>>
>>>> Is there any way to disable the “Pthreads” dependency during
>>>> the configuration/compilation of GROMACS?
>>>>
>>>> *Best regards | Mit freundlichen Grüßen*
>>>>
>>>> **
>>>>
>>>> *Guido Giuntoli***
>>>>
>>>> **
>>>>
>>>> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>>>> Hansaallee 205, 40549 Dusseldorf, Germany, *www.huawei.com*
>>>> <http://www.huawei.com/>
>>>> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB
>>>> 56063,
>>>> Managing Director: Li Peng, Li Jian, Shi Yanli**
>>>>
>>>> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf,
>>>> HRB 56063,
>>>> Geschäftsführer: Li Peng, Li Jian, Shi Yanli
>>>>
>>>> *-----------------------------------------------------------------------------------------------*
>>>>
>>>>
>>>> *This e-mail and its attachments contain confidential
>>>> information from HUAWEI, which is intended only for the
>>>> person or entity whose address is listed above. Any use of
>>>> the information contained herein in any way (including, but
>>>> not limited to, total or partial disclosure, reproduction, or
>>>> dissemination) by persons other than the intended
>>>> recipient(s) is prohibited. If you receive this e-mail in
>>>> error, please notify the sender by phone or email immediately
>>>> and delete it!*
>>>>
>>>>
>>>
>>> -- Gromacs Developers mailing list
>>>
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
>>> <http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List>
>>> before posting!
>>>
>>> * Can't post? Read
>>> http://www.gromacs.org/Support/Mailing_Lists
>>> <http://www.gromacs.org/Support/Mailing_Lists>
>>>
>>> * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>>> <https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers>
>>>
>>> or send a mail to gmx-developers-request at gromacs.org
>>> <mailto:gmx-developers-request at gromacs.org>.
>>>
>>> -- Gromacs Developers mailing list
>>>
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
>>> <http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List>
>>> before posting!
>>>
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>> <http://www.gromacs.org/Support/Mailing_Lists>
>>>
>>> * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>>> <https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers>
>>>
>>> or send a mail to gmx-developers-request at gromacs.org
>>> <mailto:gmx-developers-request at gromacs.org>.
>>>
>>>
>>>
>>> --
>>> Erik Lindahl <erik.lindahl at dbb.su.se <mailto:erik.lindahl at dbb.su.se>>
>>> Professor of Biophysics, Dept. Biochemistry & Biophysics, Stockholm
>>> University
>>> Science for Life Laboratory, Box 1031, 17121 Solna, Sweden
>>>
>
More information about the gromacs.org_gmx-developers
mailing list