[gmx-developers] Disabling Pthreads

Gilles Gouaillardet gilles at rist.or.jp
Wed Nov 11 08:27:38 CET 2020


Berk,


There is a total of 52 cores, and my observation is that each MPI task 
does spawn 52 threads.

So the worst case scenario is 48 MPI tasks each spawning 52 threads, so 
a total of 48 * 52 threads on a single node


Cheers,


Gilles

On 11/11/2020 4:23 PM, Berk Hess wrote:
> On 2020-11-11 03:37, Gilles Gouaillardet wrote:
>> Erik and all,
>>
>>
>> I am kind of facing the exact opposite issue on an other ARM processor:
>>
>> High end A64fx (Fugaku/FX1000) have 48 cores plus 2 or 4 assistant 
>> cores.
>>
>> A job is put in a cgroup of 48 cores (e.g. no assistant cores)
>>
>> Worst case scenario, a flat mpi run (48 tasks) will spawn 48 * 52 
>> cores to spin up all the cores.
> What do you mean with 48 * 52 cores? There are only 52 cores. Gromacs 
> will by default not spawn more threads in total then there are cores. 
> If you ask for 48 MPI ranks with a real MPI library Gromacs will not 
> spawn any additional threads. With thread-mpi it will spawn 48 or 52 
> total.
>
> Cheers,
>
> Berk
>>
>> 1) GROMACS is not cgroup aware and hence consider there are 52 (or 
>> 50) cores per node (this is a very minor issue)
>>
>> 2) spawning such a high number of threads caused some crashes (weird 
>> stack traces, I did not spend much time investigating)
>>
>> 3) in the case of A64fx, all cores are up and running, ready to 
>> crunch, and do not require any special tricks.
>>
>>
>> At this stage, I think the easiest path to address this on A64fx is 
>> to add yet an other cmake option to
>>
>> unconditionally skip the spin up phase.
>>
>> This could be improved by adding a command line option, or an 
>> environment variable, to change the default behavior
>>
>> (default behavior should be a cmake option imho)
>>
>>
>> Any thoughts on how to best move forward?
>>
>>
>> Cheers,
>>
>>
>> Gilles
>>
>> On 11/11/2020 6:08 AM, Erik Lindahl wrote:
>>> We might be able to work around the last aspect, but it will likely 
>>> take a couple of weeks until I can lay hands on a new ARM-based 
>>> Macbook.
>>>
>>> Long story made short: The advanced power-saving features on ARM 
>>> mean some cores are not visible until they are used, so we created a 
>>> small hack where we "spin up" the CPU by exercising all cores.
>>>
>>> We might anyway need to do something different with the new type of 
>>> big.LITTLE cores where we have 4+4 or 8+4 cores, but I can't even 
>>> start to work on that until I have suitable hardware. The good news 
>>> is that such hardware was announced a couple of hours ago, with 
>>> availability next week ;-)
>>>
>>> Cheers,
>>>
>>> Erik
>>>
>>> On Tue, Nov 10, 2020 at 9:45 PM Mark Abraham 
>>> <mark.j.abraham at gmail.com <mailto:mark.j.abraham at gmail.com>> wrote:
>>>
>>>     No, the use of std::thread in eg hardware detection also requires
>>>     a lower level threading implementation.
>>>
>>>     Mark
>>>
>>>     On Tue, Nov 10, 2020, 20:41 Berk Hess <hess at kth.se
>>>     <mailto:hess at kth.se>> wrote:
>>>
>>>         Hi,
>>>
>>>         Turning off GMX_THREAD_MPI in cmake should remove the
>>>         dependency on pthreads.
>>>
>>>         Cheers,
>>>
>>>         Berk
>>>
>>>         On 2020-11-10 18:06, Guido Giuntoli wrote:
>>>>
>>>>         Hi,
>>>>
>>>>         Is there any way to disable the “Pthreads” dependency during
>>>>         the configuration/compilation of GROMACS?
>>>>
>>>>         *Best regards | Mit freundlichen Grüßen*
>>>>
>>>>         **
>>>>
>>>>         *Guido Giuntoli***
>>>>
>>>>         **
>>>>
>>>>         HUAWEI TECHNOLOGIES Duesseldorf GmbH
>>>>         Hansaallee 205, 40549 Dusseldorf, Germany, *www.huawei.com*
>>>>         <http://www.huawei.com/>
>>>>         Registered Office: Düsseldorf, Register Court Düsseldorf, HRB
>>>>         56063,
>>>>         Managing Director: Li Peng, Li Jian, Shi Yanli**
>>>>
>>>>         Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf,
>>>>         HRB 56063,
>>>>         Geschäftsführer: Li Peng, Li Jian, Shi Yanli
>>>>
>>>> *-----------------------------------------------------------------------------------------------* 
>>>>
>>>>
>>>>         *This e-mail and its attachments contain confidential
>>>>         information from HUAWEI, which is intended only for the
>>>>         person or entity whose address is listed above. Any use of
>>>>         the information contained herein in any way (including, but
>>>>         not limited to, total or partial disclosure, reproduction, or
>>>>         dissemination) by persons other than the intended
>>>>         recipient(s) is prohibited. If you receive this e-mail in
>>>>         error, please notify the sender by phone or email immediately
>>>>         and delete it!*
>>>>
>>>>
>>>
>>>         --         Gromacs Developers mailing list
>>>
>>>         * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
>>> <http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List>
>>>         before posting!
>>>
>>>         * Can't post? Read
>>>         http://www.gromacs.org/Support/Mailing_Lists
>>>         <http://www.gromacs.org/Support/Mailing_Lists>
>>>
>>>         * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>>> <https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers> 
>>>
>>>         or send a mail to gmx-developers-request at gromacs.org
>>>         <mailto:gmx-developers-request at gromacs.org>.
>>>
>>>     --     Gromacs Developers mailing list
>>>
>>>     * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
>>> <http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List>
>>>     before posting!
>>>
>>>     * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>     <http://www.gromacs.org/Support/Mailing_Lists>
>>>
>>>     * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>>> <https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers> 
>>>
>>>     or send a mail to gmx-developers-request at gromacs.org
>>>     <mailto:gmx-developers-request at gromacs.org>.
>>>
>>>
>>>
>>> -- 
>>> Erik Lindahl <erik.lindahl at dbb.su.se <mailto:erik.lindahl at dbb.su.se>>
>>> Professor of Biophysics, Dept. Biochemistry & Biophysics, Stockholm 
>>> University
>>> Science for Life Laboratory, Box 1031, 17121 Solna, Sweden
>>>
>


More information about the gromacs.org_gmx-developers mailing list