[gmx-developers] Hardware threads vs. OpenMP threads

[Szilárd Páll]

On Thu, Jun 4, 2015 at 1:46 PM, David van der Spoel
<spoel at xray.bmc.uu.se> wrote:
> On 04/06/15 12:51, Berk Hess wrote:
>>
>> PS There is something strange on that machine. If Gromacs detects 16
>> threads, omp_get_num_procs should return 16, not 8.
>
> Nope.
> The queue system allocates 8 cores out of 16 physical cores to my job.
> GROMACS see both values, reports a conflict, and follows the hardware rather
> than OpenMP settings. I would think it should do the reverse.

You did not pass  OMP_NUM_THREADS nor "-ntomp" and the job scheduler
set the CPUSET, right? In that case there is no reliable way to know
what the user wanted unless mdrun stops trying to fill the node with
threads and defaults to use nranks=1 and ntomp=1 unless told
otherwise.

Without such a pessimistic behavior, I think the only reasonable
solution is to expect that the queue system/user sets the correct
OMP_NUM_THREADS value.

Note that respecting omp_get_num_procs() could still lead to using 16
threads in total as with thread-MPI we can (and do) simply start two
ranks when e.g. 16 hardware threads are detected but
OMP_NUM_THREADS=8.

Fore more technical details around this issue see:
http://permalink.gmane.org/gmane.science.biology.gromacs.user/76761

--
Szilárd



>>
>> Berk
>>
>> On 2015-06-04 12:49, Berk Hess wrote:
>>>
>>> Hi,
>>>
>>> I don't think anything changed in the master branch.
>>>
>>> But we do adhere to the OpenMP environment. The value reported in the
>>> message comes from omp_get_num_procs, which should be a report about
>>> the hardware available. OMP_NUM_THREADS sets the number of OpenMP
>>> threads to use, that is respected.
>>>
>>> Cheers,
>>>
>>> Berk
>>>
>>> On 2015-06-04 11:21, David van der Spoel wrote:
>>>>
>>>> Hi,
>>>>
>>>> why does GROMACS in the master branch not adhere to the OpenMP
>>>> environment?
>>>>
>>>> Number of hardware threads detected (16) does not match the number
>>>> reported by OpenMP (8).
>>>> Consider setting the launch configuration manually!
>>>> Reading file md.tpr, VERSION 5.1-beta1-dev-20150603-99a1e1f-dirty
>>>> (single precision)
>>>> Changing nstlist from 10 to 40, rlist from 1.1 to 1.1
>>>>
>>>> Using 1 MPI process
>>>> Using 16 OpenMP threads
>>>>
>>>> Cheers,
>>>
>>>
>>
>
>
> --
> David van der Spoel, Ph.D., Professor of Biology
> Dept. of Cell & Molec. Biol., Uppsala University.
> Box 596, 75124 Uppsala, Sweden. Phone:  +46184714205.
> spoel at xray.bmc.uu.se    http://folding.bmc.uu.se
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or
> send a mail to gmx-developers-request at gromacs.org.