[gmx-developers] Gromacs trying to use OpenMP instead of thread-mpi despite trying to convince it?

Mon Dec 3 19:18:27 CET 2012

On Mon, Dec 3, 2012 at 1:02 PM, Szilárd Páll <szilard.pall at cbr.su.se> wrote:

>
>  On Mon, Dec 3, 2012 at 5:12 AM, Roland Schulz <roland at utk.edu> wrote:
>
>>
>>
>>
>>  On Sun, Dec 2, 2012 at 12:58 PM, Shirts, Michael (mrs5pt) <
>> mrs5pt at eservices.virginia.edu> wrote:
>>
>>> So, more progress, but no simulations running yet.
>>>
>>> mdrun -nt 8 -ntmpi 8 gives the same error as before (I actually tried
>>> that
>>> before, and forgot to include it in my error report)
>>>
>>> mdrun -ntmpi 8 -ntomp 1 gives the error
>>> Fatal error:
>>> OMP_NUM_THREADS (8) and the number of threads requested on the command
>>> line
>>> (1) have different values
>>>  For more information and tips for troubleshooting, please check the
>>> GROMACS
>>> website at http://www.gromacs.org/Documentation/Errors
>>>
>>
>>  We probably should print a notice that OMP_NUM_THREADS is set.
>> Otherwise this is really confusing if OMP_NUM_THREADS isn't set by the user
>> but by the system.
>>
>
>  There is a note printed whenever the number of OpenMP threads is set by
> OMP_NUM_THREADS instead of -ntomp.
>
I think if even core developers don't understand/find a note, then that's a
pretty clear sign that it will be confusing to the average user ;-)

>
>
>>
>>
>>> Fatal error:
>>> OMP_NUM_THREADS is invalid: '0'
>>>
>>
>>  This is also for ntomp. Also there you wand to use 1 not 0 to disable
>> OpenMP (1 because it is total number of threads and thus 1 means serial).
>>
>>>
>>> /var/spool/PBS/mom_priv/jobs/2053253.lc5.itc.virginia.edu.SC: line 22:
>>> 12201
>>> Illegal instruction     /h3/n1/shirtsgroup/gromac
>>> s_46/install/bin/mdrun_d -ntomp 1 -ntmpi -8 -deffnm
>>> /bigtmp/mrs5pt/eth.vrescale.50
>>>
>>
>>  I suppose the CPU on the compute node is different from the build host.
>> You need to change GMX_CPU_ACCELERATION to the one correct for the compute
>> node. It could also help to set GMX_DISTRIBUTABLE_BUILD (both are cmake
>> options).
>>
>
>  GMX_DISTRIBUTABLE_BUILD only does one thing, it turns off rdtscp. Is the
> intention to provide more features for this option? If not, I don't see the
> point in not calling it GMX_DISABLE_RDTSCP.
>
I called it that for two reasons:
- Users don't know whether they want to disabled RDTSCP but they might know
whether they want to have a distributable build. In other words, in this
case the ultimate goal ("make it work on a different CPU") makes more sense
to the user then how this is achieved.
- It can be extended in the furture.

But I think we should consider to disable rdtscp by default. Unless we add
a runtime detection. The advantage is to small to cause problems. And as
far as I know the only other option which can cause illegal instruction is
GMX_CPU_ACCELERATION. But that one is pretty obvious to the user (it is a
non-advanced cmake option and I think people are much more likely to have
heard of SSE then rdtscp).

Roland

>
>  Cheers,
> --
> Szilárd
>
>
>>
>>  Roland
>>
>>
>>>
>>> Best,
>>>
>>> ~~~~~~~~~~~~
>>> Michael Shirts
>>> Assistant Professor
>>> Department of Chemical Engineering
>>> University of Virginia
>>> michael.shirts at virginia.edu
>>> (434)-243-1821
>>>
>>>
>>>  > From: Berk Hess <hess at kth.se>
>>> > Date: Sun, 2 Dec 2012 09:34:11 +0100
>>> > To: "michael.shirts at virginia.edu" <michael.shirts at virginia.edu>,
>>> Discussion
>>> > list for GROMACS development <gmx-developers at gromacs.org>
>>> > Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of
>>> > thread-mpi despite trying to convince it?
>>>  >
>>>
>>> > Hi,
>>> >
>>> > You queuing system probably doesn't set OMP_NUM_THREADS then
>>> > and I assume this machine has at least 16 (HT) cores.
>>> > mdrun -ntmpi 8 -ntomp 1
>>> > will do what you want, or:
>>> > mdrun -nt 8 -ntmpi 8
>>> >
>>> > Cheers,
>>> >
>>> > Berk
>>> >
>>> > On 12/02/2012 08:28 AM, Shirts, Michael (mrs5pt) wrote:
>>> >> Quick question:
>>> >>
>>> >> Compiling the most recent code in release-4-6, I compile without
>>> OpenMP
>>> >> (because using group rather than verlet cutoffs), and using any of the
>>> >> below:
>>> >>
>>> >> mdrun_d -ntmpi 8 -deffnm ethrun
>>> >> or
>>> >> mdrun_d -nt 8 -deffnm ethrun
>>> >> or
>>> >> mdrun_d -deffnm ethrun
>>> >> or
>>> >> mdrun_d -ntomp 0 -deffnm ethrun
>>> >> or
>>> >> mdrun_d -ntomp 0 -ntmpi8 -deffnm ethrun
>>> >>
>>> >> I get:
>>> >> Fatal error:
>>> >> OpenMP threads are requested, but Gromacs was compiled without OpenMP
>>> >> support
>>> >> For more information and tips for troubleshooting, please check the
>>> GROMACS
>>> >> website at http://www.gromacs.org/Documentation/Errors
>>> >>
>>> >> Even though I'm presumably requesting thread-mpi.  Worked fine with
>>> -nt
>>> >> previously (before the new -nt options introduced a few months back).
>>> >>
>>> >> Any suggestions or something I'm doing wrong?  Perhaps gromacs is
>>> >> interpreting the cluster environment as requesting OpenMP somehow?
>>> FWIW, the
>>> >> PBS script request line is "#PBS -l select=1:mpiprocs=8:ncpus=8".
>>> >>
>>> >> Apologies if I missed this answers somewhere out there already.
>>> >>
>>> >> Thanks,
>>> >> ~~~~~~~~~~~~
>>> >> Michael Shirts
>>> >> Assistant Professor
>>> >> Department of Chemical Engineering
>>> >> University of Virginia
>>> >> michael.shirts at virginia.edu
>>> >> (434)-243-1821
>>> >>
>>> >
>>>
>>> --
>>> gmx-developers mailing list
>>> gmx-developers at gromacs.org
>>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>>> Please don't post (un)subscribe requests to the list. Use the
>>> www interface or send it to gmx-developers-request at gromacs.org.
>>>
>>>
>>>
>>>
>>>
>>
>>
>>  --
>> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
>> 865-241-1537, ORNL PO BOX 2008 MS6309
>>
>> --
>> gmx-developers mailing list
>> gmx-developers at gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>> Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-developers-request at gromacs.org.
>>
>
>

-- 
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20121203/80d70643/attachment.html>