[gmx-developers] Gromacs trying to use OpenMP instead of thread-mpi despite trying to convince it?

Mon Dec 3 22:20:26 CET 2012

On Mon, Dec 3, 2012 at 9:38 PM, Shirts, Michael (mrs5pt) <
mrs5pt at eservices.virginia.edu> wrote:

> Just to chime in, the illegal instruction error is definitely a bad one.
> However, a FAQ that clearly flags this error and that the solution is
> fixing
> those two compiler flags as the likely fix is probably enough for now.
>

Note that modifying compiler flags won't help because in these cases you'd
typically need to "lower" the CPU acceleration. This sis currently not
possible without re-configuring.

--
Szilárd

>
> Best,
> ~~~~~~~~~~~~
> Michael Shirts
> Assistant Professor
> Department of Chemical Engineering
> University of Virginia
> michael.shirts at virginia.edu
> (434)-243-1821
>
>
> > From: Roland Schulz <roland at utk.edu>
> > Reply-To: Discussion list for GROMACS development <
> gmx-developers at gromacs.org>
> > Date: Mon, 3 Dec 2012 13:18:27 -0500
> > To: Discussion list for GROMACS development <gmx-developers at gromacs.org>
> > Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of
> > thread-mpi despite trying to convince it?
> >
> > On Mon, Dec 3, 2012 at 1:02 PM, Szilárd Páll <szilard.pall at cbr.su.se>
> wrote:
> >
> >>
> >>  On Mon, Dec 3, 2012 at 5:12 AM, Roland Schulz <roland at utk.edu> wrote:
> >>
> >>>
> >>>
> >>>
> >>>  On Sun, Dec 2, 2012 at 12:58 PM, Shirts, Michael (mrs5pt) <
> >>> mrs5pt at eservices.virginia.edu> wrote:
> >>>
> >>>> So, more progress, but no simulations running yet.
> >>>>
> >>>> mdrun -nt 8 -ntmpi 8 gives the same error as before (I actually tried
> >>>> that
> >>>> before, and forgot to include it in my error report)
> >>>>
> >>>> mdrun -ntmpi 8 -ntomp 1 gives the error
> >>>> Fatal error:
> >>>> OMP_NUM_THREADS (8) and the number of threads requested on the command
> >>>> line
> >>>> (1) have different values
> >>>>  For more information and tips for troubleshooting, please check the
> >>>> GROMACS
> >>>> website at http://www.gromacs.org/Documentation/Errors
> >>>>
> >>>
> >>>  We probably should print a notice that OMP_NUM_THREADS is set.
> >>> Otherwise this is really confusing if OMP_NUM_THREADS isn't set by the
> user
> >>> but by the system.
> >>>
> >>
> >>  There is a note printed whenever the number of OpenMP threads is set by
> >> OMP_NUM_THREADS instead of -ntomp.
> >>
> > I think if even core developers don't understand/find a note, then
> that's a
> > pretty clear sign that it will be confusing to the average user ;-)
> >
> >
> >>
> >>
> >>>
> >>>
> >>>> Fatal error:
> >>>> OMP_NUM_THREADS is invalid: '0'
> >>>>
> >>>
> >>>  This is also for ntomp. Also there you wand to use 1 not 0 to disable
> >>> OpenMP (1 because it is total number of threads and thus 1 means
> serial).
> >>>
> >>>>
> >>>> /var/spool/PBS/mom_priv/jobs/2053253.lc5.itc.virginia.edu.SC: line
> 22:
> >>>> 12201
> >>>> Illegal instruction     /h3/n1/shirtsgroup/gromac
> >>>> s_46/install/bin/mdrun_d -ntomp 1 -ntmpi -8 -deffnm
> >>>> /bigtmp/mrs5pt/eth.vrescale.50
> >>>>
> >>>
> >>>  I suppose the CPU on the compute node is different from the build
> host.
> >>> You need to change GMX_CPU_ACCELERATION to the one correct for the
> compute
> >>> node. It could also help to set GMX_DISTRIBUTABLE_BUILD (both are cmake
> >>> options).
> >>>
> >>
> >>  GMX_DISTRIBUTABLE_BUILD only does one thing, it turns off rdtscp. Is
> the
> >> intention to provide more features for this option? If not, I don't see
> the
> >> point in not calling it GMX_DISABLE_RDTSCP.
> >>
> > I called it that for two reasons:
> > - Users don't know whether they want to disabled RDTSCP but they might
> know
> > whether they want to have a distributable build. In other words, in this
> > case the ultimate goal ("make it work on a different CPU") makes more
> sense
> > to the user then how this is achieved.
> > - It can be extended in the furture.
> >
> > But I think we should consider to disable rdtscp by default. Unless we
> add
> > a runtime detection. The advantage is to small to cause problems. And as
> > far as I know the only other option which can cause illegal instruction
> is
> > GMX_CPU_ACCELERATION. But that one is pretty obvious to the user (it is a
> > non-advanced cmake option and I think people are much more likely to have
> > heard of SSE then rdtscp).
> >
> > Roland
> >
> >
> >
> >>
> >>  Cheers,
> >> --
> >> Szilárd
> >>
> >>
> >>>
> >>>  Roland
> >>>
> >>>
> >>>>
> >>>> Best,
> >>>>
> >>>> ~~~~~~~~~~~~
> >>>> Michael Shirts
> >>>> Assistant Professor
> >>>> Department of Chemical Engineering
> >>>> University of Virginia
> >>>> michael.shirts at virginia.edu
> >>>> (434)-243-1821
> >>>>
> >>>>
> >>>>> From: Berk Hess <hess at kth.se>
> >>>>> Date: Sun, 2 Dec 2012 09:34:11 +0100
> >>>>> To: "michael.shirts at virginia.edu" <michael.shirts at virginia.edu>,
> >>>> Discussion
> >>>>> list for GROMACS development <gmx-developers at gromacs.org>
> >>>>> Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of
> >>>>> thread-mpi despite trying to convince it?
> >>>>>
> >>>>
> >>>>> Hi,
> >>>>>
> >>>>> You queuing system probably doesn't set OMP_NUM_THREADS then
> >>>>> and I assume this machine has at least 16 (HT) cores.
> >>>>> mdrun -ntmpi 8 -ntomp 1
> >>>>> will do what you want, or:
> >>>>> mdrun -nt 8 -ntmpi 8
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>> Berk
> >>>>>
> >>>>> On 12/02/2012 08:28 AM, Shirts, Michael (mrs5pt) wrote:
> >>>>>> Quick question:
> >>>>>>
> >>>>>> Compiling the most recent code in release-4-6, I compile without
> >>>> OpenMP
> >>>>>> (because using group rather than verlet cutoffs), and using any of
> the
> >>>>>> below:
> >>>>>>
> >>>>>> mdrun_d -ntmpi 8 -deffnm ethrun
> >>>>>> or
> >>>>>> mdrun_d -nt 8 -deffnm ethrun
> >>>>>> or
> >>>>>> mdrun_d -deffnm ethrun
> >>>>>> or
> >>>>>> mdrun_d -ntomp 0 -deffnm ethrun
> >>>>>> or
> >>>>>> mdrun_d -ntomp 0 -ntmpi8 -deffnm ethrun
> >>>>>>
> >>>>>> I get:
> >>>>>> Fatal error:
> >>>>>> OpenMP threads are requested, but Gromacs was compiled without
> OpenMP
> >>>>>> support
> >>>>>> For more information and tips for troubleshooting, please check the
> >>>> GROMACS
> >>>>>> website at http://www.gromacs.org/Documentation/Errors
> >>>>>>
> >>>>>> Even though I'm presumably requesting thread-mpi.  Worked fine with
> >>>> -nt
> >>>>>> previously (before the new -nt options introduced a few months
> back).
> >>>>>>
> >>>>>> Any suggestions or something I'm doing wrong?  Perhaps gromacs is
> >>>>>> interpreting the cluster environment as requesting OpenMP somehow?
> >>>> FWIW, the
> >>>>>> PBS script request line is "#PBS -l select=1:mpiprocs=8:ncpus=8".
> >>>>>>
> >>>>>> Apologies if I missed this answers somewhere out there already.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> ~~~~~~~~~~~~
> >>>>>> Michael Shirts
> >>>>>> Assistant Professor
> >>>>>> Department of Chemical Engineering
> >>>>>> University of Virginia
> >>>>>> michael.shirts at virginia.edu
> >>>>>> (434)-243-1821
> >>>>>>
> >>>>>
> >>>>
> >>>> --
> >>>> gmx-developers mailing list
> >>>> gmx-developers at gromacs.org
> >>>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> >>>> Please don't post (un)subscribe requests to the list. Use the
> >>>> www interface or send it to gmx-developers-request at gromacs.org.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>  --
> >>> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> >>> 865-241-1537, ORNL PO BOX 2008 MS6309
> >>>
> >>> --
> >>> gmx-developers mailing list
> >>> gmx-developers at gromacs.org
> >>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> >>> Please don't post (un)subscribe requests to the list. Use the
> >>> www interface or send it to gmx-developers-request at gromacs.org.
> >>>
> >>
> >>
> >
> >
> > --
> > ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> > 865-241-1537, ORNL PO BOX 2008 MS6309
> > --
> > gmx-developers mailing list
> > gmx-developers at gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-developers
> > Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-developers-request at gromacs.org.
>
> --
> gmx-developers mailing list
> gmx-developers at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-developers-request at gromacs.org.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20121203/ec066fa4/attachment.html>