[gmx-developers] Gromacs trying to use OpenMP instead of thread-mpi despite trying to convince it?

Mon Dec 3 18:58:35 CET 2012

Hi,

I think both the error message when OMP_NUM_THREADS is set (especially for
group kernels) and the error message if rdtscp isn't supported (this is
disabled by GMX_DISTRIBUTABLE_BUILD ) should be improved (anything is
better than illegal instruction). So I would open a bug so we can track
that.

Roland

On Mon, Dec 3, 2012 at 10:50 AM, Shirts, Michael (mrs5pt) <
mrs5pt at eservices.virginia.edu> wrote:

> So in the end, setting GMX_DISTRIBUTABLE_BUILD seems to have been the
> solution.  I only compiled with SSE2, so I don't think it was a
> GMX_CPU_ACCELERATION mismatch.
>
> To summarize, I cleaned everything out, configured with
>
> cmake ../gromacs -DGMX_DISTRIBUTABLE_BUILD=Yes -DGMX_GPU=No
> -DGMX_DOUBLE=Yes
> -DGMX_CPU_ACCELERATION=SSE2 -DGMX_INSTALL_PRE
>
> And ran with:
>
> export OMP_NUM_THREADS=1
> $BINPATH/mdrun_d -ntmpi 8 -deffnm $DATAPATH/eth.vrescale.50
>
> And everything seems to work OK now.  Possibly I had some issues from not
> having a clean configure?  Apologies for not being able to generate a more
> reproducible error log.
>
> Thanks all for suggestions and comments.
>
> Best,
> ~~~~~~~~~~~~
> Michael Shirts
> Assistant Professor
> Department of Chemical Engineering
> University of Virginia
> michael.shirts at virginia.edu
> (434)-243-1821
>
>
> > From: Roland Schulz <roland at utk.edu>
> > Date: Sun, 2 Dec 2012 23:12:31 -0500
> > To: "michael.shirts at virginia.edu" <michael.shirts at virginia.edu>,
> Discussion
> > list for GROMACS development <gmx-developers at gromacs.org>
> > Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of
> > thread-mpi despite trying to convince it?
> >
> > On Sun, Dec 2, 2012 at 12:58 PM, Shirts, Michael (mrs5pt) <
> > mrs5pt at eservices.virginia.edu> wrote:
> >
> >> So, more progress, but no simulations running yet.
> >>
> >> mdrun -nt 8 -ntmpi 8 gives the same error as before (I actually tried
> that
> >> before, and forgot to include it in my error report)
> >>
> >> mdrun -ntmpi 8 -ntomp 1 gives the error
> >> Fatal error:
> >> OMP_NUM_THREADS (8) and the number of threads requested on the command
> line
> >> (1) have different values
> >> For more information and tips for troubleshooting, please check the
> GROMACS
> >> website at http://www.gromacs.org/Documentation/Errors
> >>
> >
> > We probably should print a notice that OMP_NUM_THREADS is set. Otherwise
> > this is really confusing if OMP_NUM_THREADS isn't set by the user but by
> > the system.
> >
> >
> >> Fatal error:
> >> OMP_NUM_THREADS is invalid: '0'
> >>
> >
> > This is also for ntomp. Also there you wand to use 1 not 0 to disable
> > OpenMP (1 because it is total number of threads and thus 1 means serial).
> >
> >>
> >> /var/spool/PBS/mom_priv/jobs/2053253.lc5.itc.virginia.edu.SC: line 22:
> >> 12201
> >> Illegal instruction     /h3/n1/shirtsgroup/gromac
> >> s_46/install/bin/mdrun_d -ntomp 1 -ntmpi -8 -deffnm
> >> /bigtmp/mrs5pt/eth.vrescale.50
> >>
> >
> > I suppose the CPU on the compute node is different from the build host.
> You
> > need to change GMX_CPU_ACCELERATION to the one correct for the compute
> > node. It could also help to set GMX_DISTRIBUTABLE_BUILD (both are cmake
> > options).
> >
> > Roland
> >
> >
> >>
> >> Best,
> >> ~~~~~~~~~~~~
> >> Michael Shirts
> >> Assistant Professor
> >> Department of Chemical Engineering
> >> University of Virginia
> >> michael.shirts at virginia.edu
> >> (434)-243-1821
> >>
> >>
> >>> From: Berk Hess <hess at kth.se>
> >>> Date: Sun, 2 Dec 2012 09:34:11 +0100
> >>> To: "michael.shirts at virginia.edu" <michael.shirts at virginia.edu>,
> >> Discussion
> >>> list for GROMACS development <gmx-developers at gromacs.org>
> >>> Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of
> >>> thread-mpi despite trying to convince it?
> >>>
> >>> Hi,
> >>>
> >>> You queuing system probably doesn't set OMP_NUM_THREADS then
> >>> and I assume this machine has at least 16 (HT) cores.
> >>> mdrun -ntmpi 8 -ntomp 1
> >>> will do what you want, or:
> >>> mdrun -nt 8 -ntmpi 8
> >>>
> >>> Cheers,
> >>>
> >>> Berk
> >>>
> >>> On 12/02/2012 08:28 AM, Shirts, Michael (mrs5pt) wrote:
> >>>> Quick question:
> >>>>
> >>>> Compiling the most recent code in release-4-6, I compile without
> OpenMP
> >>>> (because using group rather than verlet cutoffs), and using any of the
> >>>> below:
> >>>>
> >>>> mdrun_d -ntmpi 8 -deffnm ethrun
> >>>> or
> >>>> mdrun_d -nt 8 -deffnm ethrun
> >>>> or
> >>>> mdrun_d -deffnm ethrun
> >>>> or
> >>>> mdrun_d -ntomp 0 -deffnm ethrun
> >>>> or
> >>>> mdrun_d -ntomp 0 -ntmpi8 -deffnm ethrun
> >>>>
> >>>> I get:
> >>>> Fatal error:
> >>>> OpenMP threads are requested, but Gromacs was compiled without OpenMP
> >>>> support
> >>>> For more information and tips for troubleshooting, please check the
> >> GROMACS
> >>>> website at http://www.gromacs.org/Documentation/Errors
> >>>>
> >>>> Even though I'm presumably requesting thread-mpi.  Worked fine with
> -nt
> >>>> previously (before the new -nt options introduced a few months back).
> >>>>
> >>>> Any suggestions or something I'm doing wrong?  Perhaps gromacs is
> >>>> interpreting the cluster environment as requesting OpenMP somehow?
> >> FWIW, the
> >>>> PBS script request line is "#PBS -l select=1:mpiprocs=8:ncpus=8".
> >>>>
> >>>> Apologies if I missed this answers somewhere out there already.
> >>>>
> >>>> Thanks,
> >>>> ~~~~~~~~~~~~
> >>>> Michael Shirts
> >>>> Assistant Professor
> >>>> Department of Chemical Engineering
> >>>> University of Virginia
> >>>> michael.shirts at virginia.edu
> >>>> (434)-243-1821
> >>>>
> >>>
> >>
> >> --
> >> gmx-developers mailing list
> >> gmx-developers at gromacs.org
> >> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> >> Please don't post (un)subscribe requests to the list. Use the
> >> www interface or send it to gmx-developers-request at gromacs.org.
> >>
> >>
> >>
> >>
> >>
> >
> >
> > --
> > ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> > 865-241-1537, ORNL PO BOX 2008 MS6309
>
>
>
>
>
>

-- 
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20121203/bbe202ae/attachment.html>