[gmx-developers] Gromacs trying to use OpenMP instead of thread-mpi despite trying to convince it?

[Shirts, Michael (mrs5pt)]

So in the end, setting GMX_DISTRIBUTABLE_BUILD seems to have been the
solution.  I only compiled with SSE2, so I don't think it was a
GMX_CPU_ACCELERATION mismatch.

To summarize, I cleaned everything out, configured with

cmake ../gromacs -DGMX_DISTRIBUTABLE_BUILD=Yes -DGMX_GPU=No -DGMX_DOUBLE=Yes
-DGMX_CPU_ACCELERATION=SSE2 -DGMX_INSTALL_PRE

And ran with:

export OMP_NUM_THREADS=1
$BINPATH/mdrun_d -ntmpi 8 -deffnm $DATAPATH/eth.vrescale.50

And everything seems to work OK now.  Possibly I had some issues from not
having a clean configure?  Apologies for not being able to generate a more
reproducible error log.

Thanks all for suggestions and comments.

Best,
~~~~~~~~~~~~
Michael Shirts
Assistant Professor
Department of Chemical Engineering
University of Virginia
michael.shirts at virginia.edu
(434)-243-1821


> From: Roland Schulz <roland at utk.edu>
> Date: Sun, 2 Dec 2012 23:12:31 -0500
> To: "michael.shirts at virginia.edu" <michael.shirts at virginia.edu>, Discussion
> list for GROMACS development <gmx-developers at gromacs.org>
> Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of
> thread-mpi despite trying to convince it?
> 
> On Sun, Dec 2, 2012 at 12:58 PM, Shirts, Michael (mrs5pt) <
> mrs5pt at eservices.virginia.edu> wrote:
> 
>> So, more progress, but no simulations running yet.
>> 
>> mdrun -nt 8 -ntmpi 8 gives the same error as before (I actually tried that
>> before, and forgot to include it in my error report)
>> 
>> mdrun -ntmpi 8 -ntomp 1 gives the error
>> Fatal error:
>> OMP_NUM_THREADS (8) and the number of threads requested on the command line
>> (1) have different values
>> For more information and tips for troubleshooting, please check the GROMACS
>> website at http://www.gromacs.org/Documentation/Errors
>> 
> 
> We probably should print a notice that OMP_NUM_THREADS is set. Otherwise
> this is really confusing if OMP_NUM_THREADS isn't set by the user but by
> the system.
> 
> 
>> Fatal error:
>> OMP_NUM_THREADS is invalid: '0'
>> 
> 
> This is also for ntomp. Also there you wand to use 1 not 0 to disable
> OpenMP (1 because it is total number of threads and thus 1 means serial).
> 
>> 
>> /var/spool/PBS/mom_priv/jobs/2053253.lc5.itc.virginia.edu.SC: line 22:
>> 12201
>> Illegal instruction     /h3/n1/shirtsgroup/gromac
>> s_46/install/bin/mdrun_d -ntomp 1 -ntmpi -8 -deffnm
>> /bigtmp/mrs5pt/eth.vrescale.50
>> 
> 
> I suppose the CPU on the compute node is different from the build host. You
> need to change GMX_CPU_ACCELERATION to the one correct for the compute
> node. It could also help to set GMX_DISTRIBUTABLE_BUILD (both are cmake
> options).
> 
> Roland
> 
> 
>> 
>> Best,
>> ~~~~~~~~~~~~
>> Michael Shirts
>> Assistant Professor
>> Department of Chemical Engineering
>> University of Virginia
>> michael.shirts at virginia.edu
>> (434)-243-1821
>> 
>> 
>>> From: Berk Hess <hess at kth.se>
>>> Date: Sun, 2 Dec 2012 09:34:11 +0100
>>> To: "michael.shirts at virginia.edu" <michael.shirts at virginia.edu>,
>> Discussion
>>> list for GROMACS development <gmx-developers at gromacs.org>
>>> Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of
>>> thread-mpi despite trying to convince it?
>>> 
>>> Hi,
>>> 
>>> You queuing system probably doesn't set OMP_NUM_THREADS then
>>> and I assume this machine has at least 16 (HT) cores.
>>> mdrun -ntmpi 8 -ntomp 1
>>> will do what you want, or:
>>> mdrun -nt 8 -ntmpi 8
>>> 
>>> Cheers,
>>> 
>>> Berk
>>> 
>>> On 12/02/2012 08:28 AM, Shirts, Michael (mrs5pt) wrote:
>>>> Quick question:
>>>> 
>>>> Compiling the most recent code in release-4-6, I compile without OpenMP
>>>> (because using group rather than verlet cutoffs), and using any of the
>>>> below:
>>>> 
>>>> mdrun_d -ntmpi 8 -deffnm ethrun
>>>> or
>>>> mdrun_d -nt 8 -deffnm ethrun
>>>> or
>>>> mdrun_d -deffnm ethrun
>>>> or
>>>> mdrun_d -ntomp 0 -deffnm ethrun
>>>> or
>>>> mdrun_d -ntomp 0 -ntmpi8 -deffnm ethrun
>>>> 
>>>> I get:
>>>> Fatal error:
>>>> OpenMP threads are requested, but Gromacs was compiled without OpenMP
>>>> support
>>>> For more information and tips for troubleshooting, please check the
>> GROMACS
>>>> website at http://www.gromacs.org/Documentation/Errors
>>>> 
>>>> Even though I'm presumably requesting thread-mpi.  Worked fine with -nt
>>>> previously (before the new -nt options introduced a few months back).
>>>> 
>>>> Any suggestions or something I'm doing wrong?  Perhaps gromacs is
>>>> interpreting the cluster environment as requesting OpenMP somehow?
>> FWIW, the
>>>> PBS script request line is "#PBS -l select=1:mpiprocs=8:ncpus=8".
>>>> 
>>>> Apologies if I missed this answers somewhere out there already.
>>>> 
>>>> Thanks,
>>>> ~~~~~~~~~~~~
>>>> Michael Shirts
>>>> Assistant Professor
>>>> Department of Chemical Engineering
>>>> University of Virginia
>>>> michael.shirts at virginia.edu
>>>> (434)-243-1821
>>>> 
>>> 
>> 
>> --
>> gmx-developers mailing list
>> gmx-developers at gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>> Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-developers-request at gromacs.org.
>> 
>> 
>> 
>> 
>> 
> 
> 
> -- 
> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> 865-241-1537, ORNL PO BOX 2008 MS6309