[gmx-developers] Gromacs trying to use OpenMP instead of thread-mpi despite trying to convince it?

[Shirts, Michael (mrs5pt)]

So, more progress, but no simulations running yet.

mdrun -nt 8 -ntmpi 8 gives the same error as before (I actually tried that
before, and forgot to include it in my error report)

mdrun -ntmpi 8 -ntomp 1 gives the error
Fatal error:
OMP_NUM_THREADS (8) and the number of threads requested on the command line
(1) have different values
For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors

This at least gives a little more information.  I tried to set the
environment variable explicitly in the script as 0, which gives

export OMP_NUM_THREADS=0
mdrun -nt 8 -ntmpi 8

Which gives:

Fatal error:
OMP_NUM_THREADS is invalid: '0'

Finally, setting OMP_NUM_THREADS manually to one allows it to start

export OMP_NUM_THREADS=1
mdrun -ntomp 1 -ntmpi 8

But it exits with an error:

/var/spool/PBS/mom_priv/jobs/2053253.lc5.itc.virginia.edu.SC: line 22: 12201
Illegal instruction     /h3/n1/shirtsgroup/gromac
s_46/install/bin/mdrun_d -ntomp 1 -ntmpi -8 -deffnm
/bigtmp/mrs5pt/eth.vrescale.50

Best,
~~~~~~~~~~~~
Michael Shirts
Assistant Professor
Department of Chemical Engineering
University of Virginia
michael.shirts at virginia.edu
(434)-243-1821


> From: Berk Hess <hess at kth.se>
> Date: Sun, 2 Dec 2012 09:34:11 +0100
> To: "michael.shirts at virginia.edu" <michael.shirts at virginia.edu>, Discussion
> list for GROMACS development <gmx-developers at gromacs.org>
> Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of
> thread-mpi despite trying to convince it?
> 
> Hi,
> 
> You queuing system probably doesn't set OMP_NUM_THREADS then
> and I assume this machine has at least 16 (HT) cores.
> mdrun -ntmpi 8 -ntomp 1
> will do what you want, or:
> mdrun -nt 8 -ntmpi 8
> 
> Cheers,
> 
> Berk
> 
> On 12/02/2012 08:28 AM, Shirts, Michael (mrs5pt) wrote:
>> Quick question:
>> 
>> Compiling the most recent code in release-4-6, I compile without OpenMP
>> (because using group rather than verlet cutoffs), and using any of the
>> below:
>> 
>> mdrun_d -ntmpi 8 -deffnm ethrun
>> or
>> mdrun_d -nt 8 -deffnm ethrun
>> or
>> mdrun_d -deffnm ethrun
>> or
>> mdrun_d -ntomp 0 -deffnm ethrun
>> or
>> mdrun_d -ntomp 0 -ntmpi8 -deffnm ethrun
>> 
>> I get:
>> Fatal error:
>> OpenMP threads are requested, but Gromacs was compiled without OpenMP
>> support
>> For more information and tips for troubleshooting, please check the GROMACS
>> website at http://www.gromacs.org/Documentation/Errors
>> 
>> Even though I'm presumably requesting thread-mpi.  Worked fine with -nt
>> previously (before the new -nt options introduced a few months back).
>> 
>> Any suggestions or something I'm doing wrong?  Perhaps gromacs is
>> interpreting the cluster environment as requesting OpenMP somehow? FWIW, the
>> PBS script request line is "#PBS -l select=1:mpiprocs=8:ncpus=8".
>> 
>> Apologies if I missed this answers somewhere out there already.
>> 
>> Thanks,
>> ~~~~~~~~~~~~
>> Michael Shirts
>> Assistant Professor
>> Department of Chemical Engineering
>> University of Virginia
>> michael.shirts at virginia.edu
>> (434)-243-1821
>> 
>