Hi,<div><br></div><div>I think both the error message when <span style="font-family:arial,sans-serif;font-size:12.800000190734863px">OMP_NUM_THREADS is set (especially for group kernels) and the error message if </span><font face="arial, sans-serif">rdtscp isn't supported (this is disabled by </font><span style="font-family:arial,sans-serif;font-size:12.800000190734863px">GMX_DISTRIBUTABLE_BUILD</span><span style="font-family:arial,sans-serif;font-size:12.800000190734863px"> ) </span><span style="font-family:arial,sans-serif">should be improved (anything is better than illegal instruction). So I would open a bug so we can track that. </span></div>
<div><span style="font-family:arial,sans-serif"><br></span></div><div><span style="font-family:arial,sans-serif">Roland</span></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Dec 3, 2012 at 10:50 AM, Shirts, Michael (mrs5pt) <span dir="ltr"><<a href="mailto:mrs5pt@eservices.virginia.edu" target="_blank">mrs5pt@eservices.virginia.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">So in the end, setting GMX_DISTRIBUTABLE_BUILD seems to have been the<br>
solution. I only compiled with SSE2, so I don't think it was a<br>
GMX_CPU_ACCELERATION mismatch.<br>
<br>
To summarize, I cleaned everything out, configured with<br>
<br>
cmake ../gromacs -DGMX_DISTRIBUTABLE_BUILD=Yes -DGMX_GPU=No -DGMX_DOUBLE=Yes<br>
-DGMX_CPU_ACCELERATION=SSE2 -DGMX_INSTALL_PRE<br>
<br>
And ran with:<br>
<br>
export OMP_NUM_THREADS=1<br>
$BINPATH/mdrun_d -ntmpi 8 -deffnm $DATAPATH/eth.vrescale.50<br>
<br>
And everything seems to work OK now. Possibly I had some issues from not<br>
having a clean configure? Apologies for not being able to generate a more<br>
reproducible error log.<br>
<br>
Thanks all for suggestions and comments.<br>
<div class="im"><br>
Best,<br>
~~~~~~~~~~~~<br>
Michael Shirts<br>
Assistant Professor<br>
Department of Chemical Engineering<br>
University of Virginia<br>
<a href="mailto:michael.shirts@virginia.edu">michael.shirts@virginia.edu</a><br>
<a href="tel:%28434%29-243-1821" value="+14342431821">(434)-243-1821</a><br>
<br>
<br>
</div>> From: Roland Schulz <<a href="mailto:roland@utk.edu">roland@utk.edu</a>><br>
> Date: Sun, 2 Dec 2012 23:12:31 -0500<br>
<div class="HOEnZb"><div class="h5">> To: "<a href="mailto:michael.shirts@virginia.edu">michael.shirts@virginia.edu</a>" <<a href="mailto:michael.shirts@virginia.edu">michael.shirts@virginia.edu</a>>, Discussion<br>
> list for GROMACS development <<a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a>><br>
> Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of<br>
> thread-mpi despite trying to convince it?<br>
><br>
> On Sun, Dec 2, 2012 at 12:58 PM, Shirts, Michael (mrs5pt) <<br>
> <a href="mailto:mrs5pt@eservices.virginia.edu">mrs5pt@eservices.virginia.edu</a>> wrote:<br>
><br>
>> So, more progress, but no simulations running yet.<br>
>><br>
>> mdrun -nt 8 -ntmpi 8 gives the same error as before (I actually tried that<br>
>> before, and forgot to include it in my error report)<br>
>><br>
>> mdrun -ntmpi 8 -ntomp 1 gives the error<br>
>> Fatal error:<br>
>> OMP_NUM_THREADS (8) and the number of threads requested on the command line<br>
>> (1) have different values<br>
>> For more information and tips for troubleshooting, please check the GROMACS<br>
>> website at <a href="http://www.gromacs.org/Documentation/Errors" target="_blank">http://www.gromacs.org/Documentation/Errors</a><br>
>><br>
><br>
> We probably should print a notice that OMP_NUM_THREADS is set. Otherwise<br>
> this is really confusing if OMP_NUM_THREADS isn't set by the user but by<br>
> the system.<br>
><br>
><br>
>> Fatal error:<br>
>> OMP_NUM_THREADS is invalid: '0'<br>
>><br>
><br>
> This is also for ntomp. Also there you wand to use 1 not 0 to disable<br>
> OpenMP (1 because it is total number of threads and thus 1 means serial).<br>
><br>
>><br>
>> /var/spool/PBS/mom_priv/jobs/<a href="http://2053253.lc5.itc.virginia.edu.SC" target="_blank">2053253.lc5.itc.virginia.edu.SC</a>: line 22:<br>
>> 12201<br>
>> Illegal instruction /h3/n1/shirtsgroup/gromac<br>
>> s_46/install/bin/mdrun_d -ntomp 1 -ntmpi -8 -deffnm<br>
>> /bigtmp/mrs5pt/eth.vrescale.50<br>
>><br>
><br>
> I suppose the CPU on the compute node is different from the build host. You<br>
> need to change GMX_CPU_ACCELERATION to the one correct for the compute<br>
> node. It could also help to set GMX_DISTRIBUTABLE_BUILD (both are cmake<br>
> options).<br>
><br>
> Roland<br>
><br>
><br>
>><br>
>> Best,<br>
>> ~~~~~~~~~~~~<br>
>> Michael Shirts<br>
>> Assistant Professor<br>
>> Department of Chemical Engineering<br>
>> University of Virginia<br>
>> <a href="mailto:michael.shirts@virginia.edu">michael.shirts@virginia.edu</a><br>
>> <a href="tel:%28434%29-243-1821" value="+14342431821">(434)-243-1821</a><br>
>><br>
>><br>
>>> From: Berk Hess <<a href="mailto:hess@kth.se">hess@kth.se</a>><br>
>>> Date: Sun, 2 Dec 2012 09:34:11 +0100<br>
>>> To: "<a href="mailto:michael.shirts@virginia.edu">michael.shirts@virginia.edu</a>" <<a href="mailto:michael.shirts@virginia.edu">michael.shirts@virginia.edu</a>>,<br>
>> Discussion<br>
>>> list for GROMACS development <<a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a>><br>
>>> Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of<br>
>>> thread-mpi despite trying to convince it?<br>
>>><br>
>>> Hi,<br>
>>><br>
>>> You queuing system probably doesn't set OMP_NUM_THREADS then<br>
>>> and I assume this machine has at least 16 (HT) cores.<br>
>>> mdrun -ntmpi 8 -ntomp 1<br>
>>> will do what you want, or:<br>
>>> mdrun -nt 8 -ntmpi 8<br>
>>><br>
>>> Cheers,<br>
>>><br>
>>> Berk<br>
>>><br>
>>> On 12/02/2012 08:28 AM, Shirts, Michael (mrs5pt) wrote:<br>
>>>> Quick question:<br>
>>>><br>
>>>> Compiling the most recent code in release-4-6, I compile without OpenMP<br>
>>>> (because using group rather than verlet cutoffs), and using any of the<br>
>>>> below:<br>
>>>><br>
>>>> mdrun_d -ntmpi 8 -deffnm ethrun<br>
>>>> or<br>
>>>> mdrun_d -nt 8 -deffnm ethrun<br>
>>>> or<br>
>>>> mdrun_d -deffnm ethrun<br>
>>>> or<br>
>>>> mdrun_d -ntomp 0 -deffnm ethrun<br>
>>>> or<br>
>>>> mdrun_d -ntomp 0 -ntmpi8 -deffnm ethrun<br>
>>>><br>
>>>> I get:<br>
>>>> Fatal error:<br>
>>>> OpenMP threads are requested, but Gromacs was compiled without OpenMP<br>
>>>> support<br>
>>>> For more information and tips for troubleshooting, please check the<br>
>> GROMACS<br>
>>>> website at <a href="http://www.gromacs.org/Documentation/Errors" target="_blank">http://www.gromacs.org/Documentation/Errors</a><br>
>>>><br>
>>>> Even though I'm presumably requesting thread-mpi. Worked fine with -nt<br>
>>>> previously (before the new -nt options introduced a few months back).<br>
>>>><br>
>>>> Any suggestions or something I'm doing wrong? Perhaps gromacs is<br>
>>>> interpreting the cluster environment as requesting OpenMP somehow?<br>
>> FWIW, the<br>
>>>> PBS script request line is "#PBS -l select=1:mpiprocs=8:ncpus=8".<br>
>>>><br>
>>>> Apologies if I missed this answers somewhere out there already.<br>
>>>><br>
>>>> Thanks,<br>
>>>> ~~~~~~~~~~~~<br>
>>>> Michael Shirts<br>
>>>> Assistant Professor<br>
>>>> Department of Chemical Engineering<br>
>>>> University of Virginia<br>
>>>> <a href="mailto:michael.shirts@virginia.edu">michael.shirts@virginia.edu</a><br>
>>>> <a href="tel:%28434%29-243-1821" value="+14342431821">(434)-243-1821</a><br>
>>>><br>
>>><br>
>><br>
>> --<br>
>> gmx-developers mailing list<br>
>> <a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a><br>
>> <a href="http://lists.gromacs.org/mailman/listinfo/gmx-developers" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-developers</a><br>
>> Please don't post (un)subscribe requests to the list. Use the<br>
>> www interface or send it to <a href="mailto:gmx-developers-request@gromacs.org">gmx-developers-request@gromacs.org</a>.<br>
>><br>
>><br>
>><br>
>><br>
>><br>
><br>
><br>
> --<br>
> ORNL/UT Center for Molecular Biophysics <a href="http://cmb.ornl.gov" target="_blank">cmb.ornl.gov</a><br>
> <a href="tel:865-241-1537" value="+18652411537">865-241-1537</a>, ORNL PO BOX 2008 MS6309<br>
<br>
<br>
<br>
<br>
<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>ORNL/UT Center for Molecular Biophysics <a href="http://cmb.ornl.gov">cmb.ornl.gov</a><br>865-241-1537, ORNL PO BOX 2008 MS6309<br>
</div>