[gmx-developers] Gromacs trying to use OpenMP instead of thread-mpi despite trying to convince it?

[Szilárd Páll]

On Mon, Dec 3, 2012 at 6:58 PM, Roland Schulz <roland at utk.edu> wrote:

> Hi,
>
> I think both the error message when OMP_NUM_THREADS is set (especially
> for group kernels) and the error message if rdtscp isn't supported (this
> is disabled by GMX_DISTRIBUTABLE_BUILD ) should be improved (anything is
> better than illegal instruction). So I would open a bug so we can track
> that.
>

As I said in my previous mail, the former should not be a concern (but do
let me know if you don't see the message on the stderr), the latter is IMO
way too specific. Users will get illegal instruction errors relatively
often as probably many are used to being able to run binaries across
different machines, something that will often not work with 4.6. In
general, if you want to be able to issue a warning before the user gets the
illegal instruction, you need to compile the cpuid code without
optimization and call it as the very first thing to check the capability of
the CPU against what the binary was compiled with. Even this might not work
with e.g. -march, but at least it will be useful in most cases and not only
when rdtscp is not supported.

Cheers,
--
Szilárd


>
> Roland
>
>
> On Mon, Dec 3, 2012 at 10:50 AM, Shirts, Michael (mrs5pt) <
> mrs5pt at eservices.virginia.edu> wrote:
>
>> So in the end, setting GMX_DISTRIBUTABLE_BUILD seems to have been the
>> solution.  I only compiled with SSE2, so I don't think it was a
>> GMX_CPU_ACCELERATION mismatch.
>>
>> To summarize, I cleaned everything out, configured with
>>
>> cmake ../gromacs -DGMX_DISTRIBUTABLE_BUILD=Yes -DGMX_GPU=No
>> -DGMX_DOUBLE=Yes
>> -DGMX_CPU_ACCELERATION=SSE2 -DGMX_INSTALL_PRE
>>
>> And ran with:
>>
>> export OMP_NUM_THREADS=1
>> $BINPATH/mdrun_d -ntmpi 8 -deffnm $DATAPATH/eth.vrescale.50
>>
>> And everything seems to work OK now.  Possibly I had some issues from not
>> having a clean configure?  Apologies for not being able to generate a more
>> reproducible error log.
>>
>> Thanks all for suggestions and comments.
>>
>> Best,
>> ~~~~~~~~~~~~
>> Michael Shirts
>> Assistant Professor
>> Department of Chemical Engineering
>> University of Virginia
>> michael.shirts at virginia.edu
>> (434)-243-1821
>>
>>
>> > From: Roland Schulz <roland at utk.edu>
>> > Date: Sun, 2 Dec 2012 23:12:31 -0500
>>
>> > To: "michael.shirts at virginia.edu" <michael.shirts at virginia.edu>,
>> Discussion
>> > list for GROMACS development <gmx-developers at gromacs.org>
>> > Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of
>> > thread-mpi despite trying to convince it?
>> >
>> > On Sun, Dec 2, 2012 at 12:58 PM, Shirts, Michael (mrs5pt) <
>> > mrs5pt at eservices.virginia.edu> wrote:
>> >
>> >> So, more progress, but no simulations running yet.
>> >>
>> >> mdrun -nt 8 -ntmpi 8 gives the same error as before (I actually tried
>> that
>> >> before, and forgot to include it in my error report)
>> >>
>> >> mdrun -ntmpi 8 -ntomp 1 gives the error
>> >> Fatal error:
>> >> OMP_NUM_THREADS (8) and the number of threads requested on the command
>> line
>> >> (1) have different values
>> >> For more information and tips for troubleshooting, please check the
>> GROMACS
>> >> website at http://www.gromacs.org/Documentation/Errors
>> >>
>> >
>> > We probably should print a notice that OMP_NUM_THREADS is set. Otherwise
>> > this is really confusing if OMP_NUM_THREADS isn't set by the user but by
>> > the system.
>> >
>> >
>> >> Fatal error:
>> >> OMP_NUM_THREADS is invalid: '0'
>> >>
>> >
>> > This is also for ntomp. Also there you wand to use 1 not 0 to disable
>> > OpenMP (1 because it is total number of threads and thus 1 means
>> serial).
>> >
>> >>
>> >> /var/spool/PBS/mom_priv/jobs/2053253.lc5.itc.virginia.edu.SC: line 22:
>> >> 12201
>> >> Illegal instruction     /h3/n1/shirtsgroup/gromac
>> >> s_46/install/bin/mdrun_d -ntomp 1 -ntmpi -8 -deffnm
>> >> /bigtmp/mrs5pt/eth.vrescale.50
>> >>
>> >
>> > I suppose the CPU on the compute node is different from the build host.
>> You
>> > need to change GMX_CPU_ACCELERATION to the one correct for the compute
>> > node. It could also help to set GMX_DISTRIBUTABLE_BUILD (both are cmake
>> > options).
>> >
>> > Roland
>> >
>> >
>> >>
>> >> Best,
>> >> ~~~~~~~~~~~~
>> >> Michael Shirts
>> >> Assistant Professor
>> >> Department of Chemical Engineering
>> >> University of Virginia
>> >> michael.shirts at virginia.edu
>> >> (434)-243-1821
>> >>
>> >>
>> >>> From: Berk Hess <hess at kth.se>
>> >>> Date: Sun, 2 Dec 2012 09:34:11 +0100
>> >>> To: "michael.shirts at virginia.edu" <michael.shirts at virginia.edu>,
>> >> Discussion
>> >>> list for GROMACS development <gmx-developers at gromacs.org>
>> >>> Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of
>> >>> thread-mpi despite trying to convince it?
>> >>>
>> >>> Hi,
>> >>>
>> >>> You queuing system probably doesn't set OMP_NUM_THREADS then
>> >>> and I assume this machine has at least 16 (HT) cores.
>> >>> mdrun -ntmpi 8 -ntomp 1
>> >>> will do what you want, or:
>> >>> mdrun -nt 8 -ntmpi 8
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Berk
>> >>>
>> >>> On 12/02/2012 08:28 AM, Shirts, Michael (mrs5pt) wrote:
>> >>>> Quick question:
>> >>>>
>> >>>> Compiling the most recent code in release-4-6, I compile without
>> OpenMP
>> >>>> (because using group rather than verlet cutoffs), and using any of
>> the
>> >>>> below:
>> >>>>
>> >>>> mdrun_d -ntmpi 8 -deffnm ethrun
>> >>>> or
>> >>>> mdrun_d -nt 8 -deffnm ethrun
>> >>>> or
>> >>>> mdrun_d -deffnm ethrun
>> >>>> or
>> >>>> mdrun_d -ntomp 0 -deffnm ethrun
>> >>>> or
>> >>>> mdrun_d -ntomp 0 -ntmpi8 -deffnm ethrun
>> >>>>
>> >>>> I get:
>> >>>> Fatal error:
>> >>>> OpenMP threads are requested, but Gromacs was compiled without OpenMP
>> >>>> support
>> >>>> For more information and tips for troubleshooting, please check the
>> >> GROMACS
>> >>>> website at http://www.gromacs.org/Documentation/Errors
>> >>>>
>> >>>> Even though I'm presumably requesting thread-mpi.  Worked fine with
>> -nt
>> >>>> previously (before the new -nt options introduced a few months back).
>> >>>>
>> >>>> Any suggestions or something I'm doing wrong?  Perhaps gromacs is
>> >>>> interpreting the cluster environment as requesting OpenMP somehow?
>> >> FWIW, the
>> >>>> PBS script request line is "#PBS -l select=1:mpiprocs=8:ncpus=8".
>> >>>>
>> >>>> Apologies if I missed this answers somewhere out there already.
>> >>>>
>> >>>> Thanks,
>> >>>> ~~~~~~~~~~~~
>> >>>> Michael Shirts
>> >>>> Assistant Professor
>> >>>> Department of Chemical Engineering
>> >>>> University of Virginia
>> >>>> michael.shirts at virginia.edu
>> >>>> (434)-243-1821
>> >>>>
>> >>>
>> >>
>> >> --
>> >> gmx-developers mailing list
>> >> gmx-developers at gromacs.org
>> >> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>> >> Please don't post (un)subscribe requests to the list. Use the
>> >> www interface or send it to gmx-developers-request at gromacs.org.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
>> > 865-241-1537, ORNL PO BOX 2008 MS6309
>>
>>
>>
>>
>>
>>
>
>
> --
> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> 865-241-1537, ORNL PO BOX 2008 MS6309
>
> --
> gmx-developers mailing list
> gmx-developers at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-developers-request at gromacs.org.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20121203/ead97dc2/attachment.html>