[gmx-users] Gromacs 5.1.2 and OMP_NUM_THREADS

[Szilárd Páll]

Susan,

Have you tried mpirun --bind-to none? For the last few releases
OpenMPI messes with the CPUSET/affinities by default which may be
interacting badly with the Intel OpeMP library.

What about running with -pin on (or -pin off)?

Cheers,
--
Szilárd


On Tue, Jul 5, 2016 at 4:13 PM, Mark Abraham <mark.j.abraham at gmail.com> wrote:
> Hi,
>
> OpenMPI 1.10.0 has six months worth of bugs now fixed in 1.10.3, some of
> which seem plausible to explain this behaviour. There's been no GROMACS
> issue that seems similar. Please try another OpenMPI and let us know how
> you go!
>
> Mark
>
> On Tue, 5 Jul 2016 15:55 Susan Chacko <susanc at helix.nih.gov> wrote:
>
>>
>> Hi all,
>>
>> One of our users is having problems with Gromacs 5.1.2. hanging at the
>> start of an mdrun using OMP_NUM_THREADS=2. When run with OMP_NUM_THREADS=1,
>> the job runs fine.
>>
>> The stalling command is:
>> mpirun -np 128 mdrun_mpi -nb cpu -v -deffnm em
>>
>> The same command and job work fine in Gromacs 5.0.4 with OMP_NUM_THREADS=2
>>
>> Gromacs 5.0.4 and 5.1.2 were built on our system with Intel compiler
>> 2015.1.133:
>>
>> cmake ../gromacs-5.1.2  \
>> -DGMX_BUILD_OWN_FFTW=ON \
>> -DREGRESSIONTEST_DOWNLOAD=ON \
>> -DGMX_MPI=on \
>> -DGMX_BUILD_MDRUN_ONLY=on  \
>> -DBUILD_SHARED_LIBS=off
>>
>> One difference I can see is that Gromacs 5.0.4 was built with OpenMPI
>> 1.8.4, and Gromacs 5.1.2 was built with OpenMPI 1.10.0. Is that likely to
>> be the cause of the problem? If so, I could rebuilt Gromacs 5.1.2 with
>> OpenMPI 1.8.4
>>
>> Any ideas what might be causing the stall? Any other flags we should use
>> to compile?
>>
>> All suggestions appreciated,
>> Susan.
>>
>>
>> Susan Chacko, PhD
>> HPC @ NIH staff
>>
>>
>>
>>
>>
>> --
>> Gromacs Users mailing list
>>
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> send a mail to gmx-users-request at gromacs.org.
>>
> --
> Gromacs Users mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.