[gmx-users] Affinity setting for 1/16 threads failed. Version 5.0.2
Siva Dasetty
sdasett at g.clemson.edu
Wed Nov 18 17:13:03 CET 2015
Gordon:
Command line:
mdrun_mpi --version
Gromacs version: VERSION 5.0.2
Precision: single
Memory model: 64 bit
MPI library: MPI
OpenMP support: enabled
GPU support: disabled
invsqrt routine: gmx_software_invsqrt(x)
SIMD instructions: AVX_256
FFT library: fftw-3.3.3-sse2
RDTSCP usage: enabled
C++11 compilation: disabled
TNG support: enabled
Tracing support: disabled
Built on: Sun Oct 19 14:11:10 PDT 2014
Built by: root at gcn-20-88.sdsc.edu [CMAKE]
Build OS/arch: Linux 2.6.32-431.29.2.el6.x86_64 x86_64
Build CPU vendor: GenuineIntel
Build CPU brand: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
Build CPU family: 6 Model: 45 Stepping: 6
Build CPU features: aes apic avx clfsh cmov cx8 cx16 htt lahf_lm mmx msr
nonstop_tsc pcid pclmuldq pdcm pdpe1gb popcnt pse rdtscp sse2 sse3 sse4.1
sse4.2 ssse3 tdt x2apic
C compiler: /opt/mvapich2/intel/ib/bin/mpicc Intel 13.0.0.20121010
C compiler flags: -mavx -std=gnu99 -w3 -wd111 -wd177 -wd181 -wd193
-wd271 -wd304 -wd383 -wd424 -wd444 -wd522 -wd593 -wd869 -wd981 -wd1418
-wd1419 -wd1572 -wd1599 -wd2259 -wd2415 -wd2547 -wd2557 -wd3280 -wd3346
-wd11074 -wd11076 -O3 -DNDEBUG -ip -funroll-all-loops -alias-const
-ansi-alias
C++ compiler: /opt/mvapich2/intel/ib/bin/mpicxx Intel 13.0.0.20121010
C++ compiler flags: -mavx -w3 -wd111 -wd177 -wd181 -wd193 -wd271 -wd304
-wd383 -wd424 -wd444 -wd522 -wd593 -wd869 -wd981 -wd1418 -wd1419 -wd1572
-wd1599 -wd2259 -wd2415 -wd2547 -wd2557 -wd3280 -wd3346 -wd11074 -wd11076
-wd1782 -wd2282 -O3 -DNDEBUG -ip -funroll-all-loops -alias-const
-ansi-alias
Boost version: 1.55.0 (internal)
Our Cluster:
Command line:
mdrun --version
Gromacs version: VERSION 5.0.2
Precision: single
Memory model: 64 bit
MPI library: MPI
OpenMP support: enabled
GPU support: disabled
invsqrt routine: gmx_software_invsqrt(x)
SIMD instructions: SSE4.1
FFT library: fftw-3.3.3-sse2
RDTSCP usage: enabled
C++11 compilation: enabled
TNG support: enabled
Tracing support: disabled
Built on: Sun Mar 8 19:06:35 EDT 2015
Built by: sdasett at user001 [CMAKE]
Build OS/arch: Linux 2.6.32-358.23.2.el6.x86_64 x86_64
Build CPU vendor: GenuineIntel
Build CPU brand: Intel(R) Xeon(R) CPU X7542 @ 2.67GHz
Build CPU family: 6 Model: 46 Stepping: 6
Build CPU features: apic clfsh cmov cx8 cx16 htt lahf_lm mmx msr
nonstop_tsc pdcm popcnt pse rdtscp sse2 sse3 sse4.1 sse4.2 ssse3 x2apic
C compiler: /software/openmpi/1.8.1_gcc/bin/mpicc GNU 4.8.1
C compiler flags: -msse4.1 -Wno-maybe-uninitialized -Wextra
-Wno-missing-field-initializers -Wno-sign-compare -Wpointer-arith -Wall
-Wno-unused -Wunused-value -Wunused-parameter -O3 -DNDEBUG
-fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast
-Wno-array-bounds
C++ compiler: /software/openmpi/1.8.1_gcc/bin/mpicxx GNU 4.8.1
C++ compiler flags: -msse4.1 -std=c++0x -Wextra
-Wno-missing-field-initializers -Wpointer-arith -Wall -Wno-unused-function
-O3 -DNDEBUG -fomit-frame-pointer -funroll-all-loops
-fexcess-precision=fast -Wno-array-bounds
Boost version: 1.55.0 (internal)
Thanks,
On Wed, Nov 18, 2015 at 11:05 AM, Szilárd Páll <pall.szilard at gmail.com>
wrote:
> I don't see much similarity - except the type of error - between your issue
> and the one reported on redmine 1184. In that case Intel Harpertown (I
> assume Grodon is much newer), with thread-MPI was used.
>
> --
> Szilárd
>
> On Wed, Nov 18, 2015 at 4:45 PM, Siva Dasetty <sdasett at g.clemson.edu>
> wrote:
>
> > Thank you Mark. Yes I have already taken this issue to gordon and while
> am
> > awaiting for their response I am just wondering if this issue has
> anything
> > to do with the bug#1184: http://redmine.gromacs.org/issues/1184.
> >
> >
> > Thank you again for your quick response.
> >
> > On Wed, Nov 18, 2015 at 10:31 AM, Mark Abraham <mark.j.abraham at gmail.com
> >
> > wrote:
> >
> > > Hi,
> > >
> > > These are good issues to take up with the support staff of gordon.
> mdrun
> > > tries to be a good citizen and by default stays out of the way if some
> > > other part of the software stack is already managing process affinity.
> As
> > > you can see, doing it right is crucial for good performance. But mdrun
> > -pin
> > > on always works everywhere we know about.
> > >
> > > Mark
> > >
> > > On Wed, Nov 18, 2015 at 3:28 PM Siva Dasetty <sdasett at g.clemson.edu>
> > > wrote:
> > >
> > > > Dear all,
> > > >
> > > > I am running simulations using version 5.0.2 (default in gordon) and
> I
> > am
> > > > having a performance loss from 180 ns/day to 8 ns/day compared to the
> > > same
> > > > simulations that I previously ran in a different cluster.
> > > >
> > > > In both the clusters I am using a single node and 16 cpus (no gpus)
> and
> > > the
> > > > following is the command line I used:
> > > >
> > > > mdrun_mpi -s <tpr file> -v -deffnm <output file> -nb cpu -cpi <cpt
> > file>
> > > > -append -pin on
> > > >
> > > >
> > > > Following is reported in the log file:
> > > >
> > > >
> > > > WARNING: Affinity setting for 1/16 threads failed.
> > > >
> > > > This can cause performance degradation! If you think your
> > > setting
> > > > are correct, contact the GROMACS developers.
> > > >
> > > >
> > > > I even tried running a simulation without the flag -pin on and there
> is
> > > no
> > > > change in the performance.
> > > >
> > > >
> > > > Are there any other options that I can try to recover the
> performance?
> > > >
> > > >
> > > >
> > > > Additional Information:
> > > >
> > > >
> > > > The other difference I could see is in the compilers:
> > > >
> > > >
> > > > In gordon (8ns/day):
> > > >
> > > >
> > > > C compiler: /opt/mvapich2/intel/ib/bin/mpicc Intel 13.0.0.20121010
> > > >
> > > > C compiler flags: -mavx -std=gnu99 -w3 -wd111 -wd177 -wd181
> > -wd193
> > > > -wd271 -wd304 -wd383 -wd424 -wd444 -wd522 -wd593 -wd869 -wd981
> -wd1418
> > > > -wd1419 -wd1572 -wd1599 -wd2259 -wd2415 -wd2547 -wd2557 -wd3280
> -wd3346
> > > > -wd11074 -wd11076 -O3 -DNDEBUG -ip -funroll-all-loops -alias-const
> > > > -ansi-alias
> > > >
> > > > C++ compiler: /opt/mvapich2/intel/ib/bin/mpicxx Intel
> > > 13.0.0.20121010
> > > >
> > > > C++ compiler flags: -mavx -w3 -wd111 -wd177 -wd181 -wd193 -wd271
> > > -wd304
> > > > -wd383 -wd424 -wd444 -wd522 -wd593 -wd869 -wd981 -wd1418 -wd1419
> > -wd1572
> > > > -wd1599 -wd2259 -wd2415 -wd2547 -wd2557 -wd3280 -wd3346 -wd11074
> > -wd11076
> > > > -wd1782 -wd2282 -O3 -DNDEBUG -ip -funroll-all-loops -alias-const
> > > > -ansi-alias
> > > >
> > > >
> > > > In our cluster (180 ns/day):
> > > >
> > > >
> > > > C compiler: /software/openmpi/bin/mpicc GNU 4.8.1
> > > >
> > > > C compiler flags: -msse4.1 -Wno-maybe-uninitialized -Wextra
> > > -Wno-miss
> > > >
> > > > ing-field-initializers -Wno-sign-compare -Wpointer-arith -Wall
> > > -Wno-unused
> > > >
> > > > -Wunused-value -Wunused-parameter -O3 -DNDEBUG -fomit-frame-pointer
> > > -funro
> > > >
> > > > ll-all-loops -fexcess-precision=fast -Wno-array-bounds
> > > >
> > > > C++ compiler: /software/openmpi/bin/mpicxx GNU 4.8.1
> > > >
> > > > C++ compiler flags: -msse4.1 -Wextra
> > -Wno-missing-field-initializers
> > > -W
> > > >
> > > > pointer-arith -Wall -Wno-unused-function -O3 -DNDEBUG
> > > -fomit-frame-pointer
> > > >
> > > > -funroll-all-loops -fexcess-precision=fast -Wno-array-bounds
> > > >
> > > >
> > > >
> > > >
> > > > Thanks in advance for your help,
> > > >
> > > > --
> > > > Siva Dasetty
> > > > --
> > > > Gromacs Users mailing list
> > > >
> > > > * Please search the archive at
> > > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > > > posting!
> > > >
> > > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> > > >
> > > > * For (un)subscribe requests visit
> > > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users
> or
> > > > send a mail to gmx-users-request at gromacs.org.
> > > >
> > > --
> > > Gromacs Users mailing list
> > >
> > > * Please search the archive at
> > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > > posting!
> > >
> > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> > >
> > > * For (un)subscribe requests visit
> > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> > > send a mail to gmx-users-request at gromacs.org.
> > >
> >
> >
> >
> > --
> > Siva Dasetty
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> > send a mail to gmx-users-request at gromacs.org.
> >
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>
--
Siva Dasetty
More information about the gromacs.org_gmx-users
mailing list