[gmx-users] Problems with mdrun (version 4.6.5) and CPU-type: Xeon E5472

[Thomas Schlesier]

Hi all,
I am having problems with GROMACS 4.6.5 and one type of CPU on our 
cluster (other CPU-types make no problems). GROMACS was always compiled 
on the master, which uses a different CPU-type (see below at the 
beginning of the 'md.log'-file).

For the CPU type (partial output from 'cat /proc/cpuinfo'):
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Xeon(R) CPU           E5472  @ 3.00GHz
stepping	: 6
cpu MHz		: 3000.186
cache size	: 6144 KB

'grompp' 'g_rdf' and 'pdb2gmx' all work fine, only 'mdrun' gives 
problems (so I think only 'mdrun' is affected).

Error message is:
Illegal instruction
and i get a 'core dump'. (Didn't looked into the dumped file, since i 
only know i can do this with 'gdb' but have absolute no idea how 'gdb' 
works and for what i should look).

Last lines of the 'md.log' file before the crash (energy minimization of 
a small water box with cut-offs):

Removing pbc first time

++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
S. Miyamoto and P. A. Kollman
SETTLE: An Analytical Version of the SHAKE and RATTLE Algorithms for Rigid
Water Models
J. Comp. Chem. 13 (1992) pp. 952-962
-------- -------- --- Thank You --- -------- --------

So, i get the impression, that the preparation for the simulation is 
success full, and the crash happens when the actual simulation starts.

The same simulation works with gromacs 4.0.7 and 4.5.5 (on the same 
machine). So I thought SSE4.1 or the internal MPI-stuff from gromacs 
4.6.5 produces the problem.
So i compiled gromacs 4.6.5 with
-DGMX_CPU_ACCELERATION=SSE2 (or) None
and without any MPI support (and then running only a serial calculation).
Also tried for FFT the internal 'fftpack' (thought maybe the FFT part 
might be the problem - for this i also ran the simulation without PME 
and only plain cut-offs).

But the error still occurs in every case :(

I tried the compilation with two different compilers:
Intel:
icc (ICC) 12.1.2 20111128
Copyright (C) 1985-2011 Intel Corporation.  All rights reserved.
and GNU:
gcc (GCC) 4.6.4 20120511 (prerelease)
Copyright (C) 2011 Free Software Foundation, Inc.
but i makes no difference.


To give some more information:

Beginning of the 'md.log'-file:

Log file opened on Fri Jan 24 16:40:34 2014
Host: tc-3-12.local  pid: 6755  nodeid: 0  nnodes:  1
Gromacs version:    VERSION 4.6.5
Precision:          single
Memory model:       64 bit
MPI library:        none
OpenMP support:     disabled
GPU support:        disabled
invsqrt routine:    gmx_software_invsqrt(x)
CPU acceleration:   SSE4.1
FFT library:        fftpack (built-in)
Large file support: enabled
RDTSCP usage:       enabled
Built on:           Fr 24. Jan 16:26:42 CET 2014
Built by:           schlesi at dirac.chemie.uni-mainz.de [CMAKE]
Build OS/arch:      Linux 2.6.18-238.19.1.el5 x86_64
Build CPU vendor:   GenuineIntel
Build CPU brand:    Intel(R) Xeon(R) CPU           X5670  @ 2.93GHz
Build CPU family:   6   Model: 44   Stepping: 1
Build CPU features: apic clfsh cmov cx8 cx16 htt lahf_lm mmx msr 
nonstop_tsc pcid pdcm pdpe1gb popcnt pse rdtscp sse2 sse3 sse4.1 sse4.2 
ssse3
C compiler:         /share/apps/gcc/gcc-4.6.4_20120511/bin/gcc GNU gcc 
(GCC) 4.6.4 20120511 (prerelease)
C compiler flags:   -msse4.1    -Wextra -Wno-missing-field-initializers 
-Wno-sign-compare -Wall -Wno-unused -Wunused-value -Wno-unknown-pragmas 
   -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -O3 
-DNDEBUG

cmake-command for compilation (used for the last build):
(here i'm using line breaks to make it easier to read):

cmake
-DCMAKE_C_COMPILER=/share/apps/gcc/gcc-4.6.4_20120511/bin/gcc 
-DCMAKE_CXX_COMPILER=/share/apps/gcc/gcc-4.6.4_20120511/bin/g++ 
-DGMX_BUILD_OWN_FFTW=OFF
-DCMAKE_INSTALL_PREFIX=/scrcluster/schlesi/bug465 -DBUILD_SHARED_LIBS=OFF
-DGMX_PREFER_STATIC_LIBS=ON
-DGMX_CPU_ACCELERATION=SSE4.1
-DGMX_MPI_IN_PLACE=OFF
-DGMX_OPENMP=OFF
-DGMX_THREAD_MPI=OFF
-DGMX_MPI=OFF
-DGMX_FFT_LIBRARY=fftpack
..
(Sidenote: the 'GMX_MPI_IN_PLACE' isn't used - cmake tells me this).

Till now i havn't tried to compile GROMACS on any of the problematic 
machines, but i can try this on monday.


------------------------------------------------------------------------

Has anyone any idea (for solving, or to better trace the error)? Since 
i'm at the end with my knowledge (actually i'm quite happy that i came 
so far, since about this 'hardcore' computer stuff my knowledge is very 
limited).
If someone needs more informations (or the core dumped file), just tell me.

Greetings (and wish you a nice weekend)
Thomas