[gmx-users] configuration/installation of gromacs-4.6.1 on heterogeneous cluster

[Mark Abraham]

On Sat, May 4, 2013 at 1:41 AM, Martin Siegert <siegert at sfu.ca> wrote:

> Hi,
>
> I am struggling with the configuration and compilation/installation
> of gromacs-4.6.1. Our cluster has 2 different processors: the older
> generation supports sse4.1, the newer sse4.2. Configuration and
> compilation must be done on the headnode of the cluster, which
> supports sse4.2. I am using the following command to configure
> gromacs-4.6.1:
>
> CFLAGS='-fpic -O3 -axSSE4.2,SSE4.1 -xSSSE3 -ip -opt-prefetch' \
> CXXFLAGS='-fpic -O3 -axSSE4.2,SSE4.1 -xSSSE3 -ip -opt-prefetch' \
> FFLAGS='-fpic -O3 -axSSE4.2,SSE4.1 -xSSSE3 -ip -opt-prefetch' \
>

If the above is successful at generating instructions for SSE>4.1...

CC=mpicc \
> CXX=mpicxx \
> FC=mpif90 \
> LDFLAGS="-lfftw3f -lgoto2 -Wl,-rpath,/usr/local/gromacs-4.6.1/lib" \
> cmake -DCMAKE_VERBOSE_MAKEFILE=ON -DGMX_MPI=ON \
>       -DGMX_CPU_ACCELERATION=SSE4.1 -DGMX_OPENMP=OFF -DGMX_GPU=OFF \
>       -DCMAKE_INSTALL_PREFIX=/usr/local/gromacs-4.6.1 \
>       -DCMAKE_SKIP_RPATH=YES \
>       ../gromacs-4.6.1
>
> However, after compilation/installation mdrun_mpi fails on nodes that
> only support sse4.1 with "Illegal instruction".
>

... then this is not a surprise. Your compiler has been allowed to generate
SSE 4.2 instructions, I suspect.


> The CMakeCache.txt file contains the line:
>
> BUILD_CPU_FEATURES:INTERNAL=aes apic clfsh cmov cx8 cx16 htt lahf_lm mmx
> msr nonstop_tsc pcid pclmuldq pdcm pdpe1gb popcnt pse rdtscp sse2 sse3
> sse4.1 sse4.2 ssse3
>
> Since this line contains "sse4.2" it appears that the flag
> -DGMX_CPU_ACCELERATION=SSE4.1
> is ignored.
>

That list is what features are available on the CPU (mostly for helping
detect what acceleration to use, and to solve problems). The target
acceleration scheme (i.e. code with heavy use of compiler instrinsics) is
SSE4.1, which is what GROMACS will compile for if you leave it alone :-)
SSE4.2 is roughly useless for mdrun.

What is the correct way of specifying the cpu architecture within the
> cmake build system? (I never had problems with this with the pre 4.6
> versions).
>

Back then, GROMACS was nearly insensitive to compilation settings, because
of the use of assembly kernels. Now GROMACS is sensitive to compiler
version (in that compilation of SIMD instrinsics needs to work well, and
OpenMP needs to be supported, etc.) but one still doesn't generally want to
mess with the compiler flags. We have some internal disgreement about
whether we should be permitting/encouraging/facilitiating setting/checking
compiler flags. So far nobody has demonstrated a use case that suggests we
need to support more than "shut up and let GROMACS do its thing!"

Mark


>
> Cheers,
> Martin
>
> --
> Martin Siegert
> WestGrid/ComputeCanada
> Simon Fraser University
> Burnaby, British Columbia
> Canada
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>