[gmx-users] compiling with the PGI compiler

[chris.neale at utoronto.ca]

Dear Szilárd:

Thank you for the advice. I made a mistake when I said that I was  
using new Xeons. The new cluster is actually composed of AMD Opteron  
Magny-cours 6172 (12-cores at 2.1 or 2.2 GHz grouped as 24-cores per  
node). It was because of the AMD architecture that I was trying with  
the pathscale and PGI compilers in addition to Intel. For the one  
system that I have checked, the intel compilation is 20% faster than  
the pathscale compilation, although I didn't provide any special  
compilation options to either. I will continue to look into using  
pathscale optimizations for gromacs on AMDs.

If you are still interested in benchmark speeds on the opterons, then  
please point me to system and mdp files that people are using nowadays  
for comparisons and I will reply back with the benchmark information.

Thank you,
Chris.

-- original message --

Hi,

I've personally never heard of anybody using gromacs compiled with PGI.

> I am using a new cluster of Xeons and, to get the most efficient
> compilation, I have compiled gromacs-4.5.4 separately with the intel,
> pathscale, and pgi compilers.

I did try Pathscale a few months ago and AFAIR it wasn't very
difficult to get it working - although I do remember getting some
segfaults from time to time. The Intel Compiler support in CMake is
kind of broken in 4.5, you need to fiddle with the flags a bit, but it
should be quite straightforward.

> With the pgi compiler, I am most concerned about this floating point
> overflow warning:
>
> ...
> [ 19%] Building C object src/gmxlib/CMakeFiles/gmx.dir/trajana/trajana.c.o
> [ 19%] Building C object
> src/gmxlib/CMakeFiles/gmx.dir/trajana/centerofmass.c.o
> [ 19%] Building C object src/gmxlib/CMakeFiles/gmx.dir/trajana/nbsearch.c.o
> PGC-W-0129-Floating point overflow. Check constants and constant expressions
> (/home/nealechr/exe/pgi/gromacs-4.5.4/source/src/gmxlib/trajana/nbsearch.c:
> 166)

That looks like a bug, in case of real==float HUGE_VALF should be used
instead of HUGE_VAL.

> PGC/x86-64 Linux 11.8-0: compilation completed with warnings
> [ 19%] Building C object
> src/gmxlib/CMakeFiles/gmx.dir/trajana/displacement.c.o
> [ 19%] Building C object src/gmxlib/CMakeFiles/gmx.dir/trajana/position.c.o
> ...
>
>
> There are also a lot of warnings about a type cast, which seems less like a
> real problem:

Looks like all of them are enum to string-related stuff, they are harmless.

> I compiled like this:
>
> module purge
> module load pgi64/11.8 openmpi_pgi64/1.4.3_ofed
> module load cmake/2.8.5
> export CCDIR=/opt/pgi/linux86-64/11.8/bin
>
> ## set the location of the single precision FFTW isntallation
> export FFTW_LOCATION=/home/nealechr/exe/pgi/fftw-3.1.2/exec
>
> ##### Nothing below this line usually needs to be changed
>
> export CXX=pgCC
> export CC=pgcc
>
> cmake ../source/ \
>      -DFFTW3F_INCLUDE_DIR=$FFTW_LOCATION/include \
>      -DFFTW3F_LIBRARIES=$FFTW_LOCATION/lib/libfftw3f.a \
>      -DCMAKE_INSTALL_PREFIX=$(pwd) \
>      -DGMX_X11=OFF \
>      -DCMAKE_CXX_COMPILER=${CCDIR}/pgCC \
>      -DCMAKE_C_COMPILER=${CCDIR}/pgcc \
>      -DGMX_PREFER_STATIC_LIBS=ON \
>      -DGMX_MPI=OFF
>
>
> make
>
> make install

That looks fine, if you're benchmarking you might want to try the arch
specific flags - I assume PGI does have something, even if not
specific for Xeon E5.

> ##########
>
> I don't get any similar errors with the intel or pathscale compilers
> (although intel gives me "icc: command line warning #10159: invalid argument
> for option '-m'" a lot) and the pathscale compilation appears to be hung on

Yeah, as I sad that's kind of broken.

> If anybody has run into this warning, or knows enough to be sure that I
> don't need to worry about it during mdrun (it seems to be in an analysis
> file, but I'm not entirely sure), then I would be happy to hear about it. A
> gmx-users search for PGI returned zero results. I saw something here, but it
> was not very specific about the problem with pgi (
> http://www.levlafayette.com/node/175 ).

The overflow error is in the trajectory analysis tool's source files
and doesn't concern mdrun (btw, you can do make mdrun and even make
install-mdrun ;).

Would it be possible for you to share some performance numbers you're
getting on the E5?

Cheers,
--
Szilárd

> Thank you,