[gmx-developers] RE: Gromacs on 48 core magny-cours AMDs

[Igor Leontyev]

The issue might be related to configuration of our brand new cluster which 
I am testing now. On this cluster the unstable behavior of gromacs is also 
observed on Intel Xeon nodes. For gromacs installation I repeated all the 
steps that I have previously done many times on 8-core dual-Xeon 
workstation and have no problems. See bellow the compilation script.

# =====================================================================
#
# path where to install
pth_install=/home/leontyev/programs/bin/gromacs/gromacs-4.5.4
# program name suffix
suff="_mpich1.4.3"
# path of FFTW library
# SINGLE PRECISION
pth_fft=/home/leontyev/programs/bin/fftw/fftw-3.2.2/single
# path of 'open_mpi' library
pth_lam=/home/leontyev/programs/bin/mpi/openmpi/openmpi-1.4.3
export LD_LIBRARY_PATH="$pth_lam/lib"

PATH="$pth_lam/bin:$PATH"

export CPPFLAGS="-I/$pth_fft/include -I/$pth_lam/include"
export LDFLAGS="-L/$pth_fft/lib -L/$pth_lam/lib"

make distclean
# SINGLE PRECISION
./configure --without-x  --prefix=/$pth_install --program-suffix=$suff  --
enable-mpi

make -j 12 mdrun >& install.log
make install-mdrun  >> install.log
# =====================================================================

Igor


> Alexey Shvetsov wrote:
>
> Hello!
>
> Well there may be problems
> 1. Old kernel that works incorrectly with large numa
> 2. No correct process  binding to core
> 3. Configuration of gcc/math libs
> 
> What is your mpi version and versions of fftw and blas libs if you use 
> external ones.
> Also please post your cflags.
> 
> Here we have good performance on such nodes running SLES with 2.6.32 
> kernel (with gentoo-prefix on top of it with openmpi and ofed stack)
> and with Gentoo (kenrel 3.0.4) with many system optimiztions made by me 
> =)
> 
> All results are stable. Gentoo works better here becuse it doesnt has 
> bug with irq in kernel + some optimizations.