[gmx-users] PME problem on BG/P cluster

[David van der Spoel]

On 2010-06-04 17.49, LuLanyuan wrote:
> Hello,
> I got a weird problem by running Gromacs 4.0.7 on a BG/P machine 
> ("Intrepid" at Argonne national lab).
> The simulated system is a box of SPC water with 648,000 atoms and all MD 
> simulations were performed on 256 CPU cores with MPI. The compiling 
> environment was Linux with IBM compiler and libs.
> I first compile the code with flags suggested on the Wiki, such as:
> ./configure --prefix=$PREFIX \
> --host=ppc \
> --build=ppc64 \
> --disable-software-sqrt \
> --enable-ppc-sqrt=1 \
> &n bsp; --enable-ppc-altivec \
> --enable-bluegene \
> --disable-fortran \
> --enable-mpi \
> --with-fft=fftpack \
> --without-x \
> CC="mpixlc_r" \
> CFLAGS="-O3 -qarc h=450d -qtune=450" \
> MPICC="mpixlc_r"
> CXX="mpixlcxx_r"
> CXXFLAGS="-O3 -qarch=450 -qtune=450"
> F77="mpixlf77_r"
> FFLAGS="-O3 -qarch=450 -qtune=450"
> LIBS="-lmass"
> 
> Here I used fftpack to ensure that the problem is not due to the fftw 
> lib. I got the water system running will with Cut-off for 
> electrostatics. However, the systems always crashed after a few ( ~100) 
> steps if I used PME. The same system with same PME option runs fine on 
> other non-blue gene clusters I tested.
> The error message I got was sth like
> t = 0.100 ps: Water molecule starting at atom 403468 can not be settled.
> Check for bad contacts and/or reduce the timestep.
> Wrote pdb files with previous and current coordinates.
> 
> and
> 
> 2 particles communicated to PME node 63 are more than a cell length out 
> of the domain decomposition cell of their charge group
> 
>  From .log file, the kinetic energy is increasing and turned to be 
> "nan". So the system is exploding.
> 
> I found if I turned off the blue gene optimizations during configure, 
> the water system could be run without problem. For example, I used
> --enable-software-sqrt \
> --disable-ppc-sqrt \
> --disable-bluegene \
> and everything else was the same.
> I suspect there was an issue regarding the blue gene specific code and PME.
> Could anyone give any comments?
>
I would strongly suggest to use FFTW3.
All other FFT libraries are not so well tested.

-- 
David van der Spoel, Ph.D., Professor of Biology
Dept. of Cell & Molec. Biol., Uppsala University.
Box 596, 75124 Uppsala, Sweden. Phone:	+46184714205.
spoel at xray.bmc.uu.se    http://folding.bmc.uu.se