[gmx-users] PME on BlueGene

[Mark Abraham]

Jakob Wohlert wrote:
> Hi,
> 
> I'm trying to compile Gromacs 4.0.4 for BlueGene/P, and using the
> configuration options from the wiki I have succeeded insofar that I have a
> working program as long as I don't use PME.
> 
> I have tried many different variants of fftw - 2.1.5, 3.2.1, single
> precision, double precision, different compiler optimizations and so on,
> but it all ends the same: mdrun getting stuck somewhere in the
> initialization process.
> 
> However, by using the built in fft library FFTPACK instead of FFTW, PME
> will work, but that is not really an alternative.
> 
> In at least a few cases I have been able to pinpoint the location where it
> hangs - it's in pme.c, subroutine pme_dd_sendrecv. The program calls
> MPI_Sendrecv, but then nothing else happens as far as I can tell.
> 
> I'm confused and I have sort of ran out of ideas right now. Has anyone
> else encountered a problem like this, or has anyone any suggestions how to
> proceed from here?

That looks to me like the separate PME nodes are dying through some 
linking problem and the problem is only manifest on node 0 when its 
sendrecv doesn't complete. Forcing mdrun -npme 0 may confirm this when 
all the nodes die at the first point they refer to a symbol in the FFT 
library.

Otherwise, looking at warnings/errors from the linker will be required. 
Are you compiling an FFT library version for the back end, or the front end?

Mark