[gmx-users] PME on BlueGene

[Jakob Wohlert]

Mark Abraham wrote:
> Jakob Wohlert wrote:
>> Hi,
>>
>> I'm trying to compile Gromacs 4.0.4 for BlueGene/P, and using the
>> configuration options from the wiki I have succeeded insofar that I 
>> have a
>> working program as long as I don't use PME.
>>
>> I have tried many different variants of fftw - 2.1.5, 3.2.1, single
>> precision, double precision, different compiler optimizations and so on,
>> but it all ends the same: mdrun getting stuck somewhere in the
>> initialization process.
>>
>> However, by using the built in fft library FFTPACK instead of FFTW, PME
>> will work, but that is not really an alternative.
>>
>> In at least a few cases I have been able to pinpoint the location 
>> where it
>> hangs - it's in pme.c, subroutine pme_dd_sendrecv. The program calls
>> MPI_Sendrecv, but then nothing else happens as far as I can tell.
>>
>> I'm confused and I have sort of ran out of ideas right now. Has anyone
>> else encountered a problem like this, or has anyone any suggestions 
>> how to
>> proceed from here?
Thanks for your answer!

> That looks to me like the separate PME nodes are dying through some 
> linking problem and the problem is only manifest on node 0 when its 
> sendrecv doesn't complete. Forcing mdrun -npme 0 may confirm this when 
> all the nodes die at the first point they refer to a symbol in the FFT 
> library.
>
> Otherwise, looking at warnings/errors from the linker will be required. 
Ok, can you be a little more specific? Do you mean when compiling fftw, 
gromacs or both? I'm not very experienced with these kind of things.
> Are you compiling an FFT library version for the back end, or the 
> front end?
I'm trying to get it to work on the back end, on the front end it works 
fine! (So, I have fftw libraries for both).

/Jakob