[gmx-users] Limitation on the maximum number of OpenMPI threads

[Abhi Acharya]

Hello,
I was trying to run a simulation on Gromacs-4.6.3 which has been compiled
without thread MPI on a BlueGene/Q system. The configurations per node are
as follows:

 PowerPC A2, 64-bit, 1.6 GHz, 16 cores SMP, 4 threads per core

For running on 8 nodes I tried:

srun mdrun_mpi -ntomp 64

But, this gave me an error:

Program mdrun_mpi, VERSION 4.6.3
Source code file:
/home/staff/sheed/apps/gromacs-4.6.3/src/mdlib/nbnxn_search.c, line: 2520

Fatal error:
64 OpenMP threads were requested. Since the non-bonded force buffer
reduction is prohibitively slow with more than 32 threads, we do not allow
this. Use 32 or less OpenMP threads.

So, I tried using 32 and it works fine. The problem is the performance
seems to be too low; for 1 ns run it shows an estimated time of more than a
day. The same run on a

workstation with 6 cores and 2 GPU gives a performance of 17 ns/day.

I am now at loss. Any ideas what is happening ??

Regards,
Abhishek Acharya