[gmx-users] why it is so slow in Blue gene?

[Mark Abraham]

On 25/04/2012 3:24 PM, Albert wrote:
> hello:
>
>   it is blue gene P. And the gromacs is single precision in the 
> cluster. Getting Loaded...And the administrator told me that I have to 
> use the multiples of 32 in the bg_size parameter. The number specified 
> in "-np" should be 4 times bg_size.

Yes, but your system is too small to make use of 128 processors. Also, 
get rid of -launch and -nt from your command line, since they do nothing.

>   It is even slower than my own workstation with 16 core.........
>
>
>
>
> here is the log file I get:

No, that's the stdout file. Look at the end of the .log file.

>
> -------------log----------------
> Reading file npt_01.tpr, VERSION 4.5.5 (single precision)
> Loaded with Money
>
> Will use 112 particle-particle and 16 PME only nodes

This is guaranteed to lead to woeful performance with your .mdp 
settings, but you will have to look towards the beginning of the .log 
file to find out why mdrun selected this. Odds are good that your system 
size is so small that the minimum particle-particle cell size 
(constrained by rcoulomb) doesn't give mdrun any good options that use 
all the processors. You'd likely get better raw performance with twice 
the number of atoms or half the number of processors.

Mark

> This is a guess, check the performance at the end of the log file
> Making 3D domain decomposition 4 x 4 x 7
> starting mdrun 'GRowing Old MAkes el Chrono Sweat'
> 500000 steps,    500.0 ps.
> step 0
> vol 0.64! imb F 16% pme/F 0.22 step 100, will finish Wed Apr 25 
> 18:28:06 2012
> vol 0.65! imb F 17% pme/F 0.21 step 200, will finish Wed Apr 25 
> 18:09:54 2012
> vol 0.67! imb F 18% pme/F 0.21 step 300, will finish Wed Apr 25 
> 18:03:12 2012
> vol 0.69! imb F 18% pme/F 0.21 step 400, will finish Wed Apr 25 
> 17:58:25 2012
> vol 0.67! imb F 19% pme/F 0.21 step 500, will finish Wed Apr 25 
> 17:55:26 2012
> vol 0.68! imb F 19% pme/F 0.22 step 600, will finish Wed Apr 25 
> 17:53:31 2012
> vol 0.68! imb F 19% pme/F 0.22 step 700, will finish Wed Apr 25 
> 17:51:57 2012
> vol 0.68! imb F 19% pme/F 0.22 step 800, will finish Wed Apr 25 
> 17:50:32 2012
> vol 0.68! imb F 20% pme/F 0.22 step 900, will finish Wed Apr 25 
> 17:49:14 2012
> vol 0.67! imb F 21% pme/F 0.22 step 1000, will finish Wed Apr 25 
> 17:48:13 2012
> vol 0.68! imb F 20% pme/F 0.22 step 1100, will finish Wed Apr 25 
> 17:47:28 2012
> vol 0.67! imb F 21% pme/F 0.22 step 1200, will finish Wed Apr 25 
> 17:46:50 2012
> vol 0.67! imb F 21% pme/F 0.22 step 1300, will finish Wed Apr 25 
> 17:46:15 2012
>
>
>
> On 04/24/2012 06:01 PM, Hannes Loeffler wrote:
>> On Tue, 24 Apr 2012 15:42:15 +0200
>> Albert<mailmd2011 at gmail.com>  wrote:
>>
>>> hello:
>>>
>>>     I am running a 60,000 atom system with 128 core in a blue gene
>>> cluster. and it is only 1ns/day.... here is the script I used for
>> You don't give any information what exact system that is (L/P/Q?), if
>> you run single or double precision and what force field you are using.
>> But for a similar sized system using a united atom force field in
>> single precision we find about 4 ns/day on a BlueGene/P (see our
>> benchmarking reports on
>> http://www.stfc.ac.uk/CSE/randd/cbg/Benchmark/25241.aspx).  I would
>> expect a run with the CHARMM 27 force field in double precision to be
>> roughly 3 times slower.  We found scaling to 128 cores to be
>> reasonably good. Also, check our report for problems when compiling
>> with higher optimisation.
>>
>> Hannes.
>