[gmx-users] Re: MPI scaling (was RE: MPI tips)

[Mark Abraham]

David Mobley wrote:
> 
> 
>     Interesting theory but not one that can be easily tested since the
>     programs don't emit "set up took N seconds" and "processing took M
>     seconds".   There's another chunk of the program which should
>     probably also count as "set up" which is the time needed at
>     each step for the nodes to reconcile their chunks of the model
>     before the next calculation.  (My guess is that this last part doesn't
>     scale well at all.)  Perhaps you're counting this second thing in
>     your setup?  In any case, I know that the _initial_ setup before the
>     steps run doesn't take all that long, since tail -f on the log file
>     shows the steps after not too many seconds, so the initial setup
>     is not a key factor.  The per step (re)setup can't be measured but
>     could be more significant.
> 
> 
> The theory is that the setup stays constant as you increase the number 
> of steps, while the run time increases. So you can test the theory by, 
> as we've been suggesting all along, increasing the number of steps 
> you're running to see if the scaling you're seeing drastically improves. 
> It should.

Hear, hear.

Imagine setting up a multi-way video teleconference for the length of 
time it takes to say "Hello" before quitting. You'd conclude this wasn't 
a very efficient way to communicate, wouldn't you? The analogy isn't 
perfect, but the longer the conference goes, the less important is the 
time taken to configure and handshake at the beginning.

Mark