[gmx-users] Re: MPI scaling (was RE: MPI tips)

[David Mobley]

>
> Interesting theory but not one that can be easily tested since the
> programs don't emit "set up took N seconds" and "processing took M
> seconds".   There's another chunk of the program which should
> probably also count as "set up" which is the time needed at
> each step for the nodes to reconcile their chunks of the model
> before the next calculation.  (My guess is that this last part doesn't
> scale well at all.)  Perhaps you're counting this second thing in
> your setup?  In any case, I know that the _initial_ setup before the
> steps run doesn't take all that long, since tail -f on the log file
> shows the steps after not too many seconds, so the initial setup
> is not a key factor.  The per step (re)setup can't be measured but
> could be more significant.


The theory is that the setup stays constant as you increase the number of
steps, while the run time increases. So you can test the theory by, as we've
been suggesting all along, increasing the number of steps you're running to
see if the scaling you're seeing drastically improves. It should.


Again, I don't think it's actually any of these since the CPU time on
> the compute nodes falls down towards zero.  If they were setting up they
> would still be consuming CPU time. It looks a lot more like most of
> the nodes stall while waiting for data before they can proceed
> to the next step.


Much of the setup should be done by the head node before dividing up the
tasks, I think.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-users/attachments/20060202/3a9af00a/attachment.html>