[gmx-users] Strange assignment of atoms to processors with pd
Erik Marklund
erikm at xray.bmc.uu.se
Wed May 27 11:35:47 CEST 2009
This was done with verision 4.0.4, taken straight from the website. I've
already simulated a great deal with this version and I'd hate to swap
versions in the middle of the project.
As for the -debug runs, I did it for the dry analyte and for one with a
6 Å water layer. It seems that the protein is indeed divided over the
processors if no other molecules are present. When there are other
molecules, such as water, in the system then I suspect that the
molecules are divided over the processors. This doesn't explain why the
first processor got no atoms when I ran the simulations on 8 cpus though.
==== without water ===============-,
There are: 1960 Atoms
splitting topology...
There are 698 charge group borders and 1 shake borders
There are 698 total borders
Division over nodes in atoms:
492 489 488 491
Walking down the molecule graph to make constraint-blocks
CPU= 0, lastcg= 171, targetcg= 520, myshift= 2
CPU= 1, lastcg= 354, targetcg= 6, myshift= 3
CPU= 2, lastcg= 521, targetcg= 172, myshift= 2
CPU= 3, lastcg= 697, targetcg= 348, myshift= 2
pd->shift = 3, pd->bshift= 0
Division of bonded forces over processors
CPU 0 1 2 3
BONDS 259 263 250 253
ANGLES 894 867 901 870
PDIHS 110 117 87 107
RBDIHS 1002 967 1015 960
LJ14 1289 1260 1318 1234
CONSTR 242 232 244 241
Workload division
nnodes: 4
pd->shift: 3
pd->bshift: 0
Nodeid atom0 #atom cg0 #cg
0 0 492 0 172
1 492 489 172 183
2 981 488 355 167
3 1469 491 522 176
===============================-'
== with a bit of water ===============-,
There are: 5086 Atoms
There are: 1042 VSites
splitting topology...
There are 1724 charge group borders and 1043 shake borders
There are 1043 total borders
Division over nodes in atoms:
1960 1388 1392 1388
Walking down the molecule graph to make constraint-blocks
CPU= 0, lastcg= 681, targetcg= 1543, myshift= 3
CPU= 1, lastcg= 1028, targetcg= 166, myshift= 3
CPU= 2, lastcg= 1376, targetcg= 514, myshift= 2
CPU= 3, lastcg= 1723, targetcg= 862, myshift= 2
pd->shift = 3, pd->bshift= 0
Division of bonded forces over processors
CPU 0 1 2 3
BONDS 1025 0 0 0
ANGLES 3547 0 0 0
PDIHS 426 0 0 0
RBDIHS 3950 0 0 0
LJ14 5106 0 0 0
CONSTR 959 0 0 0
Workload division
nnodes: 4
pd->shift: 3
pd->bshift: 0
Nodeid atom0 #atom cg0 #cg
0 0 1960 0 682
1 1960 1388 682 347
2 3348 1392 1029 348
3 4740 1388 1377 347
Berk Hess skrev:
> Hi,
>
> This is plain 4.0 code is presume?
> This problem should be fixed then.
>
> But I now also made vacuum without cut-off working with domain
> decomposition in CVS head.
> Compared to a not-unbalanced PD (for instance only a protein, no
> water) DD is slightly slower.
> But DD will be faster than a badly balanced PD system.
>
> Berk
>
> > Date: Wed, 27 May 2009 11:04:49 +0200
> > From: spoel at xray.bmc.uu.se
> > To: gmx-users at gromacs.org
> > Subject: Re: [gmx-users] Strange assignment of atoms to processors
> with pd
> >
> > Erik Marklund wrote:
> > > David van der Spoel skrev:
> > >> Erik Marklund wrote:
> > >>> I should add that this problem only seem to arise when the
> analyte is
> > >>> covered with a thin sheet of water. When simulating a dry analyte I
> > >>> get good scaling. In the latter case the charges, and therefore the
> > >>> topology, is slightly different.
> > >> How about vsites? Did you happen to turn them off as well in the
> > >> vacuum case?
> > > Turned off in all cases. The VSites mentioned in the log file is the
> > > 4:th particle on the tip4p-water molecules.
> > OK. Did you try a one step run with -debug?
> > It may give more info on the partitioning.
> >
> > >>>
> > >>> /Erik
> > >>>
> > >>> Erik Marklund skrev:
> > >>>> Hi,
> > >>>>
> > >>>> I'm simulating non-periodic systems in vacuo, using constrained
> > >>>> h-bonds and particle decomposition. For some of my simulations the
> > >>>> cpu-usage seem far from optimal. The first cpu gets no atoms,
> while
> > >>>> the second one gets plenty and the remaining cpus get less than I
> > >>>> expected. Is this a bug?
> > >>>>
> > >>>>
> > >>>> An excerpt from the log file:
> > >>>>
> > >>>> There are: 2911 Atoms
> > >>>> There are: 317 VSites
> > >>>> splitting topology...
> > >>>> There are 999 charge group borders and 318 shake borders
> > >>>> There are 318 total borders
> > >>>> Division over nodes in atoms:
> > >>>> 0 1960 212 212 212 212 212 208
> > >>>> Walking down the molecule graph to make constraint-blocks
> > >>>> CPU= 0, lastcg= -1, targetcg= 499, myshift= 1
> > >>>> CPU= 1, lastcg= 681, targetcg= 182, myshift= 0
> > >>>> CPU= 2, lastcg= 734, targetcg= 235, myshift= 7
> > >>>> CPU= 3, lastcg= 787, targetcg= 288, myshift= 6
> > >>>> CPU= 4, lastcg= 840, targetcg= 341, myshift= 5
> > >>>> CPU= 5, lastcg= 893, targetcg= 394, myshift= 4
> > >>>> CPU= 6, lastcg= 946, targetcg= 447, myshift= 3
> > >>>> CPU= 7, lastcg= 998, targetcg= 499, myshift= 2
> > >>>> pd->shift = 7, pd->bshift= 0
> > >>>> Division of bonded forces over processors
> > >>>> CPU 0 1 2 3 4 5 6 7
> > >>>> Workload division
> > >>>> nnodes: 8
> > >>>> pd->shift: 7
> > >>>> pd->bshift: 0
> > >>>> Nodeid atom0 #atom cg0 #cg
> > >>>> 0 0 0 0 0
> > >>>> 1 0 1960 0 682
> > >>>> 2 1960 212 682 53
> > >>>> 3 2172 212 735 53
> > >>>> 4 2384 212 788 53
> > >>>> 5 2596 212 841 53
> > >>>> 6 2808 212 894 53
> > >>>> 7 3020 208 947 52
> > >>>>
> > >>>> …
> > >>>> Total Scaling: 18% of max performance
> > >>>>
> > >>>
> > >>>
> > >>
> > >>
> > >
> > >
> >
> >
> > --
> > David van der Spoel, Ph.D., Professor of Biology
> > Molec. Biophys. group, Dept. of Cell & Molec. Biol., Uppsala University.
> > Box 596, 75124 Uppsala, Sweden. Phone: +46184714205. Fax: +4618511755.
> > spoel at xray.bmc.uu.se spoel at gromacs.org http://folding.bmc.uu.se
> > _______________________________________________
> > gmx-users mailing list gmx-users at gromacs.org
> > http://www.gromacs.org/mailman/listinfo/gmx-users
> > Please search the archive at http://www.gromacs.org/search before
> posting!
> > Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-request at gromacs.org.
> > Can't post? Read http://www.gromacs.org/mailing_lists/users.php
>
> ------------------------------------------------------------------------
> What can you do with the new Windows Live? Find out
> <http://www.microsoft.com/windows/windowslive/default.aspx>
> ------------------------------------------------------------------------
>
> _______________________________________________
> gmx-users mailing list gmx-users at gromacs.org
> http://www.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at http://www.gromacs.org/search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> Can't post? Read http://www.gromacs.org/mailing_lists/users.php
--
-----------------------------------------------
Erik Marklund, PhD student
Laboratory of Molecular Biophysics,
Dept. of Cell and Molecular Biology, Uppsala University.
Husargatan 3, Box 596, 75124 Uppsala, Sweden
phone: +46 18 471 4537 fax: +46 18 511 755
erikm at xray.bmc.uu.se http://xray.bmc.uu.se/molbiophys
More information about the gromacs.org_gmx-users
mailing list