[gmx-users] domain decomposition

[jampani srinivas]

Hi

Try with -dd 2 x 2 x 2 option with 8 nodes and check the performance. I too
got the same problems, i could overcome with symmetrical -dd option.

Srinivas.

On Fri, May 7, 2010 at 8:57 AM, irene farabella <
i.farabella at mail.cryst.bbk.ac.uk> wrote:

> Hello!
>
> I am new to Gromacs and especially at parallel runs. I have some problem
> running my system using domain decomposition. I apologize but it will
> be a long mail...
>
> My system is made up by a membrane protein embedded is a mixed lipid
> bilayer (POPE/POPG).
> I tried to run it on 8 nodes but the simulation crashed due to  "A charge
> group moved too far between two domain decomposition steps" and
> high % of load imbalance between nodes. I then tested the same run on a
> single node and it worked. I tried then different amount of nodes,
> changing DD (using mdrun -dd option) and it seems that more node I am
> using the less the performance in terms of ns/day although the load
> imbalance % is highly variable. During this test I found that the
> optimal nodes for my system is running it on 6 node with a DD 3:2:1 (vol
> 0.80  imb F  1% )
>
> ---- from the log file:
>
> Initializing Domain Decomposition on 6 nodes
> Dynamic load balancing: auto
> Will sort the charge groups at every domain (re)decomposition
> Initial maximum inter charge-group distances:
>  two-body bonded interactions: 0.703 nm, LJ-14, atoms 21579 21587
>  multi-body bonded interactions: 2.038 nm, Proper Dih., atoms 20434 20405
> Minimum cell size due to bonded interactions: 2.241 nm
> Maximum distance for 5 constraints, at 120 deg. angles, all-trans: 0.820 nm
> Estimated maximum distance required for P-LINCS: 0.820 nm
> Domain decomposition grid 3 x 2 x 1, separate PME nodes 0
> Domain decomposition nodeid 0, coordinates 0 0 0
>
> Table routines are used for coulomb: TRUE
> Table routines are used for vdw:     FALSE
> Will do PME sum in reciprocal space.
> ...
>
> Making 2D domain decomposition grid 3 x 2 x 1, home cell index 0 0 0
>
> Center of mass motion removal mode is Linear
> We have the following groups for center of mass motion removal:
>  0:  Protein_POPE_POPG
>  1:  SOL_NA+_CL-
> There are: 82711 Atoms
> Charge group distribution at step 2000000: 5512 5533 5668 5694 5535 5626
> Grid: 7 x 10 x 14 cells
> Initial temperature: 309.243 K
>  --------
>
> Using this setting I finally managed to equilibrate my system by going
> trough a
> series of restrained runs.
>
> Surprisingly after 6,5 ns of non-restrained run (Step  3289500) the
> simulation
> crashes with :
> "Fatal error:
> A charge group moved too far between two domain decomposition steps
> This usually means that your system is not well equilibrated ".
>
> It seems strange that it crashes only at 3289500 steps of a non-restrained
> run.
> I am now running a short run starting from a short while before the
> crash step using a single processor and, as suspected, it is going
> smoothly. My guess is that something is going wrong with the domain
> decomposition of a such non-homogeneous system, considering that there
> are also charged lipids that complicate it but I have no idea how to
> solve/improve it.
>
> I am using gromacs-4.0.5.
>
> Any suggestion are welcome.
>
> --
> Irene Farabella
> Wellcome Trust PhD student
>
> Institute of Structural and Molecular Biology
> Department of of Biological Sciences
> Birkbeck
> University of London
> Malet Street
> London WC1E 7HX
> Telephone +44 (0)20 7631 6815
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at http://www.gromacs.org/search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> Can't post? Read http://www.gromacs.org/mailing_lists/users.php
>



-- 
*********************************************
J. Srinivasa Rao
Post-doctoral Research Associate
C/o Prof. Luis R Cruz Cruz
Computational Biophysics Group
Department of Physics
Drexel University
3141 Chestnut St
Philadelphia, PA 19104, USA.
Ph:  Off:     215-895-1989
       Mob:  704-706-4191
Web:http://jsrao.web.officelive.com/default.aspx
**********************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-users/attachments/20100507/ad558284/attachment.html>