[gmx-users] grompp -np -shuffle with position restraints

Low Soo Mei lowsm at bii.a-star.edu.sg
Mon May 12 18:24:26 CEST 2008


Hi Gromacs users,

When I run grompp for 16 nodes to prepare my file for parallel run on my 
cluster, and also -shuffle to spread the load evenly over the nodes, I 
find that the position restraints get shuffled as well.  This is my 
command line:

grompp -v -f heat.mdp -c min_end.pdb -p dmpc_npat_cluster.top -po 
heattest4shuffleout.mdp -o heattest4shuffle.tpr -pp 
heattest4shuffleout.top -np 16 -deshuf heattest4shuffle -shuffle

And this is my grompp output (in full, which is quite long, sorry) :

 :-)  G  R  O  M  A  C  S  (-:

             Gallium Rubidium Oxygen Manganese Argon Carbon Silicon

                            :-)  VERSION 3.3.3  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2008, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                 :-)  /usr/local/gromacs/bin-mpich/grompp  (-:

Option     Filename  Type         Description
------------------------------------------------------------
  -f       heat.mdp  Input, Opt!  grompp input file with MD parameters
 -po heattest4shuffleout.mdp  Output       grompp input file with MD
                                   parameters
  -c    min_end.pdb  Input        Generic structure: gro g96 pdb tpr tpb tpa
                                   xml
  -r       conf.gro  Input, Opt.  Generic structure: gro g96 pdb tpr tpb tpa
                                   xml
 -rb       conf.gro  Input, Opt.  Generic structure: gro g96 pdb tpr tpb tpa
                                   xml
  -n      index.ndx  Input, Opt.  Index file
-deshufattest4shuffle.ndx  Output, Opt! Index file
  -p dmpc_npat_cluster.top  Input        Topology file
 -pp heattest4shuffleout.top  Output, Opt! Topology file
  -o heattest4shuffle.tpr  Output       Generic run input: tpr tpb tpa xml
  -t       traj.trr  Input, Opt.  Full precision trajectory: trr trj
  -e       ener.edr  Input, Opt.  Generic energy: edr ene

Option       Type   Value   Description
------------------------------------------------------
-[no]h       bool   no      Print help info and quit
-nice        int    0       Set the nicelevel
-[no]v       bool   yes     Be loud and noisy
-time        real   -1      Take frame at or first after this time.
-np          int    16      Generate statusfile for # nodes
-[no]shuffle bool   yes     Shuffle molecules over nodes
-[no]sort    bool   no      Sort molecules according to X coordinate
-[no]rmvsbds bool   yes     Remove constant bonded interactions with virtual
                            sites
-load        string         Releative load capacity of each node on a
                            parallel machine. Be sure to use quotes around
                            the string, which should contain a number for
                            each node
-maxwarn     int    10      Number of warnings after which input processing
                            stops
-[no]check14 bool   no      Remove 1-4 interactions without Van der Waals
-[no]zero    bool   no      Set parameters for bonded interactions without
                            defaults to zero instead of generating an error
-[no]renum   bool   yes     Renumber atomtypes and minimize number of
                            atomtypes

creating statusfile for 16 nodes...
checking input for internal consistency...
calling /usr/bin/cpp...
processing topology...
Generated 483 of the 1653 non-bonded parameter combinations
Generating 1-4 interactions: fudge = 1
Generated 663 of the 1653 1-4 parameter combinations
Excluding 3 bonded neighbours for DMP 128
turning all bonds into constraints...
Excluding 2 bonded neighbours for SOL 3655
turning all bonds into constraints...
Moltype       DMP       SOL    #atoms
CPU   0         8       229      1055
CPU   1         8       229      1055
CPU   2         8       229      1055
CPU   3         8       229      1055
CPU   4         8       229      1055
CPU   5         8       229      1055
CPU   6         8       229      1055
CPU   7         8       228      1052
CPU   8         8       228      1052
CPU   9         8       228      1052
CPU  10         8       228      1052
CPU  11         8       228      1052
CPU  12         8       228      1052
CPU  13         8       228      1052
CPU  14         8       228      1052
CPU  15         8       228      1052
Made a shuffling table with 3783 entries [molecules]
processing coordinates...
Shuffling coordinates...
Entering shuffle_xv
double-checking input for internal consistency...
Reading position restraint coords from min_end.pdb
WARNING shuffling position restraints. Please scheck your results.
renumbering atomtypes...
converting bonded parameters...
#     ANGLES:   6784
#      PDIHS:   2176
#     RBDIHS:   2560
#      IDIHS:   384
#       LJ14:   3712
#     POSRES:   1408
#     CONSTR:   5760
#     SETTLE:   3655
Walking down the molecule graph to make shake-blocks
initialising group options...
processing index file...
Analysing residue names:
Opening library file /usr/local/gromacs/share/gromacs/top/aminoacids.dat
There are:  3783      OTHER residues
There are:     0    PROTEIN residues
There are:     0        DNA residues
Analysing Other...
Simulated annealing for group DMP: Single, 11 timepoints
Time (ps)   Temperature (K)
      0.0        0.0
    100.0       32.3
    200.0       64.6
    300.0       96.9
    400.0      129.2
    500.0      161.5
    600.0      193.8
    700.0      226.1
    800.0      258.4
    900.0      290.7
   1000.0-     323.0
Simulated annealing for group SOL: Single, 11 timepoints
Time (ps)   Temperature (K)
      0.0        0.0
    100.0       32.3
    200.0       64.6
    300.0       96.9
    400.0      129.2
    500.0      161.5
    600.0      193.8
    700.0      226.1
    800.0      258.4
    900.0      290.7
   1000.0-     323.0
Making dummy/rest group for Acceleration containing 16853 elements
Making dummy/rest group for Freeze containing 16853 elements
Making dummy/rest group for VCM containing 16853 elements
Number of degrees of freedom in T-Coupling group DMP is 11902.94
Number of degrees of freedom in T-Coupling group SOL is 21928.05
Making dummy/rest group for User1 containing 16853 elements
Making dummy/rest group for User2 containing 16853 elements
Making dummy/rest group for XTC containing 16853 elements
Making dummy/rest group for Or. Res. Fit containing 16853 elements
Making dummy/rest group for QMMM containing 16853 elements
T-Coupling       has 2 element(s): DMP SOL
Energy Mon.      has 2 element(s): DMP SOL
Acceleration     has 1 element(s): rest
Freeze           has 1 element(s): rest
User1            has 1 element(s): rest
User2            has 1 element(s): rest
VCM              has 1 element(s): rest
XTC              has 1 element(s): rest
Or. Res. Fit     has 1 element(s): rest
QMMM             has 1 element(s): rest
Checking consistency between energy and charge groups...
splitting topology...
Walking down the molecule graph to make shake-blocks
There are 7751 charge group borders and 3783 shake borders
There are 3783 total borders
Division over nodes in atoms:
    1052    1052    1055    1052    1055    1052    1055    1052    
1055    1052    1055    1052    1055    1052    1055    1052
writing run input file...

gcq#40: "Yeah, a Wuzz, Or a Jerk" (F. Black)

-------


The fun comes when I do the mpi mdrun on my cluster.  From time 0.044 ps 
(step 22) till the time I kill the run to put it out of its misery, I 
get SETTLE errors.  From time 0.090 ps (step 45) till the time I kill 
the run, I get LINCS errors as well.

When I grompp the same files to run on 16 nodes without the -shuffle 
flag, the position restraints are fine, the mpi mdrun is smooth, and in 
my output trajectory I can see that the position restraints are 
definitely working.  Of course, since there is no -shuffle flag, the 
load balancing is terrible.  It takes 15 hours to complete, instead of 
the 5.5 hours on a single node that I first tried.  I'm not surprised 
that it takes longer when spread out over 16 nodes because my cluster is 
using gigabit ethernet to communicate.  What's surprising is that the 
grompp -shuffle flag kills my position restraints.

Using the -sort flag in addition to the -shuffle flag (just by adding 
"-sort" to the above grompp command line) gives the same problem of 
killing my position restraints.

Anyone has any ideas?  My cluster is using 3.3.3 on intel xeon woodcrest 
nodes.  But I am not very sure if this is a 3.3.3 problem or a cluster 
MPI problem, or some interaction between the two.

Many thanks for your advice, in advance.

Cheers,
Soo Mei



More information about the gromacs.org_gmx-users mailing list