[gmx-users] Nonrepeatable results for gromacs 4.0.5

Jim Kress jimkress_58 at kressworks.org
Sun Jun 7 00:59:42 CEST 2009


I've been doing multiple runs using gromacs v 4.0.5 mdrun and a constant
topol.tpr input file.  Unfortunately, the results that I get in my md.log
differ from run to run.

For example, 

Run 1

Started mdrun on node 0 Fri May 22 22:53:51 2009

           Step           Time         Lambda
              0        0.00000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    1.95406e+02    1.04746e+02    4.97704e+01    4.13260e+01    1.40158e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    2.60139e+03   -2.64656e+04   -2.20714e+04    4.03780e+03   -1.80336e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.03142e+02   -8.46977e+02    1.92470e-05

DD  step 9 load imb.: force 29.9%

At step 10 the performance loss due to force load imbalance is 8.6 %

NOTE: Turning on dynamic load balancing

DD  step 99  vol min/aver 0.731  load imb.: force  6.9%

           Step           Time         Lambda
            100        0.20000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.05310e+02    1.30129e+02    5.63474e+01    1.81814e+01    1.44270e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    2.69491e+03   -2.69624e+04   -2.24148e+04    4.19456e+03   -1.82203e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.14910e+02   -5.19031e+02    1.76248e-05

DD  load balancing is limited by minimum cell size in dimension Y
DD  step 199  vol min/aver 0.766! load imb.: force 10.7%

           Step           Time         Lambda
            200        0.40000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.20550e+02    1.09068e+02    6.93319e+01    5.32511e+01    1.43458e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    2.78241e+03   -2.70319e+04   -2.23627e+04    4.13455e+03   -1.82281e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.10405e+02   -5.01205e+02    1.70105e-05

DD  load balancing is limited by minimum cell size in dimension Y
DD  step 299  vol min/aver 0.750! load imb.: force  3.3%

           Step           Time         Lambda
            300        0.60000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.17474e+02    8.65489e+01    5.24995e+01    4.72592e+01    1.44419e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    3.17643e+03   -2.72841e+04   -2.22597e+04    3.95024e+03   -1.83095e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    2.96568e+02    1.40098e+03    1.55861e-05

DD  step 399  vol min/aver 0.700  load imb.: force  5.9%

           Step           Time         Lambda
            400        0.80000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.43143e+02    9.93116e+01    7.16796e+01    4.63666e+01    1.46722e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    2.84150e+03   -2.70065e+04   -2.22372e+04    4.05976e+03   -1.81775e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.04791e+02    2.48551e+02    1.61141e-05

DD  step 499  vol min/aver 0.678  load imb.: force  6.6%

           Step           Time         Lambda
            500        1.00000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.19638e+02    8.98359e+01    8.99946e+01    5.16612e+01    1.46338e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    2.80267e+03   -2.68507e+04   -2.21335e+04    4.14195e+03   -1.79916e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.10961e+02   -1.17210e+02    1.71420e-05

DD  step 599  vol min/aver 0.678  load imb.: force  6.7%

           Step           Time         Lambda
            600        1.20000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.32938e+02    1.04322e+02    7.11343e+01    2.16046e+01    1.45770e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    3.07425e+03   -2.71320e+04   -2.21700e+04    4.17285e+03   -1.79972e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.13281e+02    5.60002e+01    1.97532e-05

DD  step 699  vol min/aver 0.664  load imb.: force 13.1%

----------------------------------------------------------------------------
-------------------------------------

Run 2

Step 0 is the same, but then the results start to differ more and more:

Started mdrun on node 0 Sat Jun  6 14:38:03 2009

           Step           Time         Lambda
              0        0.00000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    1.95406e+02    1.04746e+02    4.97704e+01    4.13260e+01    1.40158e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    2.60139e+03   -2.64656e+04   -2.20714e+04    4.03780e+03   -1.80336e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.03142e+02   -8.46977e+02    1.92470e-05

DD  step 9 load imb.: force 32.9%

At step 10 the performance loss due to force load imbalance is 8.8 %

NOTE: Turning on dynamic load balancing

DD  load balancing is limited by minimum cell size in dimension Y
DD  step 99  vol min/aver 0.711! load imb.: force 13.3%

           Step           Time         Lambda
            100        0.20000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.05314e+02    1.30130e+02    5.63508e+01    1.81808e+01    1.44270e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    2.69491e+03   -2.69627e+04   -2.24151e+04    4.19468e+03   -1.82204e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.14919e+02   -5.13520e+02    1.76037e-05

DD  load balancing is limited by minimum cell size in dimension Y Z
DD  step 199  vol min/aver 0.760! load imb.: force 12.7%

           Step           Time         Lambda
            200        0.40000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.20600e+02    1.09011e+02    6.92931e+01    5.32915e+01    1.43453e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    2.78045e+03   -2.70297e+04   -2.23626e+04    4.13378e+03   -1.82288e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.10348e+02   -5.07193e+02    1.69736e-05

DD  load balancing is limited by minimum cell size in dimension Y
DD  step 299  vol min/aver 0.757! load imb.: force 12.1%

           Step           Time         Lambda
            300        0.60000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.18647e+02    8.76939e+01    5.26630e+01    4.67556e+01    1.44438e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    3.15118e+03   -2.72121e+04   -2.22108e+04    3.91294e+03   -1.82978e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    2.93768e+02    1.36397e+03    1.56756e-05

DD  load balancing is limited by minimum cell size in dimension Y Z
DD  step 399  vol min/aver 0.688! load imb.: force 12.6%

           Step           Time         Lambda
            400        0.80000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.37290e+02    9.91231e+01    6.10010e+01    3.87031e+01    1.46621e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    2.68805e+03   -2.68308e+04   -2.22404e+04    4.05083e+03   -1.81896e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.04120e+02   -2.55369e+02    1.63518e-05

DD  load balancing is limited by minimum cell size in dimension Z
DD  step 499  vol min/aver 0.677! load imb.: force 10.1%

           Step           Time         Lambda
            500        1.00000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.30361e+02    8.47035e+01    8.84842e+01    4.44614e+01    1.44045e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    2.91452e+03   -2.70665e+04   -2.22635e+04    4.18886e+03   -1.80746e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.14483e+02    1.47268e+02    1.75008e-05

DD  load balancing is limited by minimum cell size in dimension Z
DD  step 599  vol min/aver 0.692! load imb.: force  7.7%

           Step           Time         Lambda
            600        1.20000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    2.19896e+02    9.93832e+01    6.10071e+01    2.95745e+01    1.45874e+03
        LJ (SR)   Coulomb (SR)      Potential    Kinetic En.   Total Energy
    2.81555e+03   -2.71300e+04   -2.24458e+04    4.17303e+03   -1.82728e+04
    Temperature Pressure (bar)  Cons. rmsd ()
    3.13294e+02   -3.05949e+02    1.64990e-05

DD  load balancing is limited by minimum cell size in dimension Z
DD  step 699  vol min/aver 0.719! load imb.: force  4.9%

----------------------------------------------------------------------------
--------------------

Any ideas why I am seeing this?

Here is the initial mdrun printed input info:


                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                            :-)  VERSION 4.0.5  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2008, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                              :-)  mdrun_mpi  (-:


++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
B. Hess and C. Kutzner and D. van der Spoel and E. Lindahl GROMACS 4:
Algorithms for highly efficient, load-balanced, and scalable molecular
simulation J. Chem. Theory Comput. 4 (2008) pp. 435-447
-------- -------- --- Thank You --- -------- --------


++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
D. van der Spoel, E. Lindahl, B. Hess, G. Groenhof, A. E. Mark and H. J. C.
Berendsen
GROMACS: Fast, Flexible and Free
J. Comp. Chem. 26 (2005) pp. 1701-1719
-------- -------- --- Thank You --- -------- --------


++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
E. Lindahl and B. Hess and D. van der Spoel GROMACS 3.0: A package for
molecular simulation and trajectory analysis J. Mol. Mod. 7 (2001) pp.
306-317
-------- -------- --- Thank You --- -------- --------


++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
H. J. C. Berendsen, D. van der Spoel and R. van Drunen
GROMACS: A message-passing parallel molecular dynamics implementation Comp.
Phys. Comm. 91 (1995) pp. 43-56
-------- -------- --- Thank You --- -------- --------

parameters of the run:
   integrator           = md
   nsteps               = 5000000
   init_step            = 0
   ns_type              = Grid
   nstlist              = 10
   ndelta               = 2
   nstcomm              = 1
   comm_mode            = Linear
   nstlog               = 100
   nstxout              = 50
   nstvout              = 0
   nstfout              = 0
   nstenergy            = 100
   nstxtcout            = 0
   init_t               = 0
   delta_t              = 0.002
   xtcprec              = 1000
   nkx                  = 0
   nky                  = 0
   nkz                  = 0
   pme_order            = 4
   ewald_rtol           = 1e-05
   ewald_geometry       = 0
   epsilon_surface      = 0
   optimize_fft         = FALSE
   ePBC                 = xyz
   bPeriodicMols        = FALSE
   bContinuation        = FALSE
   bShakeSOR            = FALSE
   etc                  = Berendsen
   epc                  = No
   epctype              = Isotropic
   tau_p                = 0.5
   ref_p (3x3):
      ref_p[    0]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      ref_p[    1]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      ref_p[    2]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
   compress (3x3):
      compress[    0]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      compress[    1]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      compress[    2]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
   refcoord_scaling     = No
   posres_com (3):
      posres_com[0]= 0.00000e+00
      posres_com[1]= 0.00000e+00
      posres_com[2]= 0.00000e+00
   posres_comB (3):
      posres_comB[0]= 0.00000e+00
      posres_comB[1]= 0.00000e+00
      posres_comB[2]= 0.00000e+00
   andersen_seed        = 815131
   rlist                = 1
   rtpi                 = 0.05
   coulombtype          = Cut-off
   rcoulomb_switch      = 0
   rcoulomb             = 1
   vdwtype              = Cut-off
   rvdw_switch          = 0
   rvdw                 = 1
   epsilon_r            = 1
   epsilon_rf           = 1
   tabext               = 1
   implicit_solvent     = No
   gb_algorithm         = Still
   gb_epsilon_solvent   = 80
   nstgbradii           = 1
   rgbradii             = 2
   gb_saltconc          = 0
   gb_obc_alpha         = 1
   gb_obc_beta          = 0.8
   gb_obc_gamma         = 4.85
   sa_surface_tension   = 2.092
   DispCorr             = No
   free_energy          = no
   init_lambda          = 0
   sc_alpha             = 0
   sc_power             = 0
   sc_sigma             = 0.3
   delta_lambda         = 0
   nwall                = 0
   wall_type            = 9-3
   wall_atomtype[0]     = -1
   wall_atomtype[1]     = -1
   wall_density[0]      = 0
   wall_density[1]      = 0
   wall_ewald_zfac      = 3
   pull                 = no
   disre                = No
   disre_weighting      = Conservative
   disre_mixed          = FALSE
   dr_fc                = 1000
   dr_tau               = 0
   nstdisreout          = 100
   orires_fc            = 0
   orires_tau           = 0
   nstorireout          = 100
   dihre-fc             = 1000
   em_stepsize          = 0.01
   em_tol               = 10
   niter                = 20
   fc_stepsize          = 0
   nstcgsteep           = 1000
   nbfgscorr            = 10
   ConstAlg             = Lincs
   shake_tol            = 0.0001
   lincs_order          = 4
   lincs_warnangle      = 30
   lincs_iter           = 1
   bd_fric              = 0
   ld_seed              = 1993
   cos_accel            = 0
   deform (3x3):
      deform[    0]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      deform[    1]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      deform[    2]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
   userint1             = 0
   userint2             = 0
   userint3             = 0
   userint4             = 0
   userreal1            = 0
   userreal2            = 0
   userreal3            = 0
   userreal4            = 0
grpopts:
   nrdf:     284.733     2919.27
   ref_t:         300         300
   tau_t:         0.1         0.1
anneal:          No          No
ann_npoints:           0           0
   acc:	           0           0           0
   nfreeze:           N           N           N
   energygrp_flags[  0]: 0
   efield-x:
      n = 0
   efield-xt:
      n = 0
   efield-y:
      n = 0
   efield-yt:
      n = 0
   efield-z:
      n = 0
   efield-zt:
      n = 0
   bQMMM                = FALSE
   QMconstraints        = 0
   QMMMscheme           = 0
   scalefactor          = 1
qm_opts:
   ngQM                 = 0

Initializing Domain Decomposition on 12 nodes Dynamic load balancing: auto
Will sort the charge groups at every domain (re)decomposition Initial
maximum inter charge-group distances:
    two-body bonded interactions: 0.597 nm, LJ-14, atoms 5 18
  multi-body bonded interactions: 0.597 nm, Proper Dih., atoms 5 18 Minimum
cell size due to bonded interactions: 0.657 nm Maximum distance for 5
constraints, at 120 deg. angles, all-trans: 0.820 nm Estimated maximum
distance required for P-LINCS: 0.820 nm This distance will limit the DD cell
size, you can override this with -rcon Scaling the initial minimum size with
1/0.8 (option -dds) = 1.25 Optimizing the DD grid for 12 cells with a
minimum initial size of 1.025 nm The maximum allowed number of cells is: X 2
Y 3 Z 2 Domain decomposition grid 2 x 3 x 2, separate PME nodes 0 Domain
decomposition nodeid 0, coordinates 0 0 0

Table routines are used for coulomb: FALSE
Table routines are used for vdw:     FALSE
Cut-off's:   NS: 1   Coulomb: 1   LJ: 1
System total charge: 1.000
Generated table with 1000 data points for 1-4 COUL.
Tabscale = 500 points/nm
Generated table with 1000 data points for 1-4 LJ6.
Tabscale = 500 points/nm
Generated table with 1000 data points for 1-4 LJ12.
Tabscale = 500 points/nm

Enabling SPC water optimization for 487 molecules.

Configuring nonbonded kernels...
Testing x86_64 SSE support... present.


Removing pbc first time

Initializing Parallel LINear Constraint Solver

++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
B. Hess
P-LINCS: A Parallel Linear Constraint Solver for molecular simulation J.
Chem. Theory Comput. 4 (2008) pp. 116-122
-------- -------- --- Thank You --- -------- --------

The number of constraints is 144
There are inter charge-group constraints, will communicate selected
coordinates each lincs iteration

++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
S. Miyamoto and P. A. Kollman
SETTLE: An Analytical Version of the SHAKE and RATTLE Algorithms for Rigid
Water Models J. Comp. Chem. 13 (1992) pp. 952-962
-------- -------- --- Thank You --- -------- --------


Linking all bonded interactions to atoms

The initial number of communication pulses is: X 1 Y 1 Z 1 The initial
domain decomposition cell size is: X 1.21 nm Y 1.05 nm Z 1.11 nm

The maximum allowed distance for charge groups involved in interactions is:
                 non-bonded interactions           1.000 nm
            two-body bonded interactions  (-rdd)   1.000 nm
          multi-body bonded interactions  (-rdd)   1.000 nm
  atoms separated by up to 5 constraints  (-rcon)  1.054 nm

When dynamic load balancing gets turned on, these settings will change to:
The maximum number of communication pulses is: X 1 Y 2 Z 1 The minimum size
for domain decomposition cells is 0.826 nm The requested allowed shrink of
DD cells (option -dds) is: 0.80 The allowed shrink of domain decomposition
cells is: X 0.82 Y 0.78 Z 0.90 The maximum allowed distance for charge
groups involved in interactions is:
                 non-bonded interactions           1.000 nm
            two-body bonded interactions  (-rdd)   1.000 nm
          multi-body bonded interactions  (-rdd)   0.826 nm
  atoms separated by up to 5 constraints  (-rcon)  0.826 nm


Making 3D domain decomposition grid 2 x 3 x 2, home cell index 0 0 0

Center of mass motion removal mode is Linear We have the following groups
for center of mass motion removal:
  0:  rest

++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
H. J. C. Berendsen, J. P. M. Postma, A. DiNola and J. R. Haak Molecular
dynamics with coupling to an external bath J. Chem. Phys. 81 (1984) pp.
3684-3690
-------- -------- --- Thank You --- -------- --------

There are: 1604 Atoms
Charge group distribution at step 0: 45 50 45 42 46 41 44 45 41 47 51 47
Grid: 4 x 4 x 4 cells

Constraining the starting coordinates (step 0)

Constraining the coordinates at t0-dt (step 0) RMS relative constraint
deviation after constraining: 2.38e-05 Initial temperature: 299.151 K

Which is, of course, identical between the runs.

Thanks for any comments/ advice.

Jim




More information about the gromacs.org_gmx-users mailing list