[gmx-users] Fwd: Simulation blowing up in parallel (h-bonds constraints)

jan.joswig at fu-berlin.de jan.joswig at fu-berlin.de
Thu Aug 22 13:47:42 CEST 2019


Dear gromacs-users,

I am encountering a puzzling problem with my simulation of a simple 
protein (~2.000 atoms, only standard amino acids, Calcium cofactor) in 
tip3p water (~10.000 molecules) using the amber99sb-ildn force field and 
lincs constraints set to "h-bonds". When I try to run the production on 
more than one MPI rank, the simulation fails with errors like:

Fatal error:
80 particles communicated to PME rank 1 are more than 2/3 times the cut-off
out of the domain decomposition cell of their charge group in dimension x.
This usually means that your system is not well equilibrated.

Or warnings like:

WARNING: Listed nonbonded interaction between particles 224 and 235
at distance 3.720 which is larger than the table limit 2.569 nm.

(see run_hbonds_tight_parallel.log)


I am quite sure that my system is well equilibrated, though (correct me, 
if I am mistaken here). The problem could not be resolved by using a 
very strict preparation protocol of the system (.mdp files attached), 
using a 1 fs timestep and high lincs (iter 2, order 6) accuracy:

  - steepest decent minimisation (emtol 100 kJ/mol) in vacuum

  - solvation, neutralisation

  - steepest decent minimisation (emtol 100 kJ/mol)

  - conjugate gradient minimisation (emtol 1 kJ/mol, not reached though 
in single precision)

  - NVT eq. 300K (v-rescale tau_t 0.1, coupling groups 
protein/non-protein, all-bonds constraints, position restraints on the 
protein) 300 ps

  - NPT eq. (Berendsen, tau_p 1, all-bonds constraints, position 
restraints on the protein) 300 ps

  - NPT 2 eq. (no position restraints on the protein) 300 ps

  - NPT 3 eq. (switching to h-bond constraints) 400 ps

  - NPT 4 eq. (switching to Parrinello-Rahman) 400 ps

Temperature, pressure, density and potential show reasonable 
fluctuations, I think, during the last run:

Energy                      Average   Err.Est.       RMSD Tot-Drift
-------------------------------------------------------------------------------
Potential                   -534764         39    766.653 5.78602 (kJ/mol)

Density                     1002.32        0.2    2.49712 -1.27332  (kg/m^3)

Temperature                 299.931      0.061    1.61166 0.0932882  (K)

Pressure                   -1.90648        0.8    120.449 2.49509 (bar)

I can not spot any geometrical clashes in the system. Keeping the 1 fs 
timestep and high lincs accuracy also during production does not help 
(as well as decreasing shake-tol and verlet-buffer-tolerance). Switching 
back to all-bonds constraints, however, allows a parallel run again. The 
simulation with h-bonds constraints runs only stable, if run serial on 
one node (only OpenMP thread usage).

I'd like to keep the h-bonds constraints because switching to all-bonds 
is discouraged by gromacs for the used force field:

NOTE 1 [file unknown]:
   You are using constraints on all bonds, whereas the forcefield has been
   parametrized only with constraints involving hydrogen atoms. We suggest
   using constraints = h-bonds instead, this will also improve performance.

Any suggestions on how to achieve a MPI parallel run with this system 
keeping the h-bonds constraints or advice on better equilibration are 
highly appreciated.

Thank you for your help!

Best regards
Jan





-------------- next part --------------
;
;	File 'nopull_tight/mdout.mdp' was generated
;	By user: janjoswig (214764)
;	On host: qcm07
;	At date: Tue Aug 20 10:34:56 2019
;
;	Created by:
;	                     :-) GROMACS - gmx grompp, 2019.1 (-:
;	
;	Executable:   /home/janjoswig/local/gromacs-2019.1/bin/gmx19
;	Data prefix:  /home/janjoswig/local/gromacs-2019.1
;	Working dir:  /home/janjoswig/Projects/Langerin/h3/setup
;	Command line:
;	  gmx grompp -f mdp.mdp -c npt4.gro -t npt4.cpt -p topol.top -o nopull_tight/run.tpr -r npt4.gro -po nopull_tight/mdout.mdp

; VARIOUS PREPROCESSING OPTIONS
; Preprocessor information: use cpp syntax.
; e.g.: -I/home/joe/doe -I/home/mary/roe
include                  = 
; e.g.: -DPOSRES -DFLEXIBLE (note these variable names are case sensitive)
define                   = 

; RUN CONTROL PARAMETERS
integrator               = md
; Start time and timestep in ps
tinit                    = 0
dt                       = 0.001
nsteps                   = 100000000
; For exact run continuation or redoing part of a run
init-step                = 0
; Part index is updated automatically on checkpointing (keeps files separate)
simulation-part          = 1
; mode for center of mass motion removal
comm-mode                = Linear
; number of steps for center of mass motion removal
nstcomm                  = 100
; group(s) for center of mass motion removal
comm-grps                = 

; LANGEVIN DYNAMICS OPTIONS
; Friction coefficient (amu/ps) and random seed
bd-fric                  = 0
ld-seed                  = -1

; ENERGY MINIMIZATION OPTIONS
; Force tolerance and initial step-size
emtol                    = 10
emstep                   = 0.01
; Max number of iterations in relax-shells
niter                    = 20
; Step size (ps^2) for minimization of flexible constraints
fcstep                   = 0
; Frequency of steepest descents steps when doing CG
nstcgsteep               = 1000
nbfgscorr                = 10

; TEST PARTICLE INSERTION OPTIONS
rtpi                     = 0.05

; OUTPUT CONTROL OPTIONS
; Output frequency for coords (x), velocities (v) and forces (f)
nstxout                  = 0
nstvout                  = 0
nstfout                  = 0
; Output frequency for energies to log file and energy file
nstlog                   = 1
nstcalcenergy            = 100
nstenergy                = 1
; Output frequency and precision for .xtc file
nstxout-compressed       = 1
compressed-x-precision   = 1000
; This selects the subset of atoms for the compressed
; trajectory file. You can select multiple groups. By
; default, all atoms will be written.
compressed-x-grps        = Protein CA
; Selection of energy groups
energygrps               = 

; NEIGHBORSEARCHING PARAMETERS
; cut-off scheme (Verlet: particle based cut-offs, group: using charge groups)
cutoff-scheme            = Verlet
; nblist update frequency
nstlist                  = 10
; ns algorithm (simple or grid)
ns_type                  = grid
; Periodic boundary conditions: xyz, no, xy
pbc                      = xyz
periodic-molecules       = no
; Allowed energy error due to the Verlet buffer in kJ/mol/ps per atom,
; a value of -1 means: use rlist
verlet-buffer-tolerance  = 0.0001
; nblist cut-off        
rlist                    = 1.4
; long-range cut-off for switched potentials

; OPTIONS FOR ELECTROSTATICS AND VDW
; Method for doing electrostatics
coulombtype              = PME
coulomb-modifier         = Potential-shift-Verlet
rcoulomb-switch          = 0
rcoulomb                 = 1.4
; Relative dielectric constant for the medium and the reaction field
epsilon-r                = 1
epsilon-rf               = 0
; Method for doing Van der Waals
vdw-type                 = Cut-off
vdw-modifier             = Potential-shift-Verlet
; cut-off lengths       
rvdw-switch              = 0
rvdw                     = 1.4
; Apply long range dispersion corrections for Energy and Pressure
DispCorr                 = EnerPres
; Extension of the potential lookup tables beyond the cut-off
table-extension          = 1
; Separate tables between energy group pairs
energygrp-table          = 
; Spacing for the PME/PPPM FFT grid
fourierspacing           = 0.16
; FFT grid size, when a value is 0 fourierspacing will be used
fourier-nx               = 0
fourier-ny               = 0
fourier-nz               = 0
; EWALD/PME/PPPM parameters
pme_order                = 6
ewald-rtol               = 1e-05
ewald-rtol-lj            = 0.001
lj-pme-comb-rule         = Geometric
ewald-geometry           = 3d
epsilon-surface          = 0
implicit-solvent         = no

; OPTIONS FOR WEAK COUPLING ALGORITHMS
; Temperature coupling  
tcoupl                   = V-rescale
nsttcouple               = -1
nh-chain-length          = 10
print-nose-hoover-chain-variables = no
; Groups to couple separately
tc-grps                  = Protein non-Protein
; Time constant (ps) and reference temperature (K)
tau_t                    = 0.1	  0.1
ref_t                    = 300 	  300
; pressure coupling     
pcoupl                   = Parrinello-Rahman
pcoupltype               = isotropic
nstpcouple               = -1
; Time constant (ps), compressibility (1/bar) and reference P (bar)
tau_p                    = 1.0
compressibility          = 4.5e-5
ref_p                    = 1.0
; Scaling of reference coordinates, No, All or COM
refcoord_scaling         = all

; OPTIONS FOR QMMM calculations
QMMM                     = no
; Groups treated Quantum Mechanically
QMMM-grps                = 
; QM method             
QMmethod                 = 
; QMMM scheme           
QMMMscheme               = normal
; QM basisset           
QMbasis                  = 
; QM charge             
QMcharge                 = 
; QM multiplicity       
QMmult                   = 
; Surface Hopping       
SH                       = 
; CAS space options     
CASorbitals              = 
CASelectrons             = 
SAon                     = 
SAoff                    = 
SAsteps                  = 
; Scale factor for MM charges
MMChargeScaleFactor      = 1

; SIMULATED ANNEALING  
; Type of annealing for each temperature group (no/single/periodic)
annealing                = 
; Number of time points to use for specifying annealing in each group
annealing-npoints        = 
; List of times at the annealing points for each group
annealing-time           = 
; Temp. at each annealing point, for each group.
annealing-temp           = 

; GENERATE VELOCITIES FOR STARTUP RUN
gen_vel                  = no
gen-temp                 = 300
gen-seed                 = -1

; OPTIONS FOR BONDS    
constraints              = h-bonds
; Type of constraint algorithm
constraint_algorithm     = lincs
; Do not constrain the start configuration
continuation             = yes
; Use successive overrelaxation to reduce the number of shake iterations
Shake-SOR                = no
; Relative tolerance of shake
shake-tol                = 0.00001
; Highest order in the expansion of the constraint coupling matrix
lincs_order              = 6
; Number of iterations in the final step of LINCS. 1 is fine for
; normal simulations, but use 2 to conserve energy in NVE runs.
; For energy minimization with constraints it should be 4 to 8.
lincs_iter               = 2
; Lincs will write a warning to the stderr if in one step a bond
; rotates over more degrees than
lincs-warnangle          = 30
; Convert harmonic bonds to morse potentials
morse                    = no

; ENERGY GROUP EXCLUSIONS
; Pairs of energy groups for which all non-bonded interactions are excluded
energygrp-excl           = 

; WALLS                
; Number of walls, type, atom types, densities and box-z scale factor for Ewald
nwall                    = 0
wall-type                = 9-3
wall-r-linpot            = -1
wall-atomtype            = 
wall-density             = 
wall-ewald-zfac          = 3

; COM PULLING          
pull                     = no

; AWH biasing          
awh                      = no

; ENFORCED ROTATION    
; Enforced rotation: No or Yes
rotation                 = no

; Group to display and/or manipulate in interactive MD session
IMD-group                = 

; NMR refinement stuff 
; Distance restraints type: No, Simple or Ensemble
disre                    = No
; Force weighting of pairs in one distance restraint: Conservative or Equal
disre-weighting          = Conservative
; Use sqrt of the time averaged times the instantaneous violation
disre-mixed              = no
disre-fc                 = 1000
disre-tau                = 0
; Output frequency for pair distances to energy file
nstdisreout              = 100
; Orientation restraints: No or Yes
orire                    = no
; Orientation restraints force constant and tau for time averaging
orire-fc                 = 0
orire-tau                = 0
orire-fitgrp             = 
; Output frequency for trace(SD) and S to energy file
nstorireout              = 100

; Free energy variables
free-energy              = no
couple-moltype           = 
couple-lambda0           = vdw-q
couple-lambda1           = vdw-q
couple-intramol          = no
init-lambda              = -1
init-lambda-state        = -1
delta-lambda             = 0
nstdhdl                  = 50
fep-lambdas              = 
mass-lambdas             = 
coul-lambdas             = 
vdw-lambdas              = 
bonded-lambdas           = 
restraint-lambdas        = 
temperature-lambdas      = 
calc-lambda-neighbors    = 1
init-lambda-weights      = 
dhdl-print-energy        = no
sc-alpha                 = 0
sc-power                 = 1
sc-r-power               = 6
sc-sigma                 = 0.3
sc-coul                  = no
separate-dhdl-file       = yes
dhdl-derivatives         = yes
dh_hist_size             = 0
dh_hist_spacing          = 0.1

; Non-equilibrium MD stuff
acc-grps                 = 
accelerate               = 
freezegrps               = 
freezedim                = 
cos-acceleration         = 0
deform                   = 

; simulated tempering variables
simulated-tempering      = no
simulated-tempering-scaling = geometric
sim-temp-low             = 300
sim-temp-high            = 300

; Ion/water position swapping for computational electrophysiology setups
; Swap positions along direction: no, X, Y, Z
swapcoords               = no
adress                   = no

; User defined thingies
user1-grps               = 
user2-grps               = 
userint1                 = 0
userint2                 = 0
userint3                 = 0
userint4                 = 0
userreal1                = 0
userreal2                = 0
userreal3                = 0
userreal4                = 0
; Electric fields
; Format for electric-field-x, etc. is: four real variables:
; amplitude (V/nm), frequency omega (1/ps), time for the pulse peak (ps),
; and sigma (ps) width of the pulse. Omega = 0 means static field,
; sigma = 0 means no pulse, leaving the field to be a cosine function.
electric-field-x         = 0 0 0 0
electric-field-y         = 0 0 0 0
electric-field-z         = 0 0 0 0


More information about the gromacs.org_gmx-users mailing list