[gmx-users] Fwd: Simulation blowing up in parallel (h-bonds constraints)
jan.joswig at fu-berlin.de
jan.joswig at fu-berlin.de
Thu Aug 22 13:47:42 CEST 2019
Dear gromacs-users,
I am encountering a puzzling problem with my simulation of a simple
protein (~2.000 atoms, only standard amino acids, Calcium cofactor) in
tip3p water (~10.000 molecules) using the amber99sb-ildn force field and
lincs constraints set to "h-bonds". When I try to run the production on
more than one MPI rank, the simulation fails with errors like:
Fatal error:
80 particles communicated to PME rank 1 are more than 2/3 times the cut-off
out of the domain decomposition cell of their charge group in dimension x.
This usually means that your system is not well equilibrated.
Or warnings like:
WARNING: Listed nonbonded interaction between particles 224 and 235
at distance 3.720 which is larger than the table limit 2.569 nm.
(see run_hbonds_tight_parallel.log)
I am quite sure that my system is well equilibrated, though (correct me,
if I am mistaken here). The problem could not be resolved by using a
very strict preparation protocol of the system (.mdp files attached),
using a 1 fs timestep and high lincs (iter 2, order 6) accuracy:
- steepest decent minimisation (emtol 100 kJ/mol) in vacuum
- solvation, neutralisation
- steepest decent minimisation (emtol 100 kJ/mol)
- conjugate gradient minimisation (emtol 1 kJ/mol, not reached though
in single precision)
- NVT eq. 300K (v-rescale tau_t 0.1, coupling groups
protein/non-protein, all-bonds constraints, position restraints on the
protein) 300 ps
- NPT eq. (Berendsen, tau_p 1, all-bonds constraints, position
restraints on the protein) 300 ps
- NPT 2 eq. (no position restraints on the protein) 300 ps
- NPT 3 eq. (switching to h-bond constraints) 400 ps
- NPT 4 eq. (switching to Parrinello-Rahman) 400 ps
Temperature, pressure, density and potential show reasonable
fluctuations, I think, during the last run:
Energy Average Err.Est. RMSD Tot-Drift
-------------------------------------------------------------------------------
Potential -534764 39 766.653 5.78602 (kJ/mol)
Density 1002.32 0.2 2.49712 -1.27332 (kg/m^3)
Temperature 299.931 0.061 1.61166 0.0932882 (K)
Pressure -1.90648 0.8 120.449 2.49509 (bar)
I can not spot any geometrical clashes in the system. Keeping the 1 fs
timestep and high lincs accuracy also during production does not help
(as well as decreasing shake-tol and verlet-buffer-tolerance). Switching
back to all-bonds constraints, however, allows a parallel run again. The
simulation with h-bonds constraints runs only stable, if run serial on
one node (only OpenMP thread usage).
I'd like to keep the h-bonds constraints because switching to all-bonds
is discouraged by gromacs for the used force field:
NOTE 1 [file unknown]:
You are using constraints on all bonds, whereas the forcefield has been
parametrized only with constraints involving hydrogen atoms. We suggest
using constraints = h-bonds instead, this will also improve performance.
Any suggestions on how to achieve a MPI parallel run with this system
keeping the h-bonds constraints or advice on better equilibration are
highly appreciated.
Thank you for your help!
Best regards
Jan
-------------- next part --------------
;
; File 'nopull_tight/mdout.mdp' was generated
; By user: janjoswig (214764)
; On host: qcm07
; At date: Tue Aug 20 10:34:56 2019
;
; Created by:
; :-) GROMACS - gmx grompp, 2019.1 (-:
;
; Executable: /home/janjoswig/local/gromacs-2019.1/bin/gmx19
; Data prefix: /home/janjoswig/local/gromacs-2019.1
; Working dir: /home/janjoswig/Projects/Langerin/h3/setup
; Command line:
; gmx grompp -f mdp.mdp -c npt4.gro -t npt4.cpt -p topol.top -o nopull_tight/run.tpr -r npt4.gro -po nopull_tight/mdout.mdp
; VARIOUS PREPROCESSING OPTIONS
; Preprocessor information: use cpp syntax.
; e.g.: -I/home/joe/doe -I/home/mary/roe
include =
; e.g.: -DPOSRES -DFLEXIBLE (note these variable names are case sensitive)
define =
; RUN CONTROL PARAMETERS
integrator = md
; Start time and timestep in ps
tinit = 0
dt = 0.001
nsteps = 100000000
; For exact run continuation or redoing part of a run
init-step = 0
; Part index is updated automatically on checkpointing (keeps files separate)
simulation-part = 1
; mode for center of mass motion removal
comm-mode = Linear
; number of steps for center of mass motion removal
nstcomm = 100
; group(s) for center of mass motion removal
comm-grps =
; LANGEVIN DYNAMICS OPTIONS
; Friction coefficient (amu/ps) and random seed
bd-fric = 0
ld-seed = -1
; ENERGY MINIMIZATION OPTIONS
; Force tolerance and initial step-size
emtol = 10
emstep = 0.01
; Max number of iterations in relax-shells
niter = 20
; Step size (ps^2) for minimization of flexible constraints
fcstep = 0
; Frequency of steepest descents steps when doing CG
nstcgsteep = 1000
nbfgscorr = 10
; TEST PARTICLE INSERTION OPTIONS
rtpi = 0.05
; OUTPUT CONTROL OPTIONS
; Output frequency for coords (x), velocities (v) and forces (f)
nstxout = 0
nstvout = 0
nstfout = 0
; Output frequency for energies to log file and energy file
nstlog = 1
nstcalcenergy = 100
nstenergy = 1
; Output frequency and precision for .xtc file
nstxout-compressed = 1
compressed-x-precision = 1000
; This selects the subset of atoms for the compressed
; trajectory file. You can select multiple groups. By
; default, all atoms will be written.
compressed-x-grps = Protein CA
; Selection of energy groups
energygrps =
; NEIGHBORSEARCHING PARAMETERS
; cut-off scheme (Verlet: particle based cut-offs, group: using charge groups)
cutoff-scheme = Verlet
; nblist update frequency
nstlist = 10
; ns algorithm (simple or grid)
ns_type = grid
; Periodic boundary conditions: xyz, no, xy
pbc = xyz
periodic-molecules = no
; Allowed energy error due to the Verlet buffer in kJ/mol/ps per atom,
; a value of -1 means: use rlist
verlet-buffer-tolerance = 0.0001
; nblist cut-off
rlist = 1.4
; long-range cut-off for switched potentials
; OPTIONS FOR ELECTROSTATICS AND VDW
; Method for doing electrostatics
coulombtype = PME
coulomb-modifier = Potential-shift-Verlet
rcoulomb-switch = 0
rcoulomb = 1.4
; Relative dielectric constant for the medium and the reaction field
epsilon-r = 1
epsilon-rf = 0
; Method for doing Van der Waals
vdw-type = Cut-off
vdw-modifier = Potential-shift-Verlet
; cut-off lengths
rvdw-switch = 0
rvdw = 1.4
; Apply long range dispersion corrections for Energy and Pressure
DispCorr = EnerPres
; Extension of the potential lookup tables beyond the cut-off
table-extension = 1
; Separate tables between energy group pairs
energygrp-table =
; Spacing for the PME/PPPM FFT grid
fourierspacing = 0.16
; FFT grid size, when a value is 0 fourierspacing will be used
fourier-nx = 0
fourier-ny = 0
fourier-nz = 0
; EWALD/PME/PPPM parameters
pme_order = 6
ewald-rtol = 1e-05
ewald-rtol-lj = 0.001
lj-pme-comb-rule = Geometric
ewald-geometry = 3d
epsilon-surface = 0
implicit-solvent = no
; OPTIONS FOR WEAK COUPLING ALGORITHMS
; Temperature coupling
tcoupl = V-rescale
nsttcouple = -1
nh-chain-length = 10
print-nose-hoover-chain-variables = no
; Groups to couple separately
tc-grps = Protein non-Protein
; Time constant (ps) and reference temperature (K)
tau_t = 0.1 0.1
ref_t = 300 300
; pressure coupling
pcoupl = Parrinello-Rahman
pcoupltype = isotropic
nstpcouple = -1
; Time constant (ps), compressibility (1/bar) and reference P (bar)
tau_p = 1.0
compressibility = 4.5e-5
ref_p = 1.0
; Scaling of reference coordinates, No, All or COM
refcoord_scaling = all
; OPTIONS FOR QMMM calculations
QMMM = no
; Groups treated Quantum Mechanically
QMMM-grps =
; QM method
QMmethod =
; QMMM scheme
QMMMscheme = normal
; QM basisset
QMbasis =
; QM charge
QMcharge =
; QM multiplicity
QMmult =
; Surface Hopping
SH =
; CAS space options
CASorbitals =
CASelectrons =
SAon =
SAoff =
SAsteps =
; Scale factor for MM charges
MMChargeScaleFactor = 1
; SIMULATED ANNEALING
; Type of annealing for each temperature group (no/single/periodic)
annealing =
; Number of time points to use for specifying annealing in each group
annealing-npoints =
; List of times at the annealing points for each group
annealing-time =
; Temp. at each annealing point, for each group.
annealing-temp =
; GENERATE VELOCITIES FOR STARTUP RUN
gen_vel = no
gen-temp = 300
gen-seed = -1
; OPTIONS FOR BONDS
constraints = h-bonds
; Type of constraint algorithm
constraint_algorithm = lincs
; Do not constrain the start configuration
continuation = yes
; Use successive overrelaxation to reduce the number of shake iterations
Shake-SOR = no
; Relative tolerance of shake
shake-tol = 0.00001
; Highest order in the expansion of the constraint coupling matrix
lincs_order = 6
; Number of iterations in the final step of LINCS. 1 is fine for
; normal simulations, but use 2 to conserve energy in NVE runs.
; For energy minimization with constraints it should be 4 to 8.
lincs_iter = 2
; Lincs will write a warning to the stderr if in one step a bond
; rotates over more degrees than
lincs-warnangle = 30
; Convert harmonic bonds to morse potentials
morse = no
; ENERGY GROUP EXCLUSIONS
; Pairs of energy groups for which all non-bonded interactions are excluded
energygrp-excl =
; WALLS
; Number of walls, type, atom types, densities and box-z scale factor for Ewald
nwall = 0
wall-type = 9-3
wall-r-linpot = -1
wall-atomtype =
wall-density =
wall-ewald-zfac = 3
; COM PULLING
pull = no
; AWH biasing
awh = no
; ENFORCED ROTATION
; Enforced rotation: No or Yes
rotation = no
; Group to display and/or manipulate in interactive MD session
IMD-group =
; NMR refinement stuff
; Distance restraints type: No, Simple or Ensemble
disre = No
; Force weighting of pairs in one distance restraint: Conservative or Equal
disre-weighting = Conservative
; Use sqrt of the time averaged times the instantaneous violation
disre-mixed = no
disre-fc = 1000
disre-tau = 0
; Output frequency for pair distances to energy file
nstdisreout = 100
; Orientation restraints: No or Yes
orire = no
; Orientation restraints force constant and tau for time averaging
orire-fc = 0
orire-tau = 0
orire-fitgrp =
; Output frequency for trace(SD) and S to energy file
nstorireout = 100
; Free energy variables
free-energy = no
couple-moltype =
couple-lambda0 = vdw-q
couple-lambda1 = vdw-q
couple-intramol = no
init-lambda = -1
init-lambda-state = -1
delta-lambda = 0
nstdhdl = 50
fep-lambdas =
mass-lambdas =
coul-lambdas =
vdw-lambdas =
bonded-lambdas =
restraint-lambdas =
temperature-lambdas =
calc-lambda-neighbors = 1
init-lambda-weights =
dhdl-print-energy = no
sc-alpha = 0
sc-power = 1
sc-r-power = 6
sc-sigma = 0.3
sc-coul = no
separate-dhdl-file = yes
dhdl-derivatives = yes
dh_hist_size = 0
dh_hist_spacing = 0.1
; Non-equilibrium MD stuff
acc-grps =
accelerate =
freezegrps =
freezedim =
cos-acceleration = 0
deform =
; simulated tempering variables
simulated-tempering = no
simulated-tempering-scaling = geometric
sim-temp-low = 300
sim-temp-high = 300
; Ion/water position swapping for computational electrophysiology setups
; Swap positions along direction: no, X, Y, Z
swapcoords = no
adress = no
; User defined thingies
user1-grps =
user2-grps =
userint1 = 0
userint2 = 0
userint3 = 0
userint4 = 0
userreal1 = 0
userreal2 = 0
userreal3 = 0
userreal4 = 0
; Electric fields
; Format for electric-field-x, etc. is: four real variables:
; amplitude (V/nm), frequency omega (1/ps), time for the pulse peak (ps),
; and sigma (ps) width of the pulse. Omega = 0 means static field,
; sigma = 0 means no pulse, leaving the field to be a cosine function.
electric-field-x = 0 0 0 0
electric-field-y = 0 0 0 0
electric-field-z = 0 0 0 0
More information about the gromacs.org_gmx-users
mailing list