[gmx-users] Contradiction beetwen Domain Decomposition in PME and the use of SPECBOND

[Олег Шавыкин]

 Dear All,

We tried to make simulation of lysine dendrimer (with repeat unit consisting of one branched lysine (LYH) and two linear lysine (CLYO and LYS) in each generation) using SPECBOND command to create branched structure.
We use next specbond.dat file:

1
CLYO    C       1       LYH     NZ      1       0.151   CLYO    LYH

We use PME for calculation of electrostatics interactions which use Domain Decomposition program.
Simulation is running well on single processor on PC using command:

mdrun -s trp_md.tpr -o trp_md.trr -c trp_pmd.gro -g md.log -e md.edr

or on 4 processors on PC using command:

mpirun -n 4 mdrun -dlb no -s trp_md.tpr -o trp_md.trr -c trp_pmd.gro -g md.log -e md.edr

We use here option "-dlb no", because mpirun for our task doesn't work on PC without it and give error message similar to that 
on big computer (see below).

However, it doesn't work on supercomputer using command:  

sbatch -n 64 -J task -t 5 -p test ompi /home/users/user1/_scratch/gromacs/bin/mdrun -dlb no -s trp_md.tpr -o traj.xtc -c trp_end.gro -npme 32 -v

We use same approach and the same command on supercomputer for similar molucule before (without SPECBOND) and it works O'K.

In the case of SPECBOND we have got next error message:

Initializing Domain Decomposition on 64 nodes
Dynamic load balancing: no
Will sort the charge groups at every domain (re)decomposition
Initial maximum inter charge-group distances:
    two-body bonded interactions: 0.410 nm, LJ-14, atoms 830 838
  multi-body bonded interactions: 4.442 nm, Improper Dih., atoms 496 495
Minimum cell size due to bonded interactions: 4.886 nm
Maximum distance for 5 constraints, at 120 deg. angles, all-trans: 0.763 nm
Estimated maximum distance required for P-LINCS: 0.763 nm
Using 32 separate PME nodes, per user request
Optimizing the DD grid for 32 cells with a minimum initial size of 4.886 nm
The maximum allowed number of cells is: X 2 Y 2 Z 2

-------------------------------------------------------
Program mdrun, VERSION 4.5.6
Source code file: domdec.c, line: 6436

Fatal error:
There is no domain decomposition for 32 nodes that is compatible with the given box and a minimum cell size of 4.88592 nm
Change the number of nodes or mdrun option -rdd
Look in the log file for details on the domain decomposition
For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors

We tried change number of nodes and used mdrun option -rdd, but without success.

We use next md.mdp:

title               =  Yo
cpp                 =  /usr/bin/cpp
constraints         =  all-bonds
integrator          =  md
dt                  =  0.002    ; ps !
nsteps              =  50000000 ; total 100 ns.
nstcomm             =  1
nstxtcout           =  1000
nstxout             =  0
nstvout             =  0
nstfout             =  0
nstlog              =  50000
nstenergy           =  5000
nstlist             =  10
ns_type             =  grid
coulombtype = PME
rvdw = 0.9
rlist = 0.9
rcoulomb = 0.9
fourierspacing = 0.12
pme_order = 4
ewald_rtol = 1e-5
; Berendsen temperature coupling is on in two groups
Tcoupl              =  v-rescale
tc-grps             =  Protein     SOL   CL
tau_t               =  0.1     0.1   0.0
ref_t               =  300     300   300
; Energy monitoring
energygrps          =  Protein     SOL    CL
; Isotropic pressure coupling is now on
Pcoupl              =  Parrinello-Rahman
Pcoupltype          = isotropic
tau_p               =  0.5
compressibility     =  4.5e-5
ref_p               =  1.0
; Generate velocites is off at 300 K.
gen_vel             =  no
gen_temp            =  300.0
gen_seed            =  371925

please, help us