[gmx-users] extending parallel simulations with grompp?

[wnoid at hec.utah.edu]

hello all.

i am sorry to bother everyone with a stupid, simple question:  how does
one extend a shuffled multiprocessor npt simulation using grompp?  i am
able to do it with tpbconv, but i would like to also use grompp.  below i
try to explain my problem more clearly.  any help would be greatly
appreciated.

i am trying to understand how to perform an "exact continuation" of an md
simulation in gromacs-3.3.  to be specific, i refer to an "exact
continuation" as an extended simulation where the original simulation plus
the extension generate identical results to a single longer simulation. 
(e.g. 2 1step simulations generate identical results to 1 2step
simulation.)

i understand that a convenient way to perform an "exact continuation" of
an md simulation is to use tpbconv.  if i understand the manual, though,
one can also accomplish this running grompp with
          "gen_vel = no", "unconstrained-start      = yes",
and also using as input to grompp the .trr, and .edr file from the
previous simulation.  ( i also reset the tinit variable so that the time
is right. )
             $grompp -f $gr_inp -c $config -p $topol\
                 -t $old_trr -e $old_ene\
                     -o $gr_out -po $gr_sum >& $gr_log
this seems to work perfectly when i test it on one cpu.  for instance, my
output .trr and .gro files are identical when compared between
            a. 1 5step simulation
            b. 5 1step simulations extended with tpbconv
            c. 5 1step simulations extended with grompp

when i perform multi-processor simulations, i use the -shuffle option to
grompp when i run the first simulation.  if i understand correctly this
generates a shuffled .trr and .tpr file.  when i extend the simulation
with tpbconv, i pass tpbconv the shuffled .trr and .tpr files from the
previous simulation.  the final 5 step .trr and .gro files are identical
for 1 5step simulation and 5 1step simulations extended with tpbconv. 
however, i am confused about how to make this work with grompp for
multiprocessor simulaions.

my protocol for testing grompp is to use the same .mdp options as for the
1 cpu job.  i have tried three different methods to pass the old .trr,
.gro, and .edr files to grompp.

attempt 1.
i try to extend the simulation using the -shuffle option again, but as
input i pass to grompp deshuffled versions of the .gro and .trr files
generated by the first simulation.  so my grompp command is:

      # grompp input
      config="md${newsim}.gro"
      gr_inp="gr${newsim}.mdp"               # deshuffled
      # grompp continuation input
      # NB. assumes deshuffled files
      old_trr="md${oldsim}.trr"              # deshuffled
      old_edr="md${oldsim}.edr"

      # grompp output
      gr_out="md${newsim}.tpr"
      gr_sum="md${newsim}.mdp"
      gr_log="gr${newsim}.log"
      gr_des="md${newsim}.ndx"

      $grompp -f $gr_inp -c $config -p $topol\
        -t $old_trr -e $old_edr\
        -shuffle -np $nnodes\
        -o $gr_out -po $gr_sum -deshuf $gr_des >& $gr_log

the grompp log file seems alright.  i get the following messages:
getting data from old trajectory ...
Reading Coordinates, Velocities and Box size from old trajectory
READ 3 BOX VELOCITIES FROM mdS-C.1-1.1.edr
READ 3 NOSE-HOOVER Xi's FROM mdS-C.1-1.1.edr

however, the mdrun blows up.  somehow the constraints are all messed up in
the extension of the simulation.

 Warning: 1-4 interaction between 678 and 694 at distance 1.185 which is
larger than the 1-4 table size 1.000 nm
This usually means your system is exploding,
Step 0, time 0.001 (ps)  LINCS WARNING
relative constraint deviation after LINCS:
max inf (between atoms 7274 and 7275) rms inf
Step 0, time 0.001 (ps)  LINCS WARNING
relative constraint deviation after LINCS:
max 1544879257967132672.000000 (between atoms 3856 and 3857) rms
79986309824774144.000000
bonds that rotated more than 30 degrees:
 atom 1 atom 2  angle  previous, current, constraint length
  10154  10157   48.3    2.9773 4670545543036928.0000      0.1470
....

and the system explodes as the log file suggests.  it appears that somehow
the wrong atoms are now being constrained together.  for instance atoms
10154 and 10157 are both HW1 atoms on different waters.  although mdrun
seems to think that they are atoms in lipids.   i thought that feeding
grompp the deshuffled results from the previous simulation and then asking
it to reshuffle them would take care of everything.

attempt 2.
on the other hand if i give grompp the shuffled .trr and .gro files from
the previous simulation and do _not_ give it the -shuffle option, i get a
slew of warnings in the grompp logfile about atom types not being
consistent.  it appears then that grompp is confused about which are the
water molecules and which are the lipids.  then again in the md log file i
get similar warnings about constraints and the system blows up.

attempt 3.
finally, i seem to get the right results when i give grompp the shuffled
.trr and .gro files from the previous simulation and also have it use
-shuffle option again.  it appears that grompp is confused that the atom
names don't line up with original (unshuffled) .top file.  as in attempt 2
it prints out a slew of warnings:

Warning: atom names in S-C.1-1.top and mdS-C.1-1.2.gro don't match (C1 -
HW1) Warning: atom names in S-C.1-1.top and mdS-C.1-1.2.gro don't match
(C2 - HW2) Warning: atom names in S-C.1-1.top and mdS-C.1-1.2.gro don't
match (C3 - OW) Warning: atom names in S-C.1-1.top and mdS-C.1-1.2.gro
don't match (C4 - HW1) Warning: atom names in S-C.1-1.top and
mdS-C.1-1.2.gro don't match (C5 - HW2) ...
(more than 20 non-matching atom names)
WARNING 4 [file "S-C.1-1.top", line 15]:
  10440 non-matching atom names
  atom names from S-C.1-1.top will be used
  atom names from mdS-C.1-1.2.gro will be ignored

so it seems like something bad is happening here like in attempt 2.  but
then the mdrun simulation does not blow up and the output is the same as
continuing with tpbconv.  what is grompp doing?  should i not worry about
these warnings?  is it somehow making two errors that cancel: after
shuffling atoms that it thinks are wrong it actually puts them back in the
right place?

i'm very confused.

thanks to anyone who read this far.  any help would be greatly appreciated.

peace,

will noid