[gmx-users] restarting a sorted shuffled trajectory

[chris.neale at utoronto.ca]

Your specific problem is probably because you sent unsorted  
(=desorted) .trr file to grompp -sort. Due to the order that grompp  
does things, the .trr must already be appropriately sorted at the time  
that it is read in. This is the reason for the complicated procedure  
that I outlined in the description section of g_desort.

Also, and I can't stress this enough: If you want to use grompp -sort  
to start and then continue by anything other than tpbconv, you need to  
use g_desort and NOT trjconv -n deshuffle.ndx. For the procedure that  
I outline you may as well delete deshuffle.ndx as soon as it is created.

I sent more complete information previously but it bounced back. Here  
it is again. Hopefully it is not a double post.

#############################

The instructions are in g_desort.
http://www.gromacs.org/pipermail/gmx-users/2007-January/025584.html

I think you misunderstand one key thing: use g_desort to go from a  
totally mangled
trajectory to a totally unmangled trajectory. If you use g_desort then  
you shouldn't need
the deshuffle.ndx file that was output from trjconv and definately do  
not first deshuffle
and then use g_desort. The program really should be called  
g_deshuffledesort but that
seemed like a too long name.

I have by now fully tested this and I use it for all of my large runs.  
The key test that
convinced me is that after I put tens of 200ps pieces back together  
(trjcat the
desorteddeshuffled .xtc files) I use g_msd to calculate the MSD vs.  
time for water and I
get a straight line without any inconsistencies at the shuffle/sort  
junctures. Also I
have used VMD to view independent water molecules and things are all  
looking good.

The benefit of sort tends to deteriorate over time so I find it best  
to do cycles of
about 200ps. This said, shuffling and sorting my runs gives me 100%  
efficiency to 4cpus =
they run 4x as fast as they do on a single node. I can't come anywhere  
near that with
plain runs.

Here is a sample script that I use to run a cycle. After running this  
once through I
simply increment the $in and $out values and run again. Actually I  
have template
submission scripts and then a daemon that uses sed to put the correct  
numbers in the
correct places to make a new input script for submission each round.

I took a look at your specific information and there is not enough  
there. Please specify
how you are doing your desorting. Also, please take a look at the  
source code where it
contains a description of how to use the program. Note that you need  
to deshuffle/desort
to get something that you can look at but then you need to re-sort  
your trr according to
the new sorting scheem that will be different than the last one and is  
currently unknown.
To do this you do a trial grompp that allows you to generate a ndx  
that can take your
normal .trr to a properly sorted/shuffled trr for input.

Sorry that it's so confusing but it's the only way that I could think  
of. So here is the
sample script first and then after that I have attached the notes that  
are included in
g_desort.c.

I am happy that this has found someone else that is interested in  
using it. Please feel
free to direct and more questions to this list.

#!/bin/sh
#$ -v
LD_LIBRARY_PATH=/usr/lib64/tls:/hpf/tools/n1ge/lib/lx24-amd64:/tools/lam/lam-7.1.2/lib
#$ -v
PATH=/hpf/tools/n1ge/bin/lx24-amd64:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/games:/opt/gnome/bin:/tools/local/bin:/home/cneale/exe:/projects/pomes/cneale/exe/gromacs-3.3.1/exec/fftw-3.1.2/bin:.:/tools/lam/lam-7.1.2/bin

MYMOL=sys1800
ED=/projects/pomes/cneale/exe/gromacs-3.3.1/exec/fftw-3.1.2/bin
DD=/projects/pomes/cneale/exe/gromacs-3.3.1/exec/fftw-3.1.2/share/gromacs/template

MD=/projects/pomes/cneale/micelle/2huge
cd ${MD}

in="_md25"
out="_md26"
mynp=8

echo "Starting grompp..."

#Shuffle the .trr input file correctly. Assume that it is not  
currently shuffled
${ED}/grompp -np ${mynp} -shuffle -sort -f ${MYMOL}${out}.mdp -c
${MYMOL}${in}_deshuffleddesorted.gro -p ${MYMOL}.top -n ${MYMOL}.ndx -o
${MYMOL}${out}_a.tpr -deshuf deshuffle${out}_a.ndx>  
output.${MYMOL}_grompp${out}_a 2>
output.${MYMOL}_grompp${out}_e_a
rm -f deshuffle${out}_a.ndx mdout.mdp
echo System | ${ED}/editconf -f ${MYMOL}${out}_a.tpr -o
${MYMOL}${out}_shuffledsortedInit_a.gro

#g_desort -f original shuffled will unshuffle
#Assuming that g_desort -f shuffled original will REshuffle
${DD}/g_desort -f ${MYMOL}${out}_shuffledsortedInit_a.gro
${MYMOL}${in}_deshuffleddesorted.gro -o reshuffleresort${MYMOL}${out}_a.ndx
${ED}/trjconv -f ${MYMOL}${in}_deshuffleddesorted.trr -o
${MYMOL}${in}_reshuffleresort.trr -n ./reshuffleresort${MYMOL}${out}_a.ndx

#Create the run input file
${ED}/grompp -np ${mynp} -shuffle -sort -f ${MYMOL}${out}.mdp -c
${MYMOL}${in}_deshuffleddesorted.gro -t  
${MYMOL}${in}_reshuffleresort.trr -p ${MYMOL}.top
-n ${MYMOL}.ndx -deshuf deshuffle${out}.ndx -e ${MYMOL}${in}.edr -o  
${MYMOL}${out}.tpr  >
output.${MYMOL}_grompp${out} 2> output.${MYMOL}_grompp${out}_e
rm -f deshuffle${out}.ndx mdout.mdp

#If a new reshuffle.ndx file differs then the run is invalid.
echo System | ${ED}/editconf -f ${MYMOL}${out}.tpr -o
${MYMOL}${out}_shuffledsortedInit.gro

${DD}/g_desort -f ${MYMOL}${out}_shuffledsortedInit.gro
${MYMOL}${in}_deshuffleddesorted.gro -o reshuffleresort${MYMOL}${out}.ndx
look=`diff -q ./reshuffleresort${MYMOL}${out}_a.ndx  
./reshuffleresort${MYMOL}${out}.ndx`
if [ -n "$look" ]; then
   echo There was a big problem. reshuffleresort${out}_a.ndx  
reshuffleresort${out}.ndx are
different.
   mv ${MYMOL}${out}.tpr ${MYMOL}${out}_notValid.tpr
fi

#Create the deshuffle file to properly handle the next run
${DD}/g_desort -f ${MYMOL}${in}_deshuffleddesorted.gro
${MYMOL}${out}_shuffledsortedInit.gro -o deshuffledesort${MYMOL}${out}.ndx

rm -f ${MYMOL}${out}_a.tpr ${MYMOL}${out}_shuffledsortedInit_a.gro
./reshuffleresort${MYMOL}${out}_a.ndx ${MYMOL}${in}_reshuffleresort.trr
${MYMOL}${out}_shuffledsortedInit.gro  
./reshuffleresort${MYMOL}${out}.ndx mdout.mdp

echo "grompp finished"

##########   MDRUN PORTION HERE

MYMOL=sys1800
ED=/projects/pomes/cneale/exe/gromacs-3.3.1/exec/fftw-3.1.2/bin
LAM=/tools/lam/lam-7.1.2/bin

MD=/projects/pomes/cneale/micelle/2huge
cd ${MD}

in="_md25"
out="_md26"
mynp=8

echo "Starting mdrun..."

#Production dynamics
${LAM}/mpirun C ${ED}/mdrun_mpi -np ${mynp} -s ${MYMOL}${out}.tpr  
-deffnm ${MYMOL}${out}
-v > output.${MYMOL}_mdrun${out} 2> output.${MYMOL}_mdrun${out}_e
echo "mdrun finished"

#Deshuffle the gro xtc and trr files. The edr file does not need this.
echo System | ${ED}/trjconv -f ${MYMOL}${out}.xtc -s ${MYMOL}${out}.tpr -n
deshuffledesort${MYMOL}${out}.ndx -o ${MYMOL}${out}_deshuffleddesorted.xtc
echo System | ${ED}/trjconv -f ${MYMOL}${out}.trr -s ${MYMOL}${out}.tpr -n
deshuffledesort${MYMOL}${out}.ndx -o ${MYMOL}${out}_deshuffleddesorted.trr
echo System | ${ED}/trjconv -f ${MYMOL}${out}.gro -s ${MYMOL}${out}.tpr -n
deshuffledesort${MYMOL}${out}.ndx -o ${MYMOL}${out}_deshuffleddesorted.gro

echo ${out} > ./finished

###################
# END SCRIPT
###################


Here are the notes:
   static char *desc[] = {
     "g_desort takes two coordinate files as input and outputs a .ndx  
file that ",
     "can be used to properly sort/shuffle or desort/deshuffle. ",
     "The -f option takes two coordinate files as input and the order  
is important: ",
     "the first file is the desired order and the second file is the  
current order. ",
     "To desort coordinates g_desort -f unsorted.gro sorted.gro and to  
resort ",
     "coordinates g_desort -f sorted.gro unsorted.gro. In order to use  
the -sort option ",
     "of grompp you will only need to desort your trajectory and  
g_desort makes this
trivial. ",
     "However, the benifit of sorting ",
     "decreases over time as the particles move away from their sorted  
positions.
Therefore ",
     "regular re-sorting is desirable. The usage section below  
describes how to do this. ",
     "The basic idea is to pre-sort the .trr file for loading into  
grompp based on a
resort ",
     ".ndx file generated by g_desort after a preliminary grompp  
without the .trr. An
additional ",
     "sorting step is added to ensure that the grompp sorting did not  
change based on the
loaded i",
     ".trr file. The mdrun in then completed and desorted prior to  
starting again. \n",
     "USAGE: \n",
     "1.   grompp -shuffle -sort -c original.gro -o sorted_temp.tpr \n",
     "2.   editconf -f sorted_temp.tpr -o sorted_temp.gro \n",
     "3.   g_desort -f sorted_temp.gro original.gro -o resort_temp.ndx \n",
     "4.   trjconv -f original.trr -n resort_temp.ndx -o sorted.trr \n",
     "5.   grompp -shuffle -sort -c original.gro -t sorted.trr -o  
sorted.tpr \n",
     "6.   editconf -f sorted.tpr -o sorted.gro \n",
     "7.   g_desort -f sorted.gro original.gro -o resort.ndx \n",
     "8.   look=`diff -q resort_temp.ndx resort.ndx`; if [ -n ${look}  
]; then exit; fi \n",
     "9.   g_desort -f original.gro sorted.gro -o desort.ndx \n",
     "10.  mdrun_mpi -s sorted.tpr \n",
     "11a. trjconv -f output.xtc -n desort.ndx -o output_desorted.xtc \n",
     "11b. trjconv -f output.trr -n desort.ndx -o output_desorted.trr \n",
     "11c. trjconv -f output.gro -n desort.ndx -o output_desorted.gro \n",
     "12.  Use these output_desorted files as input to step 1 as the  
original.xtc/trr/gro
files \n",
     "       - the .edr file does not need to be desorted \n",
     "       - g_desort takes care of sorting and shuffling at the  
same time \n",
     "           - Do not use the -deshuf deshuf.ndx file from grompp \n",
     "WARNINGS:\n",
     " - This is a beta version. It might not work properly. Don't  
trust it \n",
     " - Expecting your index files to start from the number 1. If  
they start from the
number 0 then ",
     "you must modify the fprintf output lines to remove the +1 from  
the integer that is
output \n",
     "BUGS: \n",
     "The determination of the expected number of particles per grid  
is sub-optimal ",
     "and if the ratio of system volume over grid points is too small  
then the program
will not allocate ",
     "enough memory. This is detected, but it should be fixed (perhaps  
by actually
counting before allocating\n",
   };