[gmx-users] Re: trouble parallelizing a simulation over a cluster
Hassan Shallal
hshallal at PACIFIC.EDU
Fri Dec 10 22:27:51 CET 2010
Thanks Carsten for the hint, you were exactly right, the mdrun_mpi was not linked to the openmpi library. I corrected this by recompiling the mdrun_mpi after issuing the following command:
module load openmpi/gnu
Now it is running very well
Hassan
________________________________
From: gmx-users-bounces at gromacs.org on behalf of Carsten Kutzner
Sent: Wed 12/8/2010 4:16 AM
To: Discussion list for GROMACS users
Subject: [gmx-users] Re: trouble parallelizing a simulation over a cluster
On Dec 8, 2010, at 1:03 PM, Hassan Shallal wrote:
> Thanks a lot Justin for the very helpful answers concerning the pressure equilibration. Using Berendsen Barostat over 200 ps has lead to the correct average pressure...
>
> I have another issue to discuss with you and with the Gromacs mailing list members;
>
> I have been trying to run a simulation on a computer cluster for the first time using a sub file script. What happened is that the .sub file attempted to run the simulation 24 times instead of parallelizing it over the 24 processors!!!!
>
> Here are the contents of run_1.sub file I tried to use to parallelize the simulation using qsub run_1.sub
>
> #PBS -S /bin/bash
> #PBS -N run_1
> #PBS -l nodes=3:ppn=8
> module load openmpi/gnu
> mpirun -np 24 /home/hassan/bin/bin/mdrun_mpi -deffnm run_1 -v &> run_1_update.txt
> exit $?
> What happens it that it outputs 24 run_1.log files, starting from #run_1.log1# all the way to #run_1.log23#...Has anyone faced this problem before? and If yes, any hints or solutions?
This typically happens when using a serial mdrun. You should check with ldd whether
mdrun_mpi is linked to the correct mpi library.
Carsten
>
> I do appreciate any help in that domain
> Hassan
>
> From: gmx-users-bounces at gromacs.org on behalf of Justin A. Lemkul
> Sent: Mon 12/6/2010 6:43 PM
> To: Discussion list for GROMACS users
> Subject: Re: [gmx-users] pressure fluctuations
>
>
>
> Hassan Shallal wrote:
> > Dear Gromacs users,
> >
> > I have some concerns about the both the pressure fluctuations and
> > averages I obtained during the equilibration phase. I have already read
> > through several similar posts as well as the following link
> > http://www.gromacs.org/Documentation/Terminology/Pressure. I understand
> > the pressure is a macroscopic rather than instantaneous property and the
> > average is what really matters. I also found out through similar posts
> > that negative average pressure indicates the system tendency to contract.
> >
> > In the above link, it mentioned that pressure fluctuations should
> > decrease significantly with increasing the system's size. In my cases, I
> > have a fairly big systems (case_1 with *17393* water molecules
> > and case_2 with *11946 *water molecules). However, the pressure still
> > has huge fluctuations (around 500 bars) from the reference value (1
> > bar). Here are the average pressure and density values resulting from
> > the equilibration phases of two cases, please notice the negative
> > average pressure values in both cases...
> >
> > Case_1_pressure:
> > Energy Average Err.Est. RMSD Tot-Drift
> > -------------------------------------------------------------------------------
> > Pressure *-2.48342* 0.92 369.709 -4.89668
> > (bar)
> > Case_1_density:
> > Energy Average Err.Est. RMSD Tot-Drift
> > -------------------------------------------------------------------------------
> > Density 1022.89 0.38 3.8253 2.36724
> > (kg/m^3)
> > Case_2_pressure:
> > Energy Average Err.Est. RMSD Tot-Drift
> > -------------------------------------------------------------------------------
> > Pressure *-8.25259* 2.6 423.681 -12.1722
> > (bar)
> > Case_2_density:
> > Energy Average Err.Est. RMSD Tot-Drift
> > -------------------------------------------------------------------------------
> > Density 1034.11 0.37 2.49964 1.35551
> > (kg/m^3)
> >
> > So I have some questions to address my concerns:
> > 1- each of the above systems has a protein molecule, NaCl to give 0.15 M
> > system and solvent (water) molecules... Could that tendency to contract
> > be an artifact of buffering the system with sodium and chloride ions?
> >
>
> I suppose anything is possible, but given that these are fairly standard
> conditions for most simulations, I tend to doubt it. My own (similar) systems
> do not show this problem.
>
> > 2- how to deal with the tendency of my system to contract? Should
> > I change the number of water molecules in the system?
> > or
> > Is it possible to improve the average pressure of the above systems by
> > increasing the time of equilibration from 100 ps to may be 500 ps or
> > even 1 ns?
> >
> > 3- Is there a widely used range of average pressure (for ref_p = 1 bar)
> > that indicates acceptable equilibration of the system prior to the
> > production?
> >
>
> To answer #2 and #3 simultaneously - equilibration is considered "finished" when
> your system stabilizes at the appropriate conditions (usually temperature and
> pressure). Your results indicate that your equilibrium is insufficient.
>
> > 4- I can't understand how the system has a tendency to contract whereas
> > the average density of the solvent is already slightly higher than it
> > should be (1000 kg/m^3).
>
> The contraction causes the density to rise. Pressure and density are not
> independent; density is a result of pressure.
>
> > I would like to ignore the pressure based judgement of the above
> > equilibration given that the average density values are very close to
> > the natural value (1000 kg/m^3) (by the way I am using tip3p water model
> > with CHARMM27 ff) Any comment!!
> >
>
> It is not guaranteed that simulation water models will reproduce the real
> (experimental) density of water. If memory serves, the expected density of
> TIP3P should be ~0.98 g mL^{-1}, but I could be wrong.
>
> > 5- Is the huge fluctuation of the pressure values of the above system
> > despite thier large sizes still acceptable? or large fluctuation is only
> > acceptable for small size systems and is unacceptable for large size
> > systems?
> > If it is unacceptable, any idea of how could it be alleviated or minimized?
> >
>
> The fluctuations seem reasonable. You might try equilibrating with the
> Berendsen barostat first, then switching to Parrinello-Rahman. The P-R barostat
> allows for wider fluctuations, so if the system is poorly equilibrated, your
> system will be slower to converge.
>
> -Justin
>
> > I am including the .mdp used in the above equilibration in case it is
> > needed.
> >
> > Any feedback or response to the above questions is so much appreciated..
> >
> > Great regards
> > Hassan
> >
> > .mdp used for the above equilibration
> > define = -DPOSRES ; position restrain the protein
> > ; Run parameters
> > integrator = md ; leap-frog integrator
> > nsteps = 25000 ; 4 * 25000 = 100 ps
> > dt = 0.004 ; 4 fs, virtual sites along with heavy hydrogens are used
> > ; Output control
> > nstxout = 100 ; save coordinates every 0.2 ps
> > nstvout = 100 ; save velocities every 0.2 ps
> > nstenergy = 100 ; save energies every 0.2 ps
> > nstlog = 100 ; update log file every 0.2 ps
> > ; Bond parameters
> > continuation = yes ; Restarting after NVT
> > constraint_algorithm = lincs ; holonomic constraints
> > constraints = all-bonds ; all bonds (even heavy atom-H bonds) constrained
> > lincs_iter = 1 ; accuracy of LINCS
> > lincs_order = 6 ; also related to accuracy, changed to 6 because of
> > using virtual sites along with a larger time step
> > ; Neighborsearching
> > ns_type = grid ; search neighboring grid cells
> > nstlist = 5 ; 20 fs
> > rlist = 1.2 ; short-range neighborlist cutoff, equal to rcoulomb to
> > allow for PME electrostatics (in nm)
> > ; Lennard-Jones
> > vdwtype = switch ; VDW interactions are switched of
> > between 1 and 1.2
> > rvdw_switch = 1 ;
> > rvdw = 1.2 ; short-range vdw cutoff, optimal for
> > CHARMM27 ff (in nm)
> > ; Electrostatics
> > coulombtype = PME ; Particle Mesh Ewald for long-range electrostatics
> > pme_order = 4 ; cubic interpolation
> > rcoulomb = 1.2 ; short-range electrostatic cutoff, optimal for CHARMM27
> > ff (in nm)
> > ; Temperature coupling is on
> > tcoupl = V-rescale ; modified Berendsen thermostat
> > tc-grps = Protein Non-Protein ; two coupling groups - more accurate
> > tau_t = 0.1 0.1 ; time constant, in ps
> > ref_t = 300 300 ; reference temperature, one for each group, in K
> > ; Pressure coupling is on
> > pcoupl = Parrinello-Rahman ; Pressure coupling on in NPT
> > pcoupltype = isotropic ; uniform scaling of box vectors
> > tau_p = 1 ; in ps
> > ref_p = 1.0 ; reference pressure, in bar
> > compressibility = 4.5e-5 ; isothermal compressibility of water, bar^-1
> > ; Periodic boundary conditions
> > pbc = xyz ; 3-D PBC
> > ; Dispersion correction
> > DispCorr = EnerPres ; account for switch vdW scheme
> > ; Velocity generation
> > gen_vel = no ; Velocity generation is off
> >
> >
>
> --
> ========================================
>
> Justin A. Lemkul
> Ph.D. Candidate
> ICTAS Doctoral Scholar
> MILES-IGERT Trainee
> Department of Biochemistry
> Virginia Tech
> Blacksburg, VA
> jalemkul[at]vt.edu | (540) 231-9080
> http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin
>
> ========================================
> --
> gmx-users mailing list gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> --
> gmx-users mailing list gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing list gmx-users at gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-request at gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 15166 bytes
Desc: not available
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-users/attachments/20101210/942df822/attachment.bin>
More information about the gromacs.org_gmx-users
mailing list