[gmx-users] GROMACS stalls for NPT simulation when using -npme and -dd flags
Stephen Cox
stephen.cox.10 at ucl.ac.uk
Tue Mar 13 10:18:54 CET 2012
My Gmail doesn't show the subject line when hitting reply, so apologies for
the double post, but I thought I'd put it out there with an amended subject
line in case it's of use to anyone else....
Hi,
so further to my last email, I have run a quick test on my desktop machine
(4 cores, 12Gb RAM). It seems that when running the parrinello-rahman
barostat with domain decomposition (-dd 2 2 1) that I'm still getting
memory leak (this time with GNU compilers). I followed proper equilibration
with the berendsen barostat (although I didn't think this was the problem)
and also added the "constraints = all-bonds" line, but this has made no
difference to my results (I'm using "[ constraints ]" in my topology file).
To give an idea of the rate of memory loss, initial memory consumption was
0.1% total memory per process, which rose steadily to 0.5% total memory
after 5 minutes. After 17mins, memory consumption is 1.7% total memory and
rising. Running with "-pd", memory usage is constant at 0.1% total memory.
The system is 328 TIP4P/ice + 64 all-atomic methane. This problem has
occurred for me on different architectures and with different compilers
(and different system sizes). It would be good if anybody familiar with the
source could take a look, or if anybody knows any compiler flags that would
prevent memory leak.
Thanks,
Steve
On 9 March 2012 13:32, Stephen Cox <stephen.cox.10 at ucl.ac.uk> wrote:
>
>
> On 9 March 2012 13:03, gmx-users-request at gromacs.org <
> gmx-users-request at gromacs.org> wrote:
>
>> Send gmx-users mailing list submissions to
>> gmx-users at gromacs.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> or, via email, send a message with subject or body 'help' to
>> gmx-users-request at gromacs.org
>>
>> You can reach the person managing the list at
>> gmx-users-owner at gromacs.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of gmx-users digest..."
>>
>>
>> Today's Topics:
>>
>> 1. Re: GROMACS stalls for NPT simulation when using -npme and
>> -dd flags (Mark Abraham)
>> 2. Error Message in Make clean command for installation. (a a)
>> 3. Re: Error Message in Make clean command for installation.
>> (Mark Abraham)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Fri, 09 Mar 2012 23:42:33 +1100
>> From: Mark Abraham <Mark.Abraham at anu.edu.au>
>> Subject: Re: [gmx-users] GROMACS stalls for NPT simulation when using
>> -npme and -dd flags
>> To: Discussion list for GROMACS users <gmx-users at gromacs.org>
>> Message-ID: <4F59FAB9.6010805 at anu.edu.au>
>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>
>> On 9/03/2012 9:43 PM, Stephen Cox wrote:
>> > Dear users,
>> >
>> > I'm trying to run an isotropic NPT simulation on a cubic cell
>> > containing TIP4P/ice water and methane. I'm using the
>> > Parrinello-Rahman barostat. I've been playing around with the
>> > different decomposition flags of mdrun to get better performance and
>> > scaling and have found that the standard -npme (half number of cores)
>> > works pretty well. I've also tried using the -dd flags, and I appear
>> > to get decent performance and scaling. However, after a few
>> > nanoseconds (corresponding to about 3 hours run time), the program
>> > just stalls; no output and no error messages. I realise NPT may cause
>> > domain decompositon some issues if the cell vectors vary wildly, but
>> > this isn't happening in my system.
>> >
>> > Has anybody else experienced issues with domain decomposition and NPT
>> > simulations? If so, are there any workarounds? For the moment, I've
>> > had to resort to using -pd, which is giving relatively poor
>> > performance and scaling, but at least it isn't dying!
>> >
>> > I'm using GROMACS 4.5.5 with an intel compiler (I followed the
>> > instructions online, with static linking) and using the command:
>> >
>> > #!/bin/bash -f
>> > # ---------------------------
>> > #$ -V
>> > #$ -N test
>> > #$ -S /bin/bash
>> > #$ -cwd
>> > #$ -l vf=2G
>> > #$ -pe ib-ompi 32
>> > #$ -q infiniband.q
>> >
>> > mpirun mdrun_mpi -cpnum -cpt 60 -npme 16 -dd 4 2 2
>> >
>> > Below is my grompp.mdp.
>> >
>> > Thanks,
>> > Steve
>> >
>> > P.S. I think that there may be an issue with memory leak that occurs
>> > for domain decomposition with NPT. I seem to remember seeing this
>> > happening on my desktop and my local cluster. I don't see this with
>> > NVT simulations. This would be consistent with the lack of error
>> > message: I've just run a short test run and the memory usage was
>> > climbing streadily.
>> >
>> > ; run control
>> > integrator = md
>> > dt = 0.002
>> > nsteps = -1
>> > comm_mode = linear
>> > nstcomm = 10
>> >
>> > ; energy minimization
>> > emtol = 0.01
>> > emstep = 0.01
>> >
>> > ; output control
>> > nstxout = 0
>> > nstvout = 0
>> > nstfout = 0
>> > nstlog = 0
>> > nstcalcenergy = 2500
>> > nstenergy = 2500
>> > nstxtcout = 2500
>> >
>> > ; neighbour searching
>> > nstlist = 1
>> > ns_type = grid
>> > pbc = xyz
>> > periodic_molecules = no
>> > rlist = 0.90
>> >
>> > ; electrostatics
>> > coulombtype = pme
>> > rcoulomb = 0.90
>> >
>> > ; vdw
>> > vdwtype = cut-off
>> > rvdw = 0.90
>> > dispcorr = ener
>> >
>> > ; ewald
>> > fourierspacing = 0.1
>> > pme_order = 4
>> > ewald_geometry = 3d
>> > optimize_fft = yes
>> >
>> > ; temperature coupling
>> > tcoupl = nose-hoover
>> > nh-chain-length = 10
>> > tau_t = 2.0
>> > ref_t = 255.0
>> > tc_grps = system
>> >
>> > ; pressure coupling
>> > pcoupl = parrinello-rahman
>> > pcoupltype = isotropic
>> > ref_p = 400.0
>> > tau_p = 2.0
>> > compressibility = 6.5e-5
>> >
>> > ; constraints
>> > constraint_algorithm = shake
>> > shake_tol = 0.0001
>> > lincs_order = 8
>> > lincs_iter = 2
>> >
>> > ; velocity generation
>> > gen_vel = yes
>> > gen_temp = 255.0
>> > gen_seed = -1
>>
>> You're generating velocities and immediately using a barostat that is
>> unsuitable for equilibration.
>
>
> Sorry, this is unclear: the system has already been equilibrated using NVT
> then berendsen before using the parrinello-rahman barostat. I use the
> generate velocities options to give me uncorrelated trajectories (I'm
> investigating a stochastic process and want statistics). I appreciate the
> concerns about poor equilibration, but I'm pretty sure this isn't the case
> - in my experience, poor equilibration usually results in a fairly prompt
> crash, with a lot of error messages and step.pdb files. Furthermore, the
> cell volume seems stable over the time that I manage to simulate,
> suggesting equilibration is OK.
>
>
>> You're using an integration step that
>> requires constraints=all-bonds but I don't see that.
>
>
> Could you clarify this point for me please? Thanks.
>
>
>> You may have better
>> stability if you equilibrate with Berendsen barostat and then switch.
>> I've seen no other reports of memory usage growing without bounds, but
>> if you can observe it happening after choosing a better integration
>> regime then it suggests a code problem that wants fixing.
>>
>
> I can run some tests on my desktop with a smaller system and report if I
> see memory loss.
>
>
>>
>> Mark
>>
>>
>>
> Thanks for your quick response
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-users/attachments/20120313/baa5282d/attachment.html>
More information about the gromacs.org_gmx-users
mailing list