[gmx-users] REMD slow's down drastically
Mark Abraham
mark.j.abraham at gmail.com
Tue Feb 25 09:25:45 CET 2014
Goods to know, thanks. Mileage certainly does vary.
Mark
On Feb 25, 2014 3:45 AM, "Christopher Neale" <chris.neale at alum.utoronto.ca>
wrote:
> Thank you Mark for the tips about the pinoffset. I'll try it and see if it
> affects the speed at all.
>
> Regarding the utility of hyperthreading, running on a cluster in which
> each node has 8 Nehalem processing cores, I have seen 5-15% speedup from
> using hyperthreading via 16 threads vs. using only 8 threads (in non-MPI
> gromacs). This is across about 10 simulations systems that I have worked on
> in the last four years. In all these cases, I am using -npme 0. However,
> once multiple nodes and IB fabric get involved, then hyperthreading gives
> no benefit and generally degrades the performance. Perhaps there are some
> other things getting involved here, but the only change I make is mdrun -nt
> 8 or mdrun -nt 16 and I see a speedup from -nt 16. System sizes range from
> 10K to 250K atoms. Note that I have never tried using the hyperthreading
> with REMD or any other fancy setup.
>
> Chris.
> ________________________________________
> From: gromacs.org_gmx-users-bounces at maillist.sys.kth.se <
> gromacs.org_gmx-users-bounces at maillist.sys.kth.se> on behalf of Mark
> Abraham <mark.j.abraham at gmail.com>
> Sent: 24 February 2014 21:26
> To: Discussion list for GROMACS users
> Subject: Re: [gmx-users] REMD slow's down drastically
>
> On Feb 24, 2014 11:01 PM, "Christopher Neale" <
> chris.neale at alum.utoronto.ca>
> wrote:
> >
> > Presuming that you have indeed set up the number of processors correctly
> (should be running on a different number of cored for different number of
> replicas to do a fair test), could it be a thread pinning issue?
>
> Yes, but part of the larger problem of over-loading the physical cores.
>
> > I run on a Nehalem system with 8 cores/node but, because of the Nehalem
> hyperthreading (I think), gromacs always complains if I run "mpirun -np $N
> mdrun" where $N is the number of cores
> >
> > NOTE: The number of threads is not equal to the number of (logical) cores
> > and the -pin option is set to auto: will not pin thread to cores.
> > This can lead to significant performance degradation.
> > Consider using -pin on (and -pinoffset in case you run multiple
> jobs).
> >
> > However, if I use $N = 2 times the number of cores, then I don't get that
> note, instead getting:
> >
> > "Pinning threads with a logical core stride of 1"
> >
> > Aside, if anybody has a suggestion about how I should handle the thread
> pinning in my case, or if it matters, then I would be happy to hear it (my
> throughput seems to be good though).
>
> Hyper-threading is good for applications that are memory- or user-bound (so
> enabled by default on consumer machines), so they can take advantage of CPU
> instruction-issue opportunities while stalled. GROMACS kernels are already
> CPU-bound, so there is little to gain and it generally does not pay for the
> overhead. Generally, one should not use HT; turning it off can be emulated
> with the right use of -pinoffset and using half the number of threads.
>
> > Finally, this comment is off topic, but you might want to reconsider
> having the CL ions in a separate temperature coupling group.
>
> Indeed.
>
> Mark
>
> > Chris.
> > ________________________________________
> > From: gromacs.org_gmx-users-bounces at maillist.sys.kth.se <
> gromacs.org_gmx-users-bounces at maillist.sys.kth.se> on behalf of Singam
> Karthick <sikart21 at yahoo.in>
> > Sent: 24 February 2014 02:32
> > To: gromacs.org_gmx-users at maillist.sys.kth.se
> > Subject: [gmx-users] REMD slow's down drastically
> >
> > Dear members,
> > I am trying to run REMD simulation for poly Alanine (12 residue) system.
> I used remd generator to get the range of temperature with the exchange
> probability of 0.3. I was getting the 125 replicas. I tried to simulate 125
> replicas its drastically slow down the simulation time (for 70 pico seconds
> it took around 17 hours ) could anyone please tell me how to solve this
> issue.
> >
> > Following is the MDP file
> >
> > title = G4Ga3a4a5 production.
> > ;define = ;-DPOSRES ; position restrain the protein
> > ; Run parameters
> > integrator = md ; leap-frog integrator
> > nsteps = 12500000 ; 2 * 5000000 = 3ns
> > dt = 0.002 ; 2 fs
> > ; Output control
> > nstxout = 0 ; save coordinates every 0.2 ps
> > nstvout = 10000 ; save velocities every 0.2 ps
> > nstxtcout = 500 ; save xtc coordinate every 0.2 ps
> > nstenergy = 500 ; save energies every 0.2 ps
> > nstlog = 100 ; update log file every 0.2 ps
> > ; Bond parameters
> > continuation = yes ; Restarting after NVT
> > constraint_algorithm = lincs ; holonomic constraints
> > constraints = hbonds ; all bonds (even heavy atom-H bonds)
> constrained
> > lincs_iter = 1 ; accuracy of LINCS
> > lincs_order = 4 ; also related to accuracy
> > morse = no
> > ; Neighborsearching
> > ns_type = grid ; search neighboring grid cels
> > nstlist = 5 ; 10 fs
> > rlist = 1.0 ; short-range neighborlist cutoff (in nm)
> > rcoulomb = 1.0 ; short-range electrostatic cutoff (in
> nm)
> > rvdw = 1.0 ; short-range van der Waals cutoff (in
> nm)
> > ; Electrostatics
> > coulombtype = PME ; Particle Mesh Ewald for long-range
> electrostatics
> > pme_order = 4 ; cubic interpolation
> > fourierspacing = 0.16 ; grid spacing for FFT
> > ; Temperature coupling is on
> > tcoupl = V-rescale ; modified Berendsen thermostat
> > tc-grps = protein SOL Cl ;two coupling groups - more
> accurate
> > tau_t = 0.1 0.1 0.1 ; time constant, in ps
> > ref_t = XXXXX XXXXX XXXXX ; reference temperature,
> one for each group, in K
> > ; Pressure coupling is on
> > pcoupl = Parrinello-Rahman ; Pressure coupling on in NPT
> > pcoupltype = isotropic ; uniform scaling of box vectors
> > tau_p = 2.0 ; time constant, in ps
> > ref_p = 1.0 ; reference pressure, in bar
> > compressibility = 4.5e-5 ; isothermal compressibility of water,
> bar^-1
> > ; Periodic boundary conditions
> > pbc = xyz ; 3-D PBC
> > ; Dispersion correction
> >
> > DispCorr = EnerPres ; account for cut-off vdW scheme
> >
> >
> > regards
> > singam
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>
More information about the gromacs.org_gmx-users
mailing list