[gmx-users] Looking for the source of the H-REMD slowdown when decoupling a lot of atoms

[Mark Abraham]

Hi,

Yeah, unfortunately Michael's pretty much right - the free-energy kernel is
currently that cousin nobody talks about (or to). It's essentially
unchanged since GROMACS 4.0 days, except that the Verlet scheme has some
kludge so that it can call the same kernel that the group scheme used to
call. But it has none of the optimizations and also some simplifying
pessimizations. The result is highly unsuitable for Chris's use case, but
not horrible for the more normal case of perturbing a small part of a
system. For now, I can only suggest trying a run with more than one OpenMP
thread per rank, but there's nothing in the log file snippets that fills me
with any hope that it would be noticeably faster.

We have a pile of infrastructure built and in the works that will lead to
being able to offer similarly optimized free-energy kernels, but they won't
see the light of day until next year, I'm afraid. A set of sample .tprs at
Redmine 742 would be most welcome, however - it's very good for us to know
when/that we're optimizing a workflow someone actually wants to run, but
currently has reason to avoid.

Mark

On Wed, Mar 30, 2016 at 8:05 AM Michael Shirts <mrshirts at gmail.com> wrote:

> Hi, Chris: I'm pretty sure that it's because the nonbonded free
> energies are much slower than the standard free energies.  You state:
>
> > I took a look at gmxlib/nonbonded/nb_free_energy.c in v.5.1.2, but I was
> unable to find a function called "gmx_waste_time_here()" and beyond that I
> was out of my depth.
>
> But it's much more the fact that the non-free energy nonbondeds are
> SUPER optimized.
>
> I don't see a particularly viable way around this for now. The only
> thing I can think of splitting the neighborlists into two force calls,
> and scaling the forces and energies coming out of those.  That would
> be a huge pain.
>
> On Tue, Mar 29, 2016 at 9:50 PM, Christopher Neale
> <chris.neale at alum.utoronto.ca> wrote:
> > Dear Users:
> >
> > I am trying to do some Hamiltonian replica exchange (H-REMD) in gromacs
> 5.1.2 and am running up against really large slowdowns when decoupling a
> large number of atoms. I am decoupling 5360 atoms out of the 15520 atoms in
> my system. The goal is not to get a PMF, but to enhance sampling using the
> REST approach to partially decouple lipids in a bilayer. This approach
> enhances lipid relaxation times (
> http://pubs.acs.org/doi/pdf/10.1021/ct500305u ) though the authors of
> that paper modified the gromacs code to do their own H-REMD in order to
> avoid the really slow speed they also got when decoupling lots of atoms via
> the free energy code.
> >
> > I have already posted details here http://redmine.gromacs.org/issues/742
> , which includes .mdp options and some timing output. I compare the timing
> output to a standard temperature REMD (T-REMD) run. For my usage, the
> slowdown is about 12x for H-REMD vs. T-REMD.
> >
> > I am motivated to find a solution within gromacs because the alternative
> is to use gromacs 4.6.7 with plumed (or with the aforementioned modified
> code, which is also gromacs v4). Normally that would be a viable option,
> but I am using the charmm force field and the charmm TIP3P water and I
> would rather not give up the speed boost that I see in gromacs v5.1.2,
> which allows the use of the verlet cutoff scheme and has been tested and
> shown to give the correct reproduction of charmm forces (vs. the forces one
> would get using the charmm simulation software).
> >
> > I took a look at gmxlib/nonbonded/nb_free_energy.c in v.5.1.2, but I was
> unable to find a function called "gmx_waste_time_here()" and beyond that I
> was out of my depth.
> >
> > Thank you for any pointers,
> > Chris.
> >
> >
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>