[gmx-users] Different energies of GPU and CPU-only runs with Gromacs 5.x

[Johnny Lu]

hi. can i get a link to the redmine issue?
I decided to install 4.6.7 instead of 5.0.1 because of this.

On Mon, Sep 22, 2014 at 5:52 AM, Mark Abraham <mark.j.abraham at gmail.com>
wrote:

> On Mon, Sep 22, 2014 at 11:02 AM, Machtens, Jan-Philipp <
> j.machtens at fz-juelich.de> wrote:
>
> > Thanks for the reply!
> > However, the differences between GPU runs and CPU-only runs for
> > Gromacs 5.x that I observe are statistically significant (10-100 ns for
> > each test sim) (and reproducible).
> > In contrast, with Gromacs 4.6.x I do not have significant differences
> that
> > I can apparently attribute to the use of GPUs.
> >
>
> OK. I would expect to observe a *total* energy that was constant across 4.6
> and 5 and CPU-only vs with-GPUs. Please open an issue at
> http://redmine.gromacs.org/ and describe as much detail as you can. A
> tarball with .tpr, grompp-input, and whole .log files would let us analyze
> and reproduce your observations and work out what is going on. Thanks!
>
> Is there any other reason for the deviation with GPU and GROMAC 5.x ?
>
>
> Not by design :-) All the relevant changes that I can think of either
> removed code paths for old hardware, or tweaked performance (in various
> kinds of ways). There were a bunch of fixes introduced into 4.6.6 and 5.0
> that could be relevant, but we need the gory details to know.
>
> Mark
>
> Best,
> > JP
> > ________________________________________
> > From: gromacs.org_gmx-users-bounces at maillist.sys.kth.se [
> > gromacs.org_gmx-users-bounces at maillist.sys.kth.se] on behalf of Mark
> > Abraham [mark.j.abraham at gmail.com]
> > Sent: Monday, September 22, 2014 10:25 AM
> > To: Discussion list for GROMACS users
> > Subject: Re: [gmx-users] Different energies of GPU and CPU-only runs with
> > Gromacs 5.x
> >
> > On Mon, Sep 22, 2014 at 9:44 AM, Machtens, Jan-Philipp <
> > j.machtens at fz-juelich.de> wrote:
> >
> > > Dear all,
> > > if I compare GPU-accelerated and CPU-only runs, I get significantly
> > > different results/energies with
> > > Gromacs 5.x. In contrast, with Gromacs 4.x everything is fine, i.e. no
> > > difference between CPU-only runs and with GPU.
> > >
> > > In short, with Gromacs 5.x using GPUs, I have lower total energy,
> higher
> > > short-range LJ, higher short-range Coulomb and
> > > lower reciprocal Coulomb values.
> >
> >
> > This is a normal consequence of different PME load balancing between CPU
> > and GPU. You can use the UNIX diff tool to observe that mdrun is making
> > different decisions.
> >
> > This also leads to altered density and box dimensions, etc....
> > >
> >
> > MD trajectories are generally not reproducible even if the PME load is
> > balanced the same way. Do replicates show the same kind of variation?
> >
> > Mark
> >
> > The simulations were run on a single node with either 2x GeForce GTX 780
> Ti
> > > or with 1x GTX 770.
> > > Compilers tested: gcc 4.8 and icc 14
> > > Cuda Versions: 6.0 or 6.5
> > > ForceField: Amber99sb-ildn
> > >
> > > The topol.tpr was identical for each simulation. Verlet cut-off scheme
> > was
> > > used.
> > > The mdp file can be accessed here:
> > > https://www.dropbox.com/s/kjkglu2p8p11ajn/parameter.mdp?dl=0
> > > Regression tests of the various compilations were always passed.
> > >
> > > Does someone has the some problems or has a clue what might cause the
> > > problems I have with Gromacs5.x when using GPUs ?
> > > Many thanks!
> > > Best,
> > > Jan-Philipp Machtens
> > > Postdoctoral Research Fellow
> > > Institute of Complex Systems - Cellular Biophysics (ICS-4)
> > > Forschungszentrum Jülich, Germany
> > >
> > >
> > >
> > >
> >
> ------------------------------------------------------------------------------------------------
> > >
> > >
> >
> ------------------------------------------------------------------------------------------------
> > > Forschungszentrum Juelich GmbH
> > > 52425 Juelich
> > > Sitz der Gesellschaft: Juelich
> > > Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> > > Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> > > Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> > > Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> > > Prof. Dr. Sebastian M. Schmidt
> > >
> > >
> >
> ------------------------------------------------------------------------------------------------
> > >
> > >
> >
> ------------------------------------------------------------------------------------------------
> > >
> > > --
> > > Gromacs Users mailing list
> > >
> > > * Please search the archive at
> > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > > posting!
> > >
> > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> > >
> > > * For (un)subscribe requests visit
> > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> > > send a mail to gmx-users-request at gromacs.org.
> > >
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> > send a mail to gmx-users-request at gromacs.org.
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> > send a mail to gmx-users-request at gromacs.org.
> >
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>