[gmx-users] Performance issues with Gromacs 2020 on GPUs - slower than 2019.5

[Szilárd Páll]

Hi

On Thu, Feb 27, 2020 at 11:31 AM Andreas Baer <andreas.baer at fau.de> wrote:

> Hi,
>
> with the link below, additional log files for runs with 1 GPU should be
> accessible now.
>

I meant to ask you to run single-rank GPU runs, i.e. gmx mdrun -ntmpi 1.

It would also help if you could share some input files in case if further
testing is needed.


> Thank you for the comment with the rlist, I did not know, that this will
> affect the performance negatively.


It does in multiple ways. First, you are using a rather long list buffer
which will make the nonbonded pair-interaction calculation more
computational expensive than it could be if you just used a tolerance and
let the buffer be calculated. Secondly, as setting a manual rlist disables
the automated verlet buffer calculation, it prevents mdrun from using a
dual pairl-list setup (see
http://manual.gromacs.org/documentation/2018.1/release-notes/2018/major/features.html#dual-pair-list-buffer-with-dynamic-pruning)
which has additional performance benefits.

Cheers,
--
Szilárd



> I know, about the nstcalcenergy, but
> I need it for several of my simulations.

Cheers,
> Andreas
>
> On 26.02.20 16:50, Szilárd Páll wrote:
> > Hi,
> >
> > Can you please check the performance when running on a single GPU 2019 vs
> > 2020 with your inputs?
> >
> > Also note that you are using some peculiar settings that will have an
> > adverse effect on performance (like manually set rlist disallowing the
> dual
> > pair-list setup, and nstcalcenergy=1).
> >
> > Cheers,
> >
> > --
> > Szilárd
> >
> >
> > On Wed, Feb 26, 2020 at 4:11 PM Andreas Baer <andreas.baer at fau.de>
> wrote:
> >
> >> Hello,
> >>
> >> here is a link to the logfiles.
> >>
> >>
> https://faubox.rrze.uni-erlangen.de/getlink/fiX8wP1LwSBkHRoykw6ksjqY/benchmarks_2019-2020
> >>
> >> If necessary, I can also provide some more log or tpr/gro/... files.
> >>
> >> Cheers,
> >> Andreas
> >>
> >>
> >> On 26.02.20 16:09, Paul bauer wrote:
> >>> Hello,
> >>>
> >>> you can't add attachments to the list, please upload the files
> >>> somewhere to share them.
> >>> This might be quite important to us, because the performance
> >>> regression is not expected by us.
> >>>
> >>> Cheers
> >>>
> >>> Paul
> >>>
> >>> On 26/02/2020 15:54, Andreas Baer wrote:
> >>>> Hello,
> >>>>
> >>>> from a set of benchmark tests with large systems using Gromacs
> >>>> versions 2019.5 and 2020, I obtained some unexpected results:
> >>>> With the same set of parameters and the 2020 version, I obtain a
> >>>> performance that is about 2/3 of the 2019.5 version. Interestingly,
> >>>> according to nvidia-smi, the GPU usage is about 20% higher for the
> >>>> 2020 version.
> >>>> Also from the log files it seems, that the 2020 version does the
> >>>> computations more efficiently, but spends so much more time waiting,
> >>>> that the overall performance drops.
> >>>>
> >>>> Some background info on the benchmarks:
> >>>> - System contains about 2.1 million atoms.
> >>>> - Hardware: 2x Intel Xeon Gold 6134 („Skylake“) @3.2 GHz = 16 cores +
> >>>> SMT; 4x NVIDIA Tesla V100
> >>>>    (similar results with less significant performance drop (~15%) on a
> >>>> different machine: 2 or 4 nodes with each [2x Intel Xeon 2660v2 („Ivy
> >>>> Bridge“) @ 2.2GHz = 20 cores + SMT; 2x NVIDIA Kepler K20])
> >>>> - Several options for -ntmpi, -ntomp, -bonded, -pme are used to find
> >>>> the optimal set. However the performance drop seems to be persistent
> >>>> for all such options.
> >>>>
> >>>> Two representative log files are attached.
> >>>> Does anyone have an idea, where this drop comes from, and how to
> >>>> choose the parameters for the 2020 version to circumvent this?
> >>>>
> >>>> Regards,
> >>>> Andreas
> >>>>
> >> --
> >> Gromacs Users mailing list
> >>
> >> * Please search the archive at
> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> >> posting!
> >>
> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>
> >> * For (un)subscribe requests visit
> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> >> send a mail to gmx-users-request at gromacs.org.
>
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.