[gmx-users] Gromacs 2019.2 on Power9 + Volta GPUs (building and running)

[Alex]

Well, my experience so far has been with the EM, because the rest of the
script (with all the dynamic things) needed that to finish. And it
"finished" by hitting the wall. However, your comment does touch upon what
to do with thread pinning and I will try to set '-pin on' throughout to see
if things make a difference for the better. I am less confident about
setting strides because it is unclear what the job manager provides in
terms of the available core numbers. I will play around some more and
report here.

Thanks!

Alex

On Wed, May 1, 2019 at 3:49 PM Mark Abraham <mark.j.abraham at gmail.com>
wrote:

> Hi,
>
> As with x86, GROMACS uses SIMD intrinsics on POWER9 and is thus fairly
> insensitive to the compiler's vectorisation abilities. GCC is the only
> compiler we've tested, as xlc can't compile simple C++11. As everywhere,
> you should use the latest version of gcc, as IBM spent quite some years
> landing improvements for POWER9.
>
> EM is useless as a performance indicator of a dynamical simulation, avoid
> that - it runs serial code much much more often.
>
> Your run deliberately didn't fill the available cores, so just like on x86,
> mdrun will leave the thread affinity handling to the environment, which is
> often a path to bad performance. So, if you plan on doing that often,
> you'll want to check out the mdrun performance guide docs about the mdrun
> -pin and related options.
>
> Mark
>
>
> On Wed., 1 May 2019, 23:21 Alex, <nedomacho at gmail.com> wrote:
>
> > Hi all,
> >
> > Our institution decided to be all fancy, so now we have a bunch of Power9
> > nodes, each with 80 cores + 4 Volta GPUs. Stuff is managed by slurm.
> Today
> > I did a simple EM ('gmx mdrun -ntomp 4 -ntmpi 4 -pme cpu -nb cpu') and
> the
> > performance is abysmal, I would guess 100 times slower than on anything
> > I've ever seen before.
> >
> > Our admin person emailed me the following:
> > "-- it would not surprise me if the GCC compilers were relatively bad at
> > taking advantage of POWER9 vectorization, they're likely optimized for
> > x86_64 vector stuff like SSE and AVX operations.  This was an issue in
> the
> > build, I selected "-DGMX_SIMD=IBM_VSX" for the config, but according to
> my
> > notes, that was part of an attempt to fix the "unimplemented SIMD" error
> > that was dogging me at first, and/but which was eventually cleared by
> > switching to gcc-6."
> >
> > Does anyone have any comments/suggestions on building and running GMX on
> > Power9?
> >
> > Thank you,
> >
> > Alex
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> > send a mail to gmx-users-request at gromacs.org.
> >
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>