[gmx-users] GPU acceleration

Alex nedomacho at gmail.com
Fri Jun 5 23:48:38 CEST 2015


Thanks for the info. I tried with GMX_EMULATE_GPU=1 preceding the mdrun
line, crashed after 100 steps. Is that the expected behavior?

On Fri, Jun 5, 2015 at 2:46 PM, Szilárd Páll <pall.szilard at gmail.com> wrote:

> On Fri, Jun 5, 2015 at 10:39 PM, Alex <nedomacho at gmail.com> wrote:
> > In that case, I am a bit lost, as I was sure GMXRC needed modification.
> > :)). Since I never ran anything on GPU and have an mdrun line that looks
> > like
> >
> > mdrun -nt 4 -s <tpr> -c <final structure>
>
> gmx mdrun -ntomp 4
>
> No need to do domain decomposition. Even with CPU runs it'll be faster
> if you don't.
>
> > what should be added to trigger the behavior you're suggesting? I
> > understand that it will likely blow up the system, but for the heck of
> > it...
>
> those environment variables I mentioned. Plus timer resetting, e.g.
> with -resethway if you want reliable timings.
>
> > I am now looking at the stats and see that (NxN QSTab + LJ) and (NxN
> QSTab
> > [F]) take up about 86% of all flops in my simulation. Which of those get
> > offloaded?
>
> All of that will be offloaded. However, those are theoretical (useful)
> flops, the timing table on the bottom is more relevant. Set the
> GMX_CYCLE_SUBCOUNTER=ON breakdown of the "Force".
>
> However, if you just follow the previous emulate+skip non-bondeds
> advise, you'll get a run that does the offload except that it skips
> calculating - which is useful because you have no GPU to calculate
> with.
>
> > Thank you,
> >
> > Alex
> >
> >
> > On Fri, Jun 5, 2015 at 2:26 PM, Szilárd Páll <pall.szilard at gmail.com>
> wrote:
> >
> >> Hi Alex,
> >>
> >> Why do you think GMXRC needs to be changed? You simply need to set
> >> those env. vars to trigger the GPU code-path - and you will likely
> >> have to freeze the system, otherwise it will probably not run more
> >> than a few steps without seeing an explosion. Based on this you'll
> >> know how fast you can run with GPU offload, but you can also
> >> guesstimate how fast of a GPU you need (based on the PME+Bonded force
> >> calculation time + publicly available CUDA kernel performance).
> >>
> >> Cheers,
> >> --
> >> Szilárd
> >>
> >>
> >> On Fri, Jun 5, 2015 at 10:13 PM, Alex <nedomacho at gmail.com> wrote:
> >> > Hi Szilárd,
> >> >
> >> > Well, yes, it is an old CPU, but I am actually quite content with the
> >> > ns/day performance it churns out on all four cores. I'm looking at
> >> >
> >>
> http://manual.gromacs.org/documentation/5.1-beta1/user-guide/environment-variables.html
> >> > and it is my understanding that GMXRC needs to be changed to include
> >> those
> >> > flags, but no recompiling is required. Is that correct?
> >> >
> >> > Thanks,
> >> >
> >> > Alex
> >> >
> >> >
> >> > On Fri, Jun 5, 2015 at 1:52 PM, Szilárd Páll <pall.szilard at gmail.com>
> >> wrote:
> >> >
> >> >> Hi Alex,
> >> >>
> >> >> That's a quite old CPU so it's hard to tell how fast is it, you'll
> >> >> have to try. There are ways to "guesstimate" how fast does your GPU
> >> >> have to be to keep up with your CPU cores (hint search for
> >> >> GMX_EMULATE_GPU / GMX_NO_NONBONDED on the list).
> >> >>
> >> >> In any case, unless your CPU is so slow that it balances well with
> the
> >> >> 750Ti, I would strongly recommend that you get a GTX 960 which will
> >> >> cost just a bit more, but it will be 1.6-1.8x faster at computing
> >> >> non-bonded interactions.
> >> >>
> >> >>  On Fri, Jun 5, 2015 at 9:08 PM, Alex <nedomacho at gmail.com> wrote:
> >> >> > Hi all,
> >> >> >
> >> >> > We're about to start a bit of a toy project to test GPU-accelerated
> >> GMX.
> >> >> > What I have here is my old workstation (Dell Precision T3500:
> >> Quad-core
> >> >> >  Xeon W3530 @ 2.8 GHz, 12 gigs of RAM) running Linux Mint 17.1. The
> >> plan
> >> >> is
> >> >> > to get something like GeForce GTX 750 (
> >> >> >
> >> >>
> >>
> http://www.amazon.com/GeForce-750Ti-GDDR5-Graphics-GTX750TI-OC-2GD5/dp/B00IB9P1KG/ref=sr_1_3?ie=UTF8&qid=1433529993&sr=8-3&keywords=Nvidia+GTX750
> >> >> > )
> >> >> > and maybe a new PSU.
> >> >> >
> >> >> > Questions:
> >> >> >
> >> >> > 1. With a card like this, how much acceleration (very
> approximately)
> >> can
> >> >> we
> >> >> > expect in a "standard" simulation (protein+lipid+water, PME
> >> >> electrostatics)?
> >> >>
> >> >> In PME simulations you should see about 3x speedup, more if you are
> >> >> lucky and possibly a little less if you are not so lucky with the
> >> >> computational workload. What FF are you using?
> >> >>
> >> >> > 2. Should we put any extra considerations into cooling? We expect
> >> >> week-long
> >> >> > runs, and I am not sure these cards have this type of hardcore
> gaming
> >> in
> >> >> > mind. :)
> >> >>
> >> >> I don't think so, but it depends how stuffed or dusty is that Dell
> >> >> workstation. These Maxwell generation (96x) GPUs run quite cool.
> >> >>
> >> >> > Any additional comments are highly appreciated. Please keep in mind
> >> that
> >> >> > this is a toy project to utilize an old workstation.
> >> >>
> >> >> There is a GPU fan speed issue with desktop NVIDIA cards, the fan
> >> >> rotation is limited to 60% and cards can overheat. You can force the
> >> >> fan speed higher using nvidia-settings (you'll need an X server
> >> >> running, though). For more details you can check slide 28 here:
> >> >> https://goo.gl/7DnSri
> >> >>
> >> >> Cheers,
> >> >> --
> >> >> Szilárd
> >> >>
> >> >> > Thank you,
> >> >> >
> >> >> > Alex
> >> >> > --
> >> >> > Gromacs Users mailing list
> >> >> >
> >> >> > * Please search the archive at
> >> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> >> >> posting!
> >> >> >
> >> >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >> >> >
> >> >> > * For (un)subscribe requests visit
> >> >> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users
> or
> >> >> send a mail to gmx-users-request at gromacs.org.
> >> >> --
> >> >> Gromacs Users mailing list
> >> >>
> >> >> * Please search the archive at
> >> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> >> >> posting!
> >> >>
> >> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >> >>
> >> >> * For (un)subscribe requests visit
> >> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users
> or
> >> >> send a mail to gmx-users-request at gromacs.org.
> >> > --
> >> > Gromacs Users mailing list
> >> >
> >> > * Please search the archive at
> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> >> posting!
> >> >
> >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >> >
> >> > * For (un)subscribe requests visit
> >> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> >> send a mail to gmx-users-request at gromacs.org.
> >> --
> >> Gromacs Users mailing list
> >>
> >> * Please search the archive at
> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> >> posting!
> >>
> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>
> >> * For (un)subscribe requests visit
> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> >> send a mail to gmx-users-request at gromacs.org.
> >>
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>


More information about the gromacs.org_gmx-users mailing list