[gmx-users] GPU acceleration

Fri Jun 5 22:46:42 CEST 2015

On Fri, Jun 5, 2015 at 10:39 PM, Alex <nedomacho at gmail.com> wrote:
> In that case, I am a bit lost, as I was sure GMXRC needed modification.
> :)). Since I never ran anything on GPU and have an mdrun line that looks
> like
>
> mdrun -nt 4 -s <tpr> -c <final structure>

gmx mdrun -ntomp 4

No need to do domain decomposition. Even with CPU runs it'll be faster
if you don't.

> what should be added to trigger the behavior you're suggesting? I
> understand that it will likely blow up the system, but for the heck of
> it...

those environment variables I mentioned. Plus timer resetting, e.g.
with -resethway if you want reliable timings.

> I am now looking at the stats and see that (NxN QSTab + LJ) and (NxN QSTab
> [F]) take up about 86% of all flops in my simulation. Which of those get
> offloaded?

All of that will be offloaded. However, those are theoretical (useful)
flops, the timing table on the bottom is more relevant. Set the
GMX_CYCLE_SUBCOUNTER=ON breakdown of the "Force".

However, if you just follow the previous emulate+skip non-bondeds
advise, you'll get a run that does the offload except that it skips
calculating - which is useful because you have no GPU to calculate
with.

> Thank you,
>
> Alex
>
>
> On Fri, Jun 5, 2015 at 2:26 PM, Szilárd Páll <pall.szilard at gmail.com> wrote:
>
>> Hi Alex,
>>
>> Why do you think GMXRC needs to be changed? You simply need to set
>> those env. vars to trigger the GPU code-path - and you will likely
>> have to freeze the system, otherwise it will probably not run more
>> than a few steps without seeing an explosion. Based on this you'll
>> know how fast you can run with GPU offload, but you can also
>> guesstimate how fast of a GPU you need (based on the PME+Bonded force
>> calculation time + publicly available CUDA kernel performance).
>>
>> Cheers,
>> --
>> Szilárd
>>
>>
>> On Fri, Jun 5, 2015 at 10:13 PM, Alex <nedomacho at gmail.com> wrote:
>> > Hi Szilárd,
>> >
>> > Well, yes, it is an old CPU, but I am actually quite content with the
>> > ns/day performance it churns out on all four cores. I'm looking at
>> >
>> http://manual.gromacs.org/documentation/5.1-beta1/user-guide/environment-variables.html
>> > and it is my understanding that GMXRC needs to be changed to include
>> those
>> > flags, but no recompiling is required. Is that correct?
>> >
>> > Thanks,
>> >
>> > Alex
>> >
>> >
>> > On Fri, Jun 5, 2015 at 1:52 PM, Szilárd Páll <pall.szilard at gmail.com>
>> wrote:
>> >
>> >> Hi Alex,
>> >>
>> >> That's a quite old CPU so it's hard to tell how fast is it, you'll
>> >> have to try. There are ways to "guesstimate" how fast does your GPU
>> >> have to be to keep up with your CPU cores (hint search for
>> >> GMX_EMULATE_GPU / GMX_NO_NONBONDED on the list).
>> >>
>> >> In any case, unless your CPU is so slow that it balances well with the
>> >> 750Ti, I would strongly recommend that you get a GTX 960 which will
>> >> cost just a bit more, but it will be 1.6-1.8x faster at computing
>> >> non-bonded interactions.
>> >>
>> >>  On Fri, Jun 5, 2015 at 9:08 PM, Alex <nedomacho at gmail.com> wrote:
>> >> > Hi all,
>> >> >
>> >> > We're about to start a bit of a toy project to test GPU-accelerated
>> GMX.
>> >> > What I have here is my old workstation (Dell Precision T3500:
>> Quad-core
>> >> >  Xeon W3530 @ 2.8 GHz, 12 gigs of RAM) running Linux Mint 17.1. The
>> plan
>> >> is
>> >> > to get something like GeForce GTX 750 (
>> >> >
>> >>
>> http://www.amazon.com/GeForce-750Ti-GDDR5-Graphics-GTX750TI-OC-2GD5/dp/B00IB9P1KG/ref=sr_1_3?ie=UTF8&qid=1433529993&sr=8-3&keywords=Nvidia+GTX750
>> >> > )
>> >> > and maybe a new PSU.
>> >> >
>> >> > Questions:
>> >> >
>> >> > 1. With a card like this, how much acceleration (very approximately)
>> can
>> >> we
>> >> > expect in a "standard" simulation (protein+lipid+water, PME
>> >> electrostatics)?
>> >>
>> >> In PME simulations you should see about 3x speedup, more if you are
>> >> lucky and possibly a little less if you are not so lucky with the
>> >> computational workload. What FF are you using?
>> >>
>> >> > 2. Should we put any extra considerations into cooling? We expect
>> >> week-long
>> >> > runs, and I am not sure these cards have this type of hardcore gaming
>> in
>> >> > mind. :)
>> >>
>> >> I don't think so, but it depends how stuffed or dusty is that Dell
>> >> workstation. These Maxwell generation (96x) GPUs run quite cool.
>> >>
>> >> > Any additional comments are highly appreciated. Please keep in mind
>> that
>> >> > this is a toy project to utilize an old workstation.
>> >>
>> >> There is a GPU fan speed issue with desktop NVIDIA cards, the fan
>> >> rotation is limited to 60% and cards can overheat. You can force the
>> >> fan speed higher using nvidia-settings (you'll need an X server
>> >> running, though). For more details you can check slide 28 here:
>> >> https://goo.gl/7DnSri
>> >>
>> >> Cheers,
>> >> --
>> >> Szilárd
>> >>
>> >> > Thank you,
>> >> >
>> >> > Alex
>> >> > --
>> >> > Gromacs Users mailing list
>> >> >
>> >> > * Please search the archive at
>> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> >> posting!
>> >> >
>> >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >> >
>> >> > * For (un)subscribe requests visit
>> >> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> >> send a mail to gmx-users-request at gromacs.org.
>> >> --
>> >> Gromacs Users mailing list
>> >>
>> >> * Please search the archive at
>> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> >> posting!
>> >>
>> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >>
>> >> * For (un)subscribe requests visit
>> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> >> send a mail to gmx-users-request at gromacs.org.
>> > --
>> > Gromacs Users mailing list
>> >
>> > * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> posting!
>> >
>> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >
>> > * For (un)subscribe requests visit
>> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> send a mail to gmx-users-request at gromacs.org.
>> --
>> Gromacs Users mailing list
>>
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> send a mail to gmx-users-request at gromacs.org.
>>
> --
> Gromacs Users mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.