[gmx-users] GPU-accelerated EM

[Szilárd Páll]

I'm still not sure why would the GPU-accelerated EM runs slower than
the CPU-only run -- unless the GPU in question is at best as fast at
computing nonbonded interactions as the CPU (cores) assigned to the
job.

Have you looked at log files?

--
Szilárd


On Sat, Sep 9, 2017 at 1:33 AM, Alex <nedomacho at gmail.com> wrote:
> Mark,
>
> I agree with you 100%, given how I've always used particle-based simulation
> tools. It is my understanding that my colleague is optimizing custom
> forcefields based on EM, followed by short MD. The work is also unrelated
> to what most of the GMX community does, so I stay away from this part.
> What happens then is also perception-dependent. On our older boxes without
> GPU acceleration, he'd see equally sluggish performance from EM and MD,
> while now his EM (when submitted incorrectly) runs slowly and the MD
> portion completes in a quick burst. This can be perceived as EM being the
> bottleneck. Before my initial post, we already decided on CPU-only EM and
> your advice to move the EM portion elsewhere (e.g. our old machines) is
> even better.
>
> Thanks,
>
> Alex
>
>
> On Fri, Sep 8, 2017 at 4:27 PM, Mark Abraham <mark.j.abraham at gmail.com>
> wrote:
>
>> Hi,
>>
>> The em log files report using the GPUs, and they do. It just isn't very
>> useful because the EM implementations do a lot more neighbour search
>> stages, which doesn't use the GPUs.
>>
>> So, if there's other work for the GPUs to do, then I would plan to run a
>> workflow heavily dominated by EM, on a non-GPU resource.
>>
>> We're not aware of a use case that would justify thinking about how to make
>> a correct implementation of EM that could do less searching, because in
>> most cases the time spent in EM is a tiny fraction of the time spent in MD.
>> Is the workflow useful to optimize in this way?
>>
>> Mark
>>
>> On Fri, Sep 8, 2017 at 11:53 PM Alex <nedomacho at gmail.com> wrote:
>>
>> > Hi all,
>> >
>> > I tend to run one fat node-wise simulation, but we have another user
>> whose
>> > use case is different. He runs a bunch of small jobs at the same time and
>> > in his case most of the time is taken up by the EM. We have two
>> > hyperthreaded Xeons E5 (44 cores total) and 3 GPUs.
>> >
>> > So, we try to run 10 jobs using 4 cores each. Each mdrun line includes
>> >
>> > -ntomp 4 -pin on -pinoffset x -pinstride 1 -gpu_id y
>> >
>> > where pinoffset is 0, 4, 8, etc for the first, second, etc, simulation
>> and
>> > y changes from 0 to 2 so that the first two GPUs are subscribed three
>> > times, while the third one is subscribed four times.
>> >
>> > Upon submitting this batch, each mdrun instance utilizes something like
>> 10%
>> > of CPU and the EM jobs proceed slowly. Removing GPU and setting -nb cpu
>> > seems to fix that. With MD, everything works properly, but with EM
>> there's
>> > no GPU acceleration and low CPU usage. Is this correct behavior?
>> >
>> > Thanks,
>> >
>> > Alex
>> > --
>> > Gromacs Users mailing list
>> >
>> > * Please search the archive at
>> > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> > posting!
>> >
>> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >
>> > * For (un)subscribe requests visit
>> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> > send a mail to gmx-users-request at gromacs.org.
>> >
>> --
>> Gromacs Users mailing list
>>
>> * Please search the archive at http://www.gromacs.org/
>> Support/Mailing_Lists/GMX-Users_List before posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> send a mail to gmx-users-request at gromacs.org.
>>
> --
> Gromacs Users mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.