[gmx-users] The problem of utilizing multiple GPU

[Mark Abraham]

Hi,


On Wed, 4 Sep 2019 at 12:54, sunyeping <sunyeping at aliyun.com> wrote:

> Dear everyone,
>
> I am trying to do simulation with a workstation with 72 core and 8 geforce
> 1080 GPUs.
>

72 cores, or just 36 cores each with two hyperthreads? (it matters because
you might not want to share cores between simulations, which is what you'd
get if you just assigned 9 hyperthreads per GPU and 1 GPU per simulation).


> When I do not assign a certain GPU with the command:
>       gmx mdrun -v -deffnm md
> all GPUs are used and but the utilization of each GPU is extremely low
> (only 1-2 %), and the simulation will be finished after several months.
>

Yep. Too many workers for not enough work means everyone spends time more
time coordinating than working. This is likely to improve in GROMACS 2020
(beta out shortly).

In contrast, when I assign the simulation task to only one GPU:
> gmx mdrun -v -gpu_id 0 -deffnm md
> the GPU utilization can reach 60-70%, and the simulation can be finished
> within a week. Even when I use only two GPU:
>

Utilization is only a proxy - what you actually want to measure is the rate
of simulation ie. ns/day.

 gmx mdrun -v -gpu_id 0,2 -deffnm md
>
> the GPU utilizations are very low and the simulation is very slow.
>

That could be for a variety of reasons, which you could diagnose by looking
at the performance report at the end of the log file, and comparing
different runs.


> I think I may missuse the GPU for gromacs simulation. Could you tell me
> what is the correct way to use multiple GPUs?
>

If you're happy running multiple simulations, then the easiest thing to do
is to use the existing multi-simulation support to do

mpirun -np 8 gmx_mpi -multidir dir0 dir1 dir2 ... dir7

and let mdrun handle the details. Otherwise you have to get involved in
assigning a subset of the CPU cores and GPUs to each job that both runs
fast and does not conflict. See the documentation for GROMACS for the
version you're running e.g.
http://manual.gromacs.org/documentation/current/user-guide/mdrun-performance.html#running-mdrun-within-a-single-node
.

You probably want to check out this webinar tomorrow
https://bioexcel.eu/webinar-more-bang-for-your-buck-improved-use-of-gpu-nodes-for-gromacs-2018-2019-09-05/
.

Mark


> Best regards
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>