[gmx-users] GTX 1070 Performance

[Mark Abraham]

Hi,

On Sat, Nov 19, 2016 at 11:40 AM Nikhil Maroli <scinikhil at gmail.com> wrote:

> Dear all,
> We have 3x GTX 1070 and 2x 12 core Xeon processor.
>
> My simulation system consists of 540 amino acids +ligand in water.Total
> number of atoms excluding water is ~8000.Charmm36 ff and time step of
> 100ps.
>

You have many more water atoms than this, and they dominate the work in
calculating the forces, so the total number of atoms is a much better
approximate predictor of workload.

i tested small time simulation in 2 GPU card with the following command
>
> gmx mdrun -v -ntmpi 2 -ntomp 24 -deffnm md_0_1 -gpu_id 03
>
>
> I got 50ns/day ,but as per the benchmark results and few previous mails in
> mailing list with 2 x 1070 cards i must get ~100ns/day.
>
> Gromacs installed with GPU=on flag ,Do i need to install with MPI+GPU on
> flag to achieve better performance ? or any other advice to get better
> performance with 2 and 3 GPU Cards.
>

Line 338 of your log file provides a valuable hint - you are using very
many OpenMP threads per rank, and this means you are spending 30% of your
time doing setup for PME. Those parts of the code do not scale well as the
number of OpenMP threads per rank increases. See also figures 2 and 3 from
http://onlinelibrary.wiley.com/doi/10.1002/jcc.24030/abstract for
illustration of analogous situations. Try also other divisors of 48 (the
number of hardware threads you have on your node).

gmx mdrun -ntmpi 4 -gpu_id 02
gmx mdrun -ntmpi 8 -gpu_id 02
gmx mdrun -ntmpi 12 -gpu_id 02

(You might need to use -ntmpi 4 -gpu_id 0022, etc. I forget whether we
improved that for 5.1 or 2016 version.)

Mark


>
>
> I have attached the log file.
>
>
> https://drive.google.com/file/d/0BxaQk_pcR9viSE1xWUM1Z1EtZkU/view?usp=sharing
>
>
> --
> Regards,
> Nikhil Maroli
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>