[gmx-users] Query regarding all-atom simulations on 980 Ti GPUs

[Mark Abraham]

On Fri, Jul 1, 2016 at 5:44 AM Suhani Nagpal <suhani.nagpal at gmail.com>
wrote:

> Greetings
>
> I have just recently started to work on this workstation (details as shown
> below) and have employed all-atom simulation of 30,000 atoms with this
> command (just for 1 micro-sec),
>
> gmx mdrun -deffnm md -v -ntmpi 2 -ntomp 6 -gpu_id 01 -pin on
>
>
> Running on 1 node with total 8 cores, 16 logical cores, 2 compatible GPUs
> Hardware detected:
>   CPU info:
>     Vendor: GenuineIntel
>     Brand:  Intel(R) Core(TM) i7-5960X CPU @ 3.00GHz
>     SIMD instructions most likely to fit this hardware: AVX2_256
>     SIMD instructions selected at GROMACS compile time: AVX2_256
>   GPU info:
>     Number of GPUs detected: 2
>     #0: NVIDIA GeForce GTX 980 Ti, compute cap.: 5.2, ECC:  no, stat:
> compatible
>     #1: NVIDIA GeForce GTX 980 Ti, compute cap.: 5.2, ECC:  no, stat:
> compatible
>
> Reading file md.tpr, VERSION 5.1.2 (single precision)
> Using 2 MPI threads
> Using 6 OpenMP threads per tMPI thread
>
> 2 GPUs user-selected for this run.
> Mapping of GPU IDs to the 2 PP ranks in this node: 0,1
>
>
>
> I have just given the command,
>
> and it shows:
>
> step 126800, will finish Tue Jul 12 16:11:59 2016imb F  3%
>
>
> and the imb F increases till 25-28%.
>
> According to the time, this gives a performance of less 100 ns per day.
> Because I have read other benchmarking studies, it should give above 200 ns
> at least.
>

OK, but there's nothing to go upon here. Your simulation should be running
well, but you need to start by inspecting the tail of the log file to see
what mdrun thought might be improved.


> On another note (very strange),
>
> when I subject more than 1 simulation (around 30,000 atoms) on this system,
> the performance reduces a lot (~ 50 ns per day for the second trajectory).
> and
> pin off when running more than one simulation.
>
> What am I doing wrong? I have not worked on GPUs before.
>

That's not unique to GPUs, you'll see the same kind of behaviour if you run
a CPU job (e.g. the same job with mdrun -nb cpu). mdrun performance on a
single node is "compute bound," and cannot share compute resources with
anything else and maintain performance - particularly not another mdrun.
Background information and examples for node sharing can be found at
http://manual.gromacs.org/documentation/5.1.2/user-guide/mdrun-performance.html#running-mdrun-within-a-single-node

Mark


> Kindly help
>
> Thanks a lot
>
> Suhani
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>