[gmx-users] Gromacs-4.6 on two Titans GPUs

[Mark Abraham]

On Tue, Nov 5, 2013 at 12:55 PM, James Starlight <jmsstarlight at gmail.com>wrote:

> Dear Richard,
>
>
> 1)  mdrun -ntmpi 1 -ntomp 12 -gpu_id 0 -v  -deffnm md_CaM_test
> gave me performance about 25ns/day for the explicit solved system consisted
> of 68k atoms (charmm ff. 1.0 cutoofs)
>
> gaves slightly worse performation in comparison to the 1)
>
>
Richard suggested

mdrun -ntmpi 2 -ntomp 6 -gpu_id 01 -v  -deffnm md_CaM_test,

which looks correct to me. -ntomp 6 is probably superfluous

Mark


> finally
>
> 3) mdrun -deffnm md_CaM_test
> running in the same regime as in the 2) so its also gave me 22ns/day for
> the same system.
>
> How the efficacy of using of dual-GPUs could be increased?
>
> James
>
>
> 2013/11/5 Richard Broadbent <richard.broadbent09 at imperial.ac.uk>
>
> > Dear James,
> >
> >
> > On 05/11/13 11:16, James Starlight wrote:
> >
> >> My suggestions:
> >>
> >> 1) During compilstion using -march=corei7-avx-i I have obtained error
> that
> >> somethng now found ( sorry I didnt save log) so I compile gromacs
> without
> >> this flag
> >>
> >> 2) I have twice as better performance using just 1 gpu by means of
> >>
> >> mdrun -ntmpi 1 -ntomp 12 -gpu_id 0 -v  -deffnm md_CaM_test
> >>
> >> than using of both gpus
> >>
> >> mdrun -ntmpi 2 -ntomp 12 -gpu_id 01 -v  -deffnm md_CaM_test
> >>
> >> in the last case I have obtained warning
> >>
> >> WARNING: Oversubscribing the available 12 logical CPU cores with 24
> >> threads.
> >>           This will cause considerable performance loss!
> >>
> >>  here you are requesting 2 thread mpi processes each with 12 openmp
> > threads, hence a total of 24 threads however even with hyper threading
> > enabled there are only 12 threads on your machine. Therefore, only
> allocate
> > 12. Try
> >
> > mdrun -ntmpi 2 -ntomp 6 -gpu_id 01 -v  -deffnm md_CaM_test
> >
> > or even
> >
> > mdrun -v  -deffnm md_CaM_test
> >
> > I believe it should autodetect the GPUs and run accordingly for details
> of
> > how to use gromacs with mpi/thread mpi openmp and GPUs see
> >
> > http://www.gromacs.org/Documentation/Acceleration_and_parallelization
> >
> > Which describes how to use these systems
> >
> > Richard
> >
> >
> >  How it could be fixed?
> >> All gpu are recognized correctly
> >>
> >>
> >> 2 GPUs detected:
> >>    #0: NVIDIA GeForce GTX TITAN, compute cap.: 3.5, ECC:  no, stat:
> >> compatible
> >>    #1: NVIDIA GeForce GTX TITAN, compute cap.: 3.5, ECC:  no, stat:
> >> compatible
> >>
> >>
> >> James
> >>
> >>
> >> 2013/11/4 Szilárd Páll <pall.szilard at gmail.com>
> >>
> >>  You can use the "-march=native" flag with gcc to optimize for the CPU
> >>> your are building on or e.g. -march=corei7-avx-i for Intel Ivy Bridge
> >>> CPUs.
> >>> --
> >>> Szilárd Páll
> >>>
> >>>
> >>> On Mon, Nov 4, 2013 at 12:37 PM, James Starlight <
> jmsstarlight at gmail.com
> >>> >
> >>> wrote:
> >>>
> >>>> Szilárd, thanks for suggestion!
> >>>>
> >>>> What kind of CPU optimisation should I take into account assumint that
> >>>>
> >>> I'm
> >>>
> >>>> using dual-GPU Nvidia TITAN workstation with 6 cores i7 (recognized as
> >>>> 12
> >>>> nodes in Debian).
> >>>>
> >>>> James
> >>>>
> >>>>
> >>>> 2013/11/4 Szilárd Páll <pall.szilard at gmail.com>
> >>>>
> >>>>  That should be enough. You may want to use the -march (or equivalent)
> >>>>> compiler flag for CPU optimization.
> >>>>>
> >>>>> Cheers,
> >>>>> --
> >>>>> Szilárd Páll
> >>>>>
> >>>>>
> >>>>> On Sun, Nov 3, 2013 at 10:01 AM, James Starlight <
> >>>>>
> >>>> jmsstarlight at gmail.com>
> >>>
> >>>> wrote:
> >>>>>
> >>>>>> Dear Gromacs Users!
> >>>>>>
> >>>>>> I'd like to compile lattest 4.6 Gromacs with native GPU supporting
> on
> >>>>>>
> >>>>> my
> >>>
> >>>> i7
> >>>>>
> >>>>>> cpu with dual GeForces Titans gpu mounted. With this config I'd like
> >>>>>>
> >>>>> to
> >>>
> >>>> perform simulations using cpu as well as both gpus simultaneously.
> >>>>>>
> >>>>>> What flags besides
> >>>>>>
> >>>>>> cmake .. -DGMX_GPU=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.5
> >>>>>>
> >>>>>>
> >>>>>> should I define to CMAKE for compiling optimized gromacs on such
> >>>>>>
> >>>>> workstation?
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> Thanks for help
> >>>>>>
> >>>>>> James
> >>>>>> --
> >>>>>> gmx-users mailing list    gmx-users at gromacs.org
> >>>>>> http://lists.gromacs.org/mailman/listinfo/gmx-users
> >>>>>> * Please search the archive at
> >>>>>>
> >>>>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> >>>>>
> >>>>>> * Please don't post (un)subscribe requests to the list. Use the
> >>>>>> www interface or send it to gmx-users-request at gromacs.org.
> >>>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>>>>>
> >>>>> --
> >>>>> gmx-users mailing list    gmx-users at gromacs.org
> >>>>> http://lists.gromacs.org/mailman/listinfo/gmx-users
> >>>>> * Please search the archive at
> >>>>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> >>>>> * Please don't post (un)subscribe requests to the list. Use the
> >>>>> www interface or send it to gmx-users-request at gromacs.org.
> >>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>>>>
> >>>>>  --
> >>>> gmx-users mailing list    gmx-users at gromacs.org
> >>>> http://lists.gromacs.org/mailman/listinfo/gmx-users
> >>>> * Please search the archive at
> >>>>
> >>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> >>>
> >>>> * Please don't post (un)subscribe requests to the list. Use the
> >>>> www interface or send it to gmx-users-request at gromacs.org.
> >>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>>>
> >>> --
> >>> gmx-users mailing list    gmx-users at gromacs.org
> >>> http://lists.gromacs.org/mailman/listinfo/gmx-users
> >>> * Please search the archive at
> >>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> >>> * Please don't post (un)subscribe requests to the list. Use the
> >>> www interface or send it to gmx-users-request at gromacs.org.
> >>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>>
> >>>  --
> > gmx-users mailing list    gmx-users at gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at http://www.gromacs.org/
> > Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the www
> > interface or send it to gmx-users-request at gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>