[gmx-users] Gromacs-4.6 on two Titans GPUs

[James Starlight]

Dear Richard,


1)  mdrun -ntmpi 1 -ntomp 12 -gpu_id 0 -v  -deffnm md_CaM_test
gave me performance about 25ns/day for the explicit solved system consisted
of 68k atoms (charmm ff. 1.0 cutoofs)

gaves slightly worse performation in comparison to the 1)

finally

3) mdrun -deffnm md_CaM_test
running in the same regime as in the 2) so its also gave me 22ns/day for
the same system.

How the efficacy of using of dual-GPUs could be increased?

James


2013/11/5 Richard Broadbent <richard.broadbent09 at imperial.ac.uk>

> Dear James,
>
>
> On 05/11/13 11:16, James Starlight wrote:
>
>> My suggestions:
>>
>> 1) During compilstion using -march=corei7-avx-i I have obtained error that
>> somethng now found ( sorry I didnt save log) so I compile gromacs without
>> this flag
>>
>> 2) I have twice as better performance using just 1 gpu by means of
>>
>> mdrun -ntmpi 1 -ntomp 12 -gpu_id 0 -v  -deffnm md_CaM_test
>>
>> than using of both gpus
>>
>> mdrun -ntmpi 2 -ntomp 12 -gpu_id 01 -v  -deffnm md_CaM_test
>>
>> in the last case I have obtained warning
>>
>> WARNING: Oversubscribing the available 12 logical CPU cores with 24
>> threads.
>>           This will cause considerable performance loss!
>>
>>  here you are requesting 2 thread mpi processes each with 12 openmp
> threads, hence a total of 24 threads however even with hyper threading
> enabled there are only 12 threads on your machine. Therefore, only allocate
> 12. Try
>
> mdrun -ntmpi 2 -ntomp 6 -gpu_id 01 -v  -deffnm md_CaM_test
>
> or even
>
> mdrun -v  -deffnm md_CaM_test
>
> I believe it should autodetect the GPUs and run accordingly for details of
> how to use gromacs with mpi/thread mpi openmp and GPUs see
>
> http://www.gromacs.org/Documentation/Acceleration_and_parallelization
>
> Which describes how to use these systems
>
> Richard
>
>
>  How it could be fixed?
>> All gpu are recognized correctly
>>
>>
>> 2 GPUs detected:
>>    #0: NVIDIA GeForce GTX TITAN, compute cap.: 3.5, ECC:  no, stat:
>> compatible
>>    #1: NVIDIA GeForce GTX TITAN, compute cap.: 3.5, ECC:  no, stat:
>> compatible
>>
>>
>> James
>>
>>
>> 2013/11/4 Szilárd Páll <pall.szilard at gmail.com>
>>
>>  You can use the "-march=native" flag with gcc to optimize for the CPU
>>> your are building on or e.g. -march=corei7-avx-i for Intel Ivy Bridge
>>> CPUs.
>>> --
>>> Szilárd Páll
>>>
>>>
>>> On Mon, Nov 4, 2013 at 12:37 PM, James Starlight <jmsstarlight at gmail.com
>>> >
>>> wrote:
>>>
>>>> Szilárd, thanks for suggestion!
>>>>
>>>> What kind of CPU optimisation should I take into account assumint that
>>>>
>>> I'm
>>>
>>>> using dual-GPU Nvidia TITAN workstation with 6 cores i7 (recognized as
>>>> 12
>>>> nodes in Debian).
>>>>
>>>> James
>>>>
>>>>
>>>> 2013/11/4 Szilárd Páll <pall.szilard at gmail.com>
>>>>
>>>>  That should be enough. You may want to use the -march (or equivalent)
>>>>> compiler flag for CPU optimization.
>>>>>
>>>>> Cheers,
>>>>> --
>>>>> Szilárd Páll
>>>>>
>>>>>
>>>>> On Sun, Nov 3, 2013 at 10:01 AM, James Starlight <
>>>>>
>>>> jmsstarlight at gmail.com>
>>>
>>>> wrote:
>>>>>
>>>>>> Dear Gromacs Users!
>>>>>>
>>>>>> I'd like to compile lattest 4.6 Gromacs with native GPU supporting on
>>>>>>
>>>>> my
>>>
>>>> i7
>>>>>
>>>>>> cpu with dual GeForces Titans gpu mounted. With this config I'd like
>>>>>>
>>>>> to
>>>
>>>> perform simulations using cpu as well as both gpus simultaneously.
>>>>>>
>>>>>> What flags besides
>>>>>>
>>>>>> cmake .. -DGMX_GPU=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.5
>>>>>>
>>>>>>
>>>>>> should I define to CMAKE for compiling optimized gromacs on such
>>>>>>
>>>>> workstation?
>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks for help
>>>>>>
>>>>>> James
>>>>>> --
>>>>>> gmx-users mailing list    gmx-users at gromacs.org
>>>>>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>>>>>> * Please search the archive at
>>>>>>
>>>>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>>>>
>>>>>> * Please don't post (un)subscribe requests to the list. Use the
>>>>>> www interface or send it to gmx-users-request at gromacs.org.
>>>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>>>>
>>>>> --
>>>>> gmx-users mailing list    gmx-users at gromacs.org
>>>>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>>>>> * Please search the archive at
>>>>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>>>> * Please don't post (un)subscribe requests to the list. Use the
>>>>> www interface or send it to gmx-users-request at gromacs.org.
>>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>>>
>>>>>  --
>>>> gmx-users mailing list    gmx-users at gromacs.org
>>>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>>>> * Please search the archive at
>>>>
>>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>>
>>>> * Please don't post (un)subscribe requests to the list. Use the
>>>> www interface or send it to gmx-users-request at gromacs.org.
>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>>
>>> --
>>> gmx-users mailing list    gmx-users at gromacs.org
>>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>> * Please don't post (un)subscribe requests to the list. Use the
>>> www interface or send it to gmx-users-request at gromacs.org.
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>>  --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/
> Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>