[gmx-users] Gomacs and multiple GPU

[Szilárd Páll]

Hi,

Do any other programs, e.g. CUDA SDK samples (like deviceQuery) detect
both GPUs?
I find it *very-very* unlikely that only mdrun would not be able
detect both GPUs.

Again, are you sure it's not the use of CUDA_VISIBLE_DEVICES that
creates the confusion? Please share your full command line (with env
vars an all) and resulting log output?

Cheers,
--
Szilárd


On Thu, Dec 1, 2016 at 5:10 PM, Alexis Michon <alexis.michon at ibcp.fr> wrote:
> Hi,
>
> Thank for you reply. Yes, i have read them and know them. I use them
> will success with other software.
>
> My first and main problem is mdrun doesn't see the second GPU on the
> system. And i try to find the root cause of and the remediation.
>
> Hardware is a bi xeon with 2 Titan-xp GPU,  so my goal is to run 2
> differentes simulations with two mdrun process each with one dedicated GPU.
>
> Do you have any idea ?
>
> Cheers
> Alexis
>
> On 01/12/16 16:38, Szilárd Páll wrote:
>> Alexis,
>>
>> What is the problem? Have you read the relevant docs of what
>> CUDA_VISIBLE_DEVICES does?
>> https://docs.nvidia.com/cuda/cuda-c-programming-guide/#env-vars
>>
>>
>> BTW,  you can use, but you *do not need* CUDA_VISIBLE devices to
>> control the mapping of GPU(s) to mdrun process, the -gpu_id variable
>> (or equivalent env var) alone is enough.
>>
>> To run multiple processes per node, do consider:
>> - using mdrun -multi
>> - using 2 or more mdrun runs per GPU
>>
>> You can find the reason, motivating performance examples, and sample
>> commands here (in particular Fig 5 and related section):
>> dx.doi.org/10.1002/jcc.24030
>>
>> Cheers,
>> --
>> Szilárd
>>
>>
>> On Thu, Dec 1, 2016 at 3:38 PM, Alexis Michon <alexis.michon at ibcp.fr> wrote:
>>> Hi,
>>>
>>> thank you for your answer
>>>
>>> the Cmdline is:
>>> gmx mdrun  -deffnm nvt
>>>
>>> I played a bit with option -gpu_id, use $CUDA_VISIBLE_DEVICES  and read
>>> the page http://www.gromacs.org/GPU_acceleration without any succes. One
>>> output is in nvt.log.11,
>>>
>>> nvidia-smi returs is in  nvidia-smi.log
>>>
>>> Nvidia-smi detect 2 GPUs, but not mdrun.
>>>
>>> nvidia-smi.log : https://icloud.ibcp.fr/index.php/s/08jqFsZpWNlSUGh
>>> nvt.log : https://icloud.ibcp.fr/index.php/s/vC5UxCCAhyw9FuC
>>>
>>> Alexis
>>>
>>>
>>>
>>> On 01/12/16 11:34, Mark Abraham wrote:
>>>> Hi,
>>>>
>>>> On Wed, Nov 30, 2016 at 5:38 PM Alexis Michon <alexis.michon at ibcp.fr> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> We have build gromacs 2016.1 from source with "DMX_GPU=on" on a bi
>>>>> processor bi GPU machine, mdrun detect and run fine on only 1 GPU. How
>>>>> could we force mdrun to detect the second GPU ?
>>>>>
>>>> If it's compatible, powered and supported by your driver, then mdrun will
>>>> find it. Presumably nvidia-smi tool will help you work out what's going on.
>>>>
>>>> We would like to run 2 mdrun instances on a machine equiped with 2 GPUs,
>>>>> each mdrun will use 1GPU. How could we tell  mdrun to use a specific GPU ?
>>>>>
>>>> Guidance is here
>>>> http://manual.gromacs.org/documentation/2016.1/user-guide/mdrun-performance.html#examples-for-mdrun-on-one-node
>>>>
>>>> Mark
>>>>
>>>>
>>>>> Cheers,
>>>>> Alexis
>>>>>
>>>>> --
>>>>> Citation : "It’s not enough to be busy; so are the ants. The question is:
>>>>> what are we busy about?" - Henry David Thoreau
>>>>> Alexis MICHON, responsable informatique
>>>>> CNRS IBCP, 7 passage du vercors, 69007 LYON, France
>>>>> Mail : alexis.michon at ibcp.fr  Tel : 04.72.72.26.03 - 06.27.56.34.80
>>>>> CNRS IBCP - UMS 5760 - http://www.ibcp.fr/
>>>>> Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34
>>>>>
>>>>>
>>>>> --
>>>>> Gromacs Users mailing list
>>>>>
>>>>> * Please search the archive at
>>>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>>>>> posting!
>>>>>
>>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>>>
>>>>> * For (un)subscribe requests visit
>>>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>>>>> send a mail to gmx-users-request at gromacs.org.
>>> --
>>> Citation : "It’s not enough to be busy; so are the ants. The question is: what are we busy about?" - Henry David Thoreau
>>> Alexis MICHON, responsable informatique
>>> CNRS IBCP, 7 passage du vercors, 69007 LYON, France
>>> Mail : alexis.michon at ibcp.fr  Tel : 04.72.72.26.03 - 06.27.56.34.80
>>> CNRS IBCP - UMS 5760 - http://www.ibcp.fr/
>>> Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34
>>>
>>>
>>>
>>> --
>>> Gromacs Users mailing list
>>>
>>> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>>>
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>> * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.
>
> --
> Citation : "It’s not enough to be busy; so are the ants. The question is: what are we busy about?" - Henry David Thoreau
> Alexis MICHON, responsable informatique
> CNRS IBCP, 7 passage du vercors, 69007 LYON, France
> Mail : alexis.michon at ibcp.fr  Tel : 04.72.72.26.03 - 06.27.56.34.80
> CNRS IBCP - UMS 5760 - http://www.ibcp.fr/
> Empreinte : C9:45:2D:7C:79:7F:0B:79:CA:C8:0B:68:41:A2:8C:EE:EA:72:82:34
>
>
>
> --
> Gromacs Users mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.