[gmx-users] multiple GPU on multiple nodes

[Szilárd Páll]

On Tue, Feb 4, 2014 at 1:51 AM, cyberjhon <cyberjhon at hotmail.com> wrote:
> Dear Szilárd
>
> Thanks for your answer.
>
> To submit the job I do;
>
> qsub -l nodes=2:ppn=16,walltime=12:00:00
>
> Then, to run gromacs I can do:
> aprun -n 1 mdrun_mpi -deffnm protein

Those qsub and aprun commands are not in like with each other. Your
submission requests 2 nodes and 16 ranks/processes per node, that this
32 ranks in total. However, your aprun lauch requests only a single
rank (and only two below). Please consult the machine's documentation
on how to launch jobs correctly.

--
Szilárd

> And, I get the message that you mention, which is good
> "1 GPU detected on host nid00900:" does not mean that only one GPU was
> detected."
>
> But If I do:
> aprun -n 2 mdrun_mpi -deffnm protein
>
> I get the error mentioned before
> -------------------------------------------------------
> Program mdrun_mpi, VERSION 4.6.2
> Source code file:
> /N/soft/cle4/gromacs/gromacs-4.6.2/src/gmxlib/gmx_detect_hardware.c, line:
> 356
>
> Fatal error:
> Incorrect launch configuration: mismatching number of PP MPI processes and
> GPUs per node.
> mdrun_mpi was started with 2 PP MPI processes per node, but only 1 GPU were
> detected.
> For more information and tips for troubleshooting, please check the GROMACS
> website at http://www.gromacs.org/Documentation/Errors
> -------------------------------------------------------
>
> Even if I use -gpu_id 0 or -gpu_id 00, they do not work
>
> I also tried what you told me before about CRAY_CUDA_MPS  and
> CRAY_CUDA_PROXY
> export CRAY_CUDA_MPS= 1
> export CRAY_CUDA_PROXY=1
>
> I executed these commands before the aprun, and they did not work
>
> So, can you tell me what is the command that you use on BW to use the GPU
> located in each node?
>
> Thanks
>
> John Michael
>
>
>
>
>
>
>
> --
> View this message in context: http://gromacs.5086.x6.nabble.com/multiple-GPU-on-multiple-nodes-tp5014085p5014254.html
> Sent from the GROMACS Users Forum mailing list archive at Nabble.com.
> --
> Gromacs Users mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.