[gmx-users] Mismatching number of PP MPI processes and GPUs per node

[Carsten Kutzner]

Hi,

On Mar 11, 2013, at 10:50 AM, George Patargias <gpat at bioacademy.gr> wrote:

> Hello
> 
> Sorry for posting this again.
> 
> I am trying to run GROMACS 4.6 compiled with MPI and GPU acceleration
> (CUDA 5.0 lib) using the following SGE batch script.
> 
> #!/bin/sh
> #$ -V
> #$ -S /bin/sh
> #$ -N test-gpus
> #$ -l h="xgrid-node02"
> #$ -pe mpi_fill_up 12
> #$ -cwd
> 
> source /opt/NetUsers/pgkeka/gromacs-4.6_gpu_mpi/bin/GMXRC
> export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-5.0/lib:$DYLD_LIBRARY_PATH
> 
> mpirun -np 12 mdrun_mpi -s test.tpr -deffnm test_out -nb gpu
> 
> After detection of the installed GPU card
> 
> 1 GPU detected on host xgrid-node02.xgrid:
>  #0: NVIDIA Quadro 4000, compute cap.: 2.0, ECC:  no, stat: compatible
> 
> GROMACS issues the following error
> 
> Incorrect launch configuration: mismatching number of PP MPI processes and
> GPUs per node. mdrun_mpi was started with 12 PP MPI processes per node,
> but only 1 GPU were detected.
> 
> It can't be that we need to run GROMACS only on a single core so that it
> matches the single GPU card.
Have you compiled mdrun_mpi with OpenMP threads support? Then, if you
do 

mpirun -np 1 mdrun_mpi …

it should start one MPI process with 12 OpenMP threads, which should give
you what you want. You can also manually specify the number of OpenMP threads
by adding 

-ntomp 12

Carsten

> 

> 
> Do you have any idea what has to be done?
> 
> Many thanks.
> 
> Dr. George Patargias
> Postdoctoral Researcher
> Biomedical Research Foundation
> Academy of Athens
> 4, Soranou Ephessiou
> 115 27
> Athens
> Greece
> 
> Office: +302106597568
> 
> -- 
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the 
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists


--
Dr. Carsten Kutzner
Max Planck Institute for Biophysical Chemistry
Theoretical and Computational Biophysics
Am Fassberg 11, 37077 Goettingen, Germany
Tel. +49-551-2012313, Fax: +49-551-2012302
http://www.mpibpc.mpg.de/grubmueller/kutzner
http://www.mpibpc.mpg.de/grubmueller/sppexa