[gmx-users] GROMACS-4.6.3 CUDA version on multiple nodes each having 2 GPUs

[Mark Abraham]

On Mon, Nov 18, 2013 at 12:06 PM, Prajapati, Jigneshkumar Dahyabhai <
j.prajapati at jacobs-university.de> wrote:

> Hi Carsten,
>
> Thanks for reply. Everything is working fine on single node. The problem
> starts when I move to two nodes.
>
> I have tried with the option that you have mentioned earlier and this is
> the error I got ,
> mpirun -np 4 mdrun
>
> mismatching number of PP MPI processes and GPUs per node.
> mdrun was started with 4 PP MPI processes per node, but only 2 GPUs were
> detected.
>

That sounds like your hostfile / mpirun command line is not set up so that
mpirun knows to dispatch two processes to each node.

Mark


>
> Likewise, I have tried many options and on many occasions job runs but
> without using GPUs on second node . More specifically, my problem is when I
> use two nodes each having two GPU cards, GROMACS detects GPUs on first node
> only. It fails to detect GPU cards on second node (check the em.log file in
> attachments). I am not sure what is the reason. Please let me know if I
> could try something else.
>
> Thank you.
>
> -Jignesh
>
>
>
>
>
> ________________________________________
> From: gromacs.org_gmx-users-bounces at maillist.sys.kth.se [
> gromacs.org_gmx-users-bounces at maillist.sys.kth.se] on behalf of Carsten
> Kutzner [ckutzne at gwdg.de]
> Sent: Thursday, November 14, 2013 10:54 AM
> To: Discussion list for GROMACS users
> Subject: Re: [gmx-users] GROMACS-4.6.3 CUDA version on multiple nodes each
>      having 2 GPUs
>
> Hi,
>
> if you run on a single node with 2 GPUs, this command line should work:
>
> > mpirun -np 2  mdrun -v -deffnm $configfile
>
> If you run on two nodes, try this:
> mpirun -np 4 mdrun
>
> Choosing -np equal to the total number of GPUs should work (although it
> might
> not be the best option performance-wise).
>
> For a better performance you can try
> mpirun -np 4 mdrun -gpu_id 0011    on a single node
> or
> mpirun -np 8 mdrun -gpu_id 0011    on two nodes
>
> Carsten
>
>
> On Nov 13, 2013, at 7:55 PM, "Prajapati, Jigneshkumar Dahyabhai" <
> j.prajapati at jacobs-university.de> wrote:
>
> > Hello,
> >
> > I am trying to run MPI, OpenMP and CUDA enable GROMACS 4.6.3 on nodes
> having 12 cores (2 CPUs) and 2 GPUs (Tesla M2090) each. The problem is when
> I launch job GROMCAS is using only GPUs on first node come across and
> failing to use GPUs on other nodes.
> >
> > The command I used for two gpu enable nodes was,
> >
> > mpirun -np 2  mdrun -v -deffnm $configfile
> >
> > I tried with many other options but none of them worked. The one thing
> needs to remember here is that on all the nodes, GPUs got id 0 and 1 so
> -gpu_id option also didn't work.
> >
> > This old thread gave me some idea but I didn't understand it completely.
> > http://lists.gromacs.org/pipermail/gmx-users/2013-March/079802.html
> >
> > Please suggests me the possible solutions for this issue.
> >
> > Thank you
> > --Jignesh
> > --
> > gmx-users mailing list    gmx-users at gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-request at gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
>
> --
> Dr. Carsten Kutzner
> Max Planck Institute for Biophysical Chemistry
> Theoretical and Computational Biophysics
> Am Fassberg 11, 37077 Goettingen, Germany
> Tel. +49-551-2012313, Fax: +49-551-2012302
> http://www.mpibpc.mpg.de/grubmueller/kutzner
> http://www.mpibpc.mpg.de/grubmueller/sppexa
>
> --
> gromacs.org_gmx-users mailing list
> gromacs.org_gmx-users at maillist.sys.kth.se
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> --
> gromacs.org_gmx-users mailing list
> gromacs.org_gmx-users at maillist.sys.kth.se
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>