[gmx-users] About the configuration of Gromacs on multiple nodes with GPU

[Szilárd Páll]

Hi,

You can certainly use your hardware setup. I assume you've been looking at
the log/console output based on which it might seem that mdrun is only
using the GPUs in the first (=master) node. However, that is not the case,
it's just that the current hardware and launch configuration reporting is
unfortunately not very flexible and it assumes homogeneous hardware and run
configuration. Whatever gets printed to the console and log shows *only*
information related to the master node. Hence, if you have inhomogeneous
hardware with different CPUs or GPUs, mdrun will try to use all resources -
as long as your launch configuration allows it.

If you want to run across inhomogeneous node configurations, you can pass
node-specific launch/run parameters to mdrun using separate environment
variables per node - the GMX_GPU_ID environment variable to set the GPUs
per node to be used and OMP_NUM_THREADS to set manually set the number of
threads per rank. What you'd need to do is to write a wrapper around mdrun
which will set GMX_GPU_ID=01 on the first node and GMX_GPU_ID=0 (or "00",
for the details see http://goo.gl/a3ZMi).

Cheers,

--
Szilárd


On Sat, Mar 30, 2013 at 10:19 AM, 崔学文 <hp376619815 at gmail.com> wrote:

>  I'm running Gromacs 4.6.1 on 2 nodes with 3 GPUs(2 on host1,1 on host2,
> all nvidia K20c), and I'm using openMPI 1.6.4 to let Gromacs run parallel
> on 2 nodes. However, I found that Gromacs can only find the GPUs on the
> local node which it's launched, it can't automatically use the GPUs on the
> other node.
>  I've been wondering whether current version of Gromacs support such kind
> of configuration(multiple GPUs on multiple nodes). Could you please give me
> some advice on how to fix this problem?
>
>   With many thanks.
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>