[gmx-users] Multi-GPU optimization, "DD without halo exchange is not supported"

[Leandro Bortot]

Thank you for your reply, Alan.

I didn't realize that I need to use thread-MPI in order to use the GPU
communication features.

Best regards,
Leandro

-------
Leandro Oliveira Bortot
Postdoctoral Fellow
<http://linkedin.com> https://www.linkedin.com/in/leandro-obt/
Laboratory of Computational Biology
Brazilian Biosciences National Laboratory (LNBio)
Brazilian Center for Research in Energy and Materials (CNPEM)
Zip Code 13083-970, Campinas, São Paulo, Brazil.



On Thu, Apr 16, 2020 at 10:37 AM Alan Gray <alang at nvidia.com> wrote:

> Hi Leandro,
>
> I see from your command line that you are using an external MPI library
> and running gmx_mpi via mpirun. The new GPU communication features are
> currently only supported with use of thread MPI (see
> http://www.gromacs.org/Documentation/Acceleration_and_parallelization#Multithreading_with_thread-MPI),
> and this therefore means only within a single compute node. (It looks like
> we need better reporting on this.) We have removed this limitation in a
> branch of the development repository at
> https://gitlab.com/gromacs/gromacs/-/merge_requests/37, but this still
> needs some more work before it is merged into the master branch.
>
> I also see that you are using 16 MPI tasks - I suspect that (regardless of
> the new GPU features) your case might not scale this high since it will be
> limited by the single PME MPI task, but it is best to experiment and tune
> these options (similarly for -nstlist and -ntomp, as Szilard already
> mentioned).
>
> Best regards,
>
> Alan
>
>
> -----------------------------------------------------------------------------------
> This email message is for the sole use of the intended recipient(s) and
> may contain
> confidential information.  Any unauthorized review, use, disclosure or
> distribution
> is prohibited.  If you are not the intended recipient, please contact the
> sender by
> reply email and destroy all copies of the original message.
>
> -----------------------------------------------------------------------------------
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>