[gmx-users] Multi-GPU optimization, "DD without halo exchange is not supported"

[Alan Gray]

Hi Leandro,

I see from your command line that you are using an external MPI library and running gmx_mpi via mpirun. The new GPU communication features are currently only supported with use of thread MPI (see http://www.gromacs.org/Documentation/Acceleration_and_parallelization#Multithreading_with_thread-MPI), and this therefore means only within a single compute node. (It looks like we need better reporting on this.) We have removed this limitation in a branch of the development repository at https://gitlab.com/gromacs/gromacs/-/merge_requests/37, but this still needs some more work before it is merged into the master branch.

I also see that you are using 16 MPI tasks - I suspect that (regardless of the new GPU features) your case might not scale this high since it will be limited by the single PME MPI task, but it is best to experiment and tune these options (similarly for -nstlist and -ntomp, as Szilard already mentioned).

Best regards,

Alan

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------