[gmx-developers] mpi and thread command line options

[Szilárd Páll]

On Tue, Jul 10, 2012 at 11:15 PM, Berk Hess <hess at kth.se> wrote:
> Hi,
>
> We are working on the final part of the 4.6 release, which is making the MPI
> and OpenMP thread setup automated, fully checked and user friendly.
> We have to decide on the naming of the options.
> Roland has an implementation of mpi spawn ready. This would allow to do
> mdrun -np #processes instead of using mpirun (at least with openmpi).

Would this feature add anything but the convenience of being able to
run without mpirun on a single node? Without MPI spawning working
reliably in most cases (or with the ability to detect with a high
certainty when it does not), enabling an -np mdrun option would just
lead to confusion when mdrun exits with cryptic MPI error due to not
being able to spawn.

Therefore, I'd be OK with a new *hidden* -np option that only works in
single-node case, but not with a non-hidden one advertised in the
documentation/wiki.

> The question now is how to name the (thread-)mpi option and the openmp
> thread option. I suggested -np and -nt, but I now think we should not change
> the functionality of -nt.

I agree, trying to both merge the semantics of -np and -nt and at the
same time re-purpose -nt would lead to a lot of confusion.

> So I would suggest -np and -nth (or -ntsm for shared memory threads).

I'm wondering, would the following a command line not be
confusing(assume a dual-socket hex-core machine):
mdrun -nt 2 -nth 6,
mdrun -nt 2 -ntsm 6,
even more so than what the current developlement code requires
OMP_NUM_THREADS=6 mdrun -nt 2 ?

I don't like -ntsm very much, but the single "h" difference netween
-nt and -nth is not ideal, I think.

Additionally, do we want to keep "-nt" on the long run? To me it
sounds too general especially that the thread-MPI parallelization is
not even "proper" multithreading. So I'd rather make it more verbose
e.g. "-ntmpi" and have with the same functionality as the deprecated
-nt.

> Then -nt can be kept in 4.6 for backward compatibility, with a warning.
> I think we don't want to use -ntomp as we might move away from OpenMP.
> Another question is if the processes MPI and thread-mpi option should
> have the same name. An inexperienced user might submit a thread-mpi mdrun

You mean the default name of the mdrun binary? I would still keep the
mdrun_mpi because I think it's an important distinction.

> job on multiple nodes and get unwanted effects. A way to catch this could
> be to give a fatal error when you ask for more threads then there are cores
> in the node, with the hint that you shoud compile with MPI to run over
> multiple
> nodes and a hint that you can set an env.var. if you really want to use
> more threads than cores.
>
> So in short, my suggestion is: -np and -nth
>
> If you think you have a better suggestion, please tell me.
>
> Cheers,
>
> Berk
>
> --
> gmx-developers mailing list
> gmx-developers at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> Please don't post (un)subscribe requests to the list. Use the www interface
> or send it to gmx-developers-request at gromacs.org.