[gmx-developers] mpi and thread command line options

[Alexey Shvetsov]

Hi all!

I think -np for number of mpi process and -nth would be good. But how 
it will handle situation for example with 48 core node (e.g. 4x opteron 
61xx) so it will have 8 numa nodes in one node. Optimal situation may be 
(in case we have more then one interconnect port per platform) to run 
one mpi process per numa node (total 8 mpi processe) and 6 omp threades 
per numa node (mpi porcess)

Berk Hess писал 2012-07-11 01:15:
> Hi,
>
> We are working on the final part of the 4.6 release, which is making 
> the MPI
> and OpenMP thread setup automated, fully checked and user friendly.
> We have to decide on the naming of the options.
> Roland has an implementation of mpi spawn ready. This would allow to 
> do
> mdrun -np #processes instead of using mpirun (at least with openmpi).
> The question now is how to name the (thread-)mpi option and the 
> openmp
> thread option. I suggested -np and -nt, but I now think we should not 
> change
> the functionality of -nt.
> So I would suggest -np and -nth (or -ntsm for shared memory threads).
> Then -nt can be kept in 4.6 for backward compatibility, with a 
> warning.
> I think we don't want to use -ntomp as we might move away from 
> OpenMP.
> Another question is if the processes MPI and thread-mpi option should
> have the same name. An inexperienced user might submit a thread-mpi 
> mdrun
> job on multiple nodes and get unwanted effects. A way to catch this 
> could
> be to give a fatal error when you ask for more threads then there are 
> cores
> in the node, with the hint that you shoud compile with MPI to run
> over multiple
> nodes and a hint that you can set an env.var. if you really want to 
> use
> more threads than cores.
>
> So in short, my suggestion is: -np and -nth
>
> If you think you have a better suggestion, please tell me.
>
> Cheers,
>
> Berk

-- 
Best Regards,
Alexey 'Alexxy' Shvetsov
Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, 
Gatchina, Russia
Department of Molecular and Radiation Biophysics
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexxyum at gmail.com
mailto:alexxy at gentoo.org
mailto:alexxy at omrb.pnpi.spb.ru