[gmx-developers] mpi and thread command line options

[Christoph Junghans]

2012/7/11 Roland Schulz <roland at utk.edu>:
> On Wed, Jul 11, 2012 at 7:19 AM, Alexey Shvetsov
> <alexxy at omrb.pnpi.spb.ru> wrote:
>> Roland Schulz писал 2012-07-11 10:47:
>>> On Wed, Jul 11, 2012 at 2:33 AM, Alexey Shvetsov
>>> <alexxy at omrb.pnpi.spb.ru> wrote:
>>>> Hi!
>>>>
>>>> mvpich2/mvapich (as well as its derviations like platform
>>>> mpi,pc-mpi,intel-mpi) also will behave differently. So user can get
>>>> cryptic message about launching mpd, in case of launching mdrun -np
>>>> directly
>>> Not quite. mpich2 requires for MPI_Comm_spawn to work that the
>>> application is run with mpirun/mpiexec. See
>>>
>>> http://lists.mcs.anl.gov/pipermail/mpich-discuss/2012-June/012638.html
>>> for the details. We would need to detect that and don't try to spawn
>>> in that case (and run in serial with a warning).
>>> Thus mpich2 would require: mpirun mdrun -np x. Of course that isn't
>>> more convenient than mpirun -np x mdrun. The only advantage would be
>>> that as with tmpi "mpirun mdrun" would automatically use as many
>>> cores
>>> as are available and are useful for the system, whereas without spawn
>>> the user needs to decides the number of cores and we can't have any
>>> automatic mechanism helping the user.
>>>
>>> Roland
>>
>> Ok. But how this will work with batch systems that automaticaly send
>> number of processes to mpiexec, effective launch command will be
>>
>> $ mpiexec mdrun_mpi $mdrunargs
>
> The idea was to only do any spawn if mdrun_mpi is started in serial
> (mpiexec -n 1). It was only meant to make mdrun_mpi behave the same as
> tmpi mdrun for a single node. On clusters with batch system nothing
> would have changed over the current situation.
1.) I agree with Roland that keeping the -nt option would be
misleading, even if -nt gets a new meaning - "number of tasks", where
tasks can be threads or mpi tasks.
Also from my experience, half of the users are not aware of the -nt
option either, they just start mdrun without any special setting,
which means "guess" and we should keep that.
So making -nt obsolete is not bad, I think.

2.) One class of system that has not been discussed yet, are the one
with different number of OMP threads per node (like the Intel MIC),
that should also be possible by explicitly defining OMP_NUM_THREADS on
each node.

3.) With all these thread options and combination, we will need
something like g_tune_mdrun, which I guess could be an extension of
g_tune_pme.

Christoph



>
> Roland
>
>>
>>
>>
>>>
>>>>
>>>> Roland Schulz писал 2012-07-11 05:09:
>>>>> On Tue, Jul 10, 2012 at 8:09 PM, Szilárd Páll
>>>>> <szilard.pall at cbr.su.se> wrote:
>>>>>> On Tue, Jul 10, 2012 at 11:15 PM, Berk Hess <hess at kth.se> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> We are working on the final part of the 4.6 release, which is
>>>>>>> making the MPI
>>>>>>> and OpenMP thread setup automated, fully checked and user
>>>>>>> friendly.
>>>>>>> We have to decide on the naming of the options.
>>>>>>> Roland has an implementation of mpi spawn ready. This would allow
>>>>>>> to do
>>>>>>> mdrun -np #processes instead of using mpirun (at least with
>>>>>>> openmpi).
>>>>>>
>>>>>> Would this feature add anything but the convenience of being able
>>>>>> to
>>>>>> run without mpirun on a single node? Without MPI spawning working
>>>>>> reliably in most cases (or with the ability to detect with a high
>>>>>> certainty when it does not), enabling an -np mdrun option would
>>>>>> just
>>>>>> lead to confusion when mdrun exits with cryptic MPI error due to
>>>>>> not
>>>>>> being able to spawn.
>>>>> The idea was to make mdrun behave the same whether it is compiled
>>>>> with
>>>>> real MPI or tMPI. Thus also only support a single node. But MPICH
>>>>> is
>>>>> behaving quite stupid and they also don't seem to care. And only
>>>>> supporting it for OpenMPI is probably also more confusing then
>>>>> helpful
>>>>> (then tmpi+OpenMPI would behave the same but MPICH/MVAPICH would
>>>>> behave different). So you are probably right that it is better to
>>>>> not
>>>>> add spawn at all.
>>>>>
>>>>>> Therefore, I'd be OK with a new *hidden* -np option that only
>>>>>> works
>>>>>> in
>>>>>> single-node case, but not with a non-hidden one advertised in the
>>>>>> documentation/wiki.
>>>>> As a hidden option it would only help for testing. But I don't
>>>>> think
>>>>> it is worth adding it for just that.
>>>>>
>>>>> Roland
>>>>
>>>> --
>>>> Best Regards,
>>>> Alexey 'Alexxy' Shvetsov
>>>> Petersburg Nuclear Physics Institute, NRC Kurchatov Institute,
>>>> Gatchina, Russia
>>>> Department of Molecular and Radiation Biophysics
>>>> Gentoo Team Ru
>>>> Gentoo Linux Dev
>>>> mailto:alexxyum at gmail.com
>>>> mailto:alexxy at gentoo.org
>>>> mailto:alexxy at omrb.pnpi.spb.ru
>>>> --
>>>> gmx-developers mailing list
>>>> gmx-developers at gromacs.org
>>>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>>>> Please don't post (un)subscribe requests to the list. Use the
>>>> www interface or send it to gmx-developers-request at gromacs.org.
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
>>> 865-241-1537, ORNL PO BOX 2008 MS6309
>>
>> --
>> Best Regards,
>> Alexey 'Alexxy' Shvetsov
>> Petersburg Nuclear Physics Institute, NRC Kurchatov Institute,
>> Gatchina, Russia
>> Department of Molecular and Radiation Biophysics
>> Gentoo Team Ru
>> Gentoo Linux Dev
>> mailto:alexxyum at gmail.com
>> mailto:alexxy at gentoo.org
>> mailto:alexxy at omrb.pnpi.spb.ru
>> --
>> gmx-developers mailing list
>> gmx-developers at gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>> Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-developers-request at gromacs.org.
>>
>>
>>
>
>
>
> --
> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> 865-241-1537, ORNL PO BOX 2008 MS6309
> --
> gmx-developers mailing list
> gmx-developers at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-developers-request at gromacs.org.



-- 
Christoph Junghans
Web: http://www.compphys.de