[gmx-developers] DD support in ASPC

[Berk Hess]

Hi,

Keeping track of history is tedious, both for domain decomposition and 
check pointing.

I assume the method should actually not be history dependent and this is 
only for efficiency?

Cheers,

Berk

On 03/06/2012 10:57 AM, Tomáš Trnka wrote:
> Hello,
>
> the ASPC code as currently submitted for 4.6 review doesn't work with domain
> decomposition. This stems from the fact that I have neither any significant
> experience in MPI programming nor deep understanding of the DD core.
>
> However, supporting DD shouldn't be too hard. For each shell, ASPC keeps a
> history of its relative displacement from the carrying nucleus and uses it to
> predict the new position in each integration step. There are no cross-shell
> dependencies (a shell's history is only used to predict the position for that
> shell). So the only thing that's necessary for working DD is that when a shell
> crosses from one node to another, it needs to carry its history over.
>
> Could anyone please provide me with some pointers how this can be best
> achieved? I've tried several approaches but every time the thing got too
> complicated too soon:
>
> 1) extend t_state with the ASPC history arrays - main problems with this were
> that t_state is touched in hundreds of places throughout the code; also the
> placement of shells onto individual DD nodes would need to be recorded
> somewhere (another global index?)
>
> 2) keep the history within gmx_shellfc_t and collect/distribute in
> make_local_shells() - old shell indices would need to be kept somewhere there
> too to be able to do the collection or possibly the indices could be collected
> from the nodes, too; I have a bad feeling about doing collection in here as it
> is called from dd_partition_system, so it's rather counter-intuitive and could
> break something
>
> 3) ???
>
> Thanks for any suggestions!
> Best regards
>
> Tomáš Trnka