[gmx-developers] Increase the scope of dd domains
Mark Abraham
mark.j.abraham at gmail.com
Tue Sep 1 08:57:54 CEST 2015
Hi,
On Tue, Sep 1, 2015 at 6:26 AM Xingcheng Lin <linxingcheng50311 at gmail.com>
wrote:
> Hi,
>
> I have a further question also. It seems that the value of rlist is the
> scope of the increased cells apart from the home cell. In 2D and 3D domain
> decomposition, is this the same case? In other words, does the home node
> know the coordinates of all the particles within the rlist in 2D/3D domain
> decomposition?
>
Yes, by design. Do check out the algorithm description in chapter 3 of the
reference manual, including the flow chart.
Mark
> Thank you,
> Xingcheng
>
>
> On Sun, Aug 30, 2015 at 5:00 AM, <
> gromacs.org_gmx-developers-request at maillist.sys.kth.se> wrote:
>
>> Send gromacs.org_gmx-developers mailing list submissions to
>> gromacs.org_gmx-developers at maillist.sys.kth.se
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>>
>> or, via email, send a message with subject or body 'help' to
>> gromacs.org_gmx-developers-request at maillist.sys.kth.se
>>
>> You can reach the person managing the list at
>> gromacs.org_gmx-developers-owner at maillist.sys.kth.se
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of gromacs.org_gmx-developers digest..."
>>
>>
>> Today's Topics:
>>
>> 1. Re: Increase the scope of dd domains (Xingcheng Lin)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Sat, 29 Aug 2015 11:23:36 -0500
>> From: Xingcheng Lin <linxingcheng50311 at gmail.com>
>> To: gromacs.org_gmx-developers at maillist.sys.kth.se
>
>
>> Subject: Re: [gmx-developers] Increase the scope of dd domains
>> Message-ID:
>>
> <CAPugp-EV_R-vs4cgBAyv_x3TXv=
>> 01UKex2GRjTcjv98Zxrq2vA at mail.gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>
>
>>
>> Even when I choose
>>
>> cutoff-scheme=Verlet
>> cutoff-buffer-drift=-1
>> rlist=1.5
>>
>> some pairs are still missing.
>>
>> I do print them out from different node and use MPI_Reduce to combine them
>> together. That is where I found out it is not a problem if all pairs sit
>> at
>> one node, while it shows up when pairs starts to be distributed into two
>> nodes.
>>
>> To be clear, I put this code at the end of *do_force* function, after the
>> *post_process_forces* function. I think the coordinates have been
>
>
>> communicated up to this point, am I correct?
>>
>> if (bDoForces)
>> {
>> post_process_forces(fplog, cr, step, nrnb, wcycle,
>> top, box, x, f, vir_force, mdatoms, graph, fr, vsite,
>> flags);
>> }
>> if(bNS)
>> {
>> if (DOMAINDECOMP(cr))
>> {
>> int my_rank;
>> int npes;
>> int globalQC;
>> int localQC;
>> MPI_Comm_size(MPI_COMM_WORLD, &npes);
>> MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
>> if (my_rank == 0)
>> {
>> globalQC = 0;
>> }
>> // Barrier for the initialization;
>> MPI_Barrier(MPI_COMM_WORLD);
>>
>> localQC = calQ_dd( x, f, step, cr, inputrec, box );
>>
>> // Barrier for Qcontacts accumulation;
>> MPI_Barrier(MPI_COMM_WORLD);
>> MPI_Reduce(&localQC, &globalQC, 1, MPI_INT, MPI_SUM, 0,
>> MPI_COMM_WORLD);
>>
>> ...
>>
>> Then I print out globalQC, it shows some pairs are missing.
>>
>> The code for calQdd is same as before (struct data is known statically):
>>
>> int calQdd ( rvec x[], rvec f[], int step, t_commrec *cr, t_inputrec *ir,
>> matrix box )
>> {
>> int Npairs = 0;
>> int i, ai, aj;
>> /* for domain decomposition ---------------*/
>> int localai, localaj;
>> int localcell;
>> /* for domain decomposition ---------------*/
>> real r_N, r_N2, dr, dr2 ;
>> rvec dx ;
>> /* Set the pbc ----------------------------*/
>> t_pbc pbc;
>> set_pbc(&pbc, ir->ePBC, box);
>>
>> FILE *fp;
>> fp = fopen("pairs.dat","a");
>> if(fp == NULL){
>> fprintf(stderr, "Can't open input file !\n");
>> exit(1);
>> }
>>
>> if ( NP > 0 )
>> {
>> for (i=0; i<NP; ++i)
>> {
>> ai = data.first[i] ;
>>
>> if (ga2la_get_home(cr->dd->ga2la, ai, &localai))
>> {
>> aj = data.second[i] ;
>>
>> if (ga2la_get(cr->dd->ga2la, aj, &localaj, &localcell))
>> {
>>
>> fprintf(fp, "%d\t%d\n", ai, aj);
>>
>> r_N = data.r_native[i] ;
>> r_N2 = r_N * r_N;
>> pbc_dx(&pbc, x[localai], x[localaj], dx) ;
>> dr2 = iprod( dx, dx ) ;
>> if (dr2 < r_N2)
>> {
>> Npairs += 1;
>> }
>> }
>> }
>> }
>> }
>> fclose(fp);
>> return Npairs;
>> }
>>
>> As you can see, here I also print out the pairs recognized. Though I
>> increase rlist=1.5 still some pairs are missing (though their mutual
>> distance is less than 6 angstroms).
>>
>> Thanks,
>> Xingcheng
>>
>> On Sat, Aug 29, 2015 at 5:00 AM, <
>> gromacs.org_gmx-developers-request at maillist.sys.kth.se> wrote:
>>
>> > Send gromacs.org_gmx-developers mailing list submissions to
>> > gromacs.org_gmx-developers at maillist.sys.kth.se
>> >
>> > To subscribe or unsubscribe via the World Wide Web, visit
>> >
>> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> >
>> > or, via email, send a message with subject or body 'help' to
>> > gromacs.org_gmx-developers-request at maillist.sys.kth.se
>> >
>> > You can reach the person managing the list at
>> > gromacs.org_gmx-developers-owner at maillist.sys.kth.se
>> >
>> > When replying, please edit your Subject line so it is more specific
>> > than "Re: Contents of gromacs.org_gmx-developers digest..."
>> >
>> >
>> > Today's Topics:
>> >
>> > 1. Re: Increase the scope of dd domains (Berk Hess)
>> >
>> >
>> > ----------------------------------------------------------------------
>> >
>> > Message: 1
>> > Date: Fri, 28 Aug 2015 20:35:50 +0200
>> > From: Berk Hess <hess at kth.se>
>> > To: gmx-developers at gromacs.org
>> > Subject: Re: [gmx-developers] Increase the scope of dd domains
>> > Message-ID: <55E0AA06.8020404 at kth.se>
>> > Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>> >
>> > Hi,
>> >
>> > If rlist=1, you should have access to all pairs up to 1 nm.
>> >
>> > One more question:
>> > You do check/print the pairs on all ranks, not only on the master rank?
>> >
>> > Cheers,
>> >
>> > Berk
>> >
>> > On 08/28/2015 08:15 PM, Xingcheng Lin wrote:
>> > > Hi Berk,
>> > >
>> > > Yes, I think I was calling that after the coordinate communication.
>> > > The place where I code it in is after the command:
>> > >
>> > > if (bDoForces)
>> > > {
>> > > post_process_forces(fplog, cr, step, nrnb, wcycle,
>> > > top, box, x, f, vir_force, mdatoms, graph, fr, vsite,
>> > > flags);
>> > > }
>> > >
>> > > After that I use a
>> > > if(bNS){
>> > > if (DOMAINDECOMP(cr)){
>> > >
>> > > to make sure it is done every neighboring list step.
>> > >
>> > > Is this correct? BTW, I was coding that in gromacs4.6.3 version (this
>> > > is for historical reason because I have coded other unrelated things
>> > > in other files).
>> > >
>> > > I have checked that the ga2la_get_home in the first loop find all ai
>> > > atoms, but the second loop ga2la_get missed some aj atoms. One of the
>> > > reasons I guess is because the missing atom pairs are too far apart
>> > > (but less than 6 angstroms) in Cartesian space from each other, so
>> > > that even by including all cells of the home node it still could not
>> > > find it. That could be a problem, but I don't know exactly how -dd
>> > > find this pair partner in its neighboring cells.
>> > >
>> > > Thank you,
>> > > Xingcheng
>> > >
>> >
>> > -------------- next part --------------
>> > An HTML attachment was scrubbed...
>> > URL: <
>> >
>> http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150828/0da41613/attachment-0001.html
>> > >
>> >
>> > ------------------------------
>> >
>> > --
>> > Gromacs Developers mailing list
>> >
>> > * Please search the archive at
>> > http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>> > posting!
>> >
>> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >
>> > * For (un)subscribe requests visit
>> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> > or send a mail to gmx-developers-request at gromacs.org.
>> >
>> > End of gromacs.org_gmx-developers Digest, Vol 136, Issue 25
>> > ***********************************************************
>> >
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>>
> URL: <
>> http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150829/80175e7b/attachment-0001.html
>> >
>>
>> ------------------------------
>>
>
>> --
>> Gromacs Developers mailing list
>>
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>> posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> or send a mail to gmx-developers-request at gromacs.org.
>>
>> End of gromacs.org_gmx-developers Digest, Vol 136, Issue 26
>> ***********************************************************
>>
>
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> or send a mail to gmx-developers-request at gromacs.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150901/f544c7be/attachment-0003.html>
More information about the gromacs.org_gmx-developers
mailing list