[gmx-developers] Increase the scope of dd domains
Xingcheng Lin
linxingcheng50311 at gmail.com
Tue Sep 1 06:25:53 CEST 2015
Hi,
I have a further question also. It seems that the value of rlist is the
scope of the increased cells apart from the home cell. In 2D and 3D domain
decomposition, is this the same case? In other words, does the home node
know the coordinates of all the particles within the rlist in 2D/3D domain
decomposition?
Thank you,
Xingcheng
On Sun, Aug 30, 2015 at 5:00 AM, <
gromacs.org_gmx-developers-request at maillist.sys.kth.se> wrote:
> Send gromacs.org_gmx-developers mailing list submissions to
> gromacs.org_gmx-developers at maillist.sys.kth.se
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>
> or, via email, send a message with subject or body 'help' to
> gromacs.org_gmx-developers-request at maillist.sys.kth.se
>
> You can reach the person managing the list at
> gromacs.org_gmx-developers-owner at maillist.sys.kth.se
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of gromacs.org_gmx-developers digest..."
>
>
> Today's Topics:
>
> 1. Re: Increase the scope of dd domains (Xingcheng Lin)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 29 Aug 2015 11:23:36 -0500
> From: Xingcheng Lin <linxingcheng50311 at gmail.com>
> To: gromacs.org_gmx-developers at maillist.sys.kth.se
> Subject: Re: [gmx-developers] Increase the scope of dd domains
> Message-ID:
> <CAPugp-EV_R-vs4cgBAyv_x3TXv=
> 01UKex2GRjTcjv98Zxrq2vA at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Even when I choose
>
> cutoff-scheme=Verlet
> cutoff-buffer-drift=-1
> rlist=1.5
>
> some pairs are still missing.
>
> I do print them out from different node and use MPI_Reduce to combine them
> together. That is where I found out it is not a problem if all pairs sit at
> one node, while it shows up when pairs starts to be distributed into two
> nodes.
>
> To be clear, I put this code at the end of *do_force* function, after the
> *post_process_forces* function. I think the coordinates have been
> communicated up to this point, am I correct?
>
> if (bDoForces)
> {
> post_process_forces(fplog, cr, step, nrnb, wcycle,
> top, box, x, f, vir_force, mdatoms, graph, fr, vsite,
> flags);
> }
> if(bNS)
> {
> if (DOMAINDECOMP(cr))
> {
> int my_rank;
> int npes;
> int globalQC;
> int localQC;
> MPI_Comm_size(MPI_COMM_WORLD, &npes);
> MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
> if (my_rank == 0)
> {
> globalQC = 0;
> }
> // Barrier for the initialization;
> MPI_Barrier(MPI_COMM_WORLD);
>
> localQC = calQ_dd( x, f, step, cr, inputrec, box );
>
> // Barrier for Qcontacts accumulation;
> MPI_Barrier(MPI_COMM_WORLD);
> MPI_Reduce(&localQC, &globalQC, 1, MPI_INT, MPI_SUM, 0,
> MPI_COMM_WORLD);
>
> ...
>
> Then I print out globalQC, it shows some pairs are missing.
>
> The code for calQdd is same as before (struct data is known statically):
>
> int calQdd ( rvec x[], rvec f[], int step, t_commrec *cr, t_inputrec *ir,
> matrix box )
> {
> int Npairs = 0;
> int i, ai, aj;
> /* for domain decomposition ---------------*/
> int localai, localaj;
> int localcell;
> /* for domain decomposition ---------------*/
> real r_N, r_N2, dr, dr2 ;
> rvec dx ;
> /* Set the pbc ----------------------------*/
> t_pbc pbc;
> set_pbc(&pbc, ir->ePBC, box);
>
> FILE *fp;
> fp = fopen("pairs.dat","a");
> if(fp == NULL){
> fprintf(stderr, "Can't open input file !\n");
> exit(1);
> }
>
> if ( NP > 0 )
> {
> for (i=0; i<NP; ++i)
> {
> ai = data.first[i] ;
>
> if (ga2la_get_home(cr->dd->ga2la, ai, &localai))
> {
> aj = data.second[i] ;
>
> if (ga2la_get(cr->dd->ga2la, aj, &localaj, &localcell))
> {
>
> fprintf(fp, "%d\t%d\n", ai, aj);
>
> r_N = data.r_native[i] ;
> r_N2 = r_N * r_N;
> pbc_dx(&pbc, x[localai], x[localaj], dx) ;
> dr2 = iprod( dx, dx ) ;
> if (dr2 < r_N2)
> {
> Npairs += 1;
> }
> }
> }
> }
> }
> fclose(fp);
> return Npairs;
> }
>
> As you can see, here I also print out the pairs recognized. Though I
> increase rlist=1.5 still some pairs are missing (though their mutual
> distance is less than 6 angstroms).
>
> Thanks,
> Xingcheng
>
> On Sat, Aug 29, 2015 at 5:00 AM, <
> gromacs.org_gmx-developers-request at maillist.sys.kth.se> wrote:
>
> > Send gromacs.org_gmx-developers mailing list submissions to
> > gromacs.org_gmx-developers at maillist.sys.kth.se
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> >
> > or, via email, send a message with subject or body 'help' to
> > gromacs.org_gmx-developers-request at maillist.sys.kth.se
> >
> > You can reach the person managing the list at
> > gromacs.org_gmx-developers-owner at maillist.sys.kth.se
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of gromacs.org_gmx-developers digest..."
> >
> >
> > Today's Topics:
> >
> > 1. Re: Increase the scope of dd domains (Berk Hess)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Fri, 28 Aug 2015 20:35:50 +0200
> > From: Berk Hess <hess at kth.se>
> > To: gmx-developers at gromacs.org
> > Subject: Re: [gmx-developers] Increase the scope of dd domains
> > Message-ID: <55E0AA06.8020404 at kth.se>
> > Content-Type: text/plain; charset="windows-1252"; Format="flowed"
> >
> > Hi,
> >
> > If rlist=1, you should have access to all pairs up to 1 nm.
> >
> > One more question:
> > You do check/print the pairs on all ranks, not only on the master rank?
> >
> > Cheers,
> >
> > Berk
> >
> > On 08/28/2015 08:15 PM, Xingcheng Lin wrote:
> > > Hi Berk,
> > >
> > > Yes, I think I was calling that after the coordinate communication.
> > > The place where I code it in is after the command:
> > >
> > > if (bDoForces)
> > > {
> > > post_process_forces(fplog, cr, step, nrnb, wcycle,
> > > top, box, x, f, vir_force, mdatoms, graph, fr, vsite,
> > > flags);
> > > }
> > >
> > > After that I use a
> > > if(bNS){
> > > if (DOMAINDECOMP(cr)){
> > >
> > > to make sure it is done every neighboring list step.
> > >
> > > Is this correct? BTW, I was coding that in gromacs4.6.3 version (this
> > > is for historical reason because I have coded other unrelated things
> > > in other files).
> > >
> > > I have checked that the ga2la_get_home in the first loop find all ai
> > > atoms, but the second loop ga2la_get missed some aj atoms. One of the
> > > reasons I guess is because the missing atom pairs are too far apart
> > > (but less than 6 angstroms) in Cartesian space from each other, so
> > > that even by including all cells of the home node it still could not
> > > find it. That could be a problem, but I don't know exactly how -dd
> > > find this pair partner in its neighboring cells.
> > >
> > > Thank you,
> > > Xingcheng
> > >
> >
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <
> >
> http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150828/0da41613/attachment-0001.html
> > >
> >
> > ------------------------------
> >
> > --
> > Gromacs Developers mailing list
> >
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> > posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> > or send a mail to gmx-developers-request at gromacs.org.
> >
> > End of gromacs.org_gmx-developers Digest, Vol 136, Issue 25
> > ***********************************************************
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150829/80175e7b/attachment-0001.html
> >
>
> ------------------------------
>
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> or send a mail to gmx-developers-request at gromacs.org.
>
> End of gromacs.org_gmx-developers Digest, Vol 136, Issue 26
> ***********************************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150901/55ac443b/attachment.html>
More information about the gromacs.org_gmx-developers
mailing list