[gmx-developers] Increase the scope of dd domains
Xingcheng Lin
linxingcheng50311 at gmail.com
Sat Aug 29 18:24:19 CEST 2015
Even when I choose
cutoff-scheme=Verlet
cutoff-buffer-drift=-1
rlist=1.5
some pairs are still missing.
I do print them out from different node and use MPI_Reduce to combine them
together. That is where I found out it is not a problem if all pairs sit at
one node, while it shows up when pairs starts to be distributed into two
nodes.
To be clear, I put this code at the end of *do_force* function, after the
*post_process_forces* function. I think the coordinates have been
communicated up to this point, am I correct?
if (bDoForces)
{
post_process_forces(fplog, cr, step, nrnb, wcycle,
top, box, x, f, vir_force, mdatoms, graph, fr, vsite,
flags);
}
if(bNS)
{
if (DOMAINDECOMP(cr))
{
int my_rank;
int npes;
int globalQC;
int localQC;
MPI_Comm_size(MPI_COMM_WORLD, &npes);
MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
if (my_rank == 0)
{
globalQC = 0;
}
// Barrier for the initialization;
MPI_Barrier(MPI_COMM_WORLD);
localQC = calQ_dd( x, f, step, cr, inputrec, box );
// Barrier for Qcontacts accumulation;
MPI_Barrier(MPI_COMM_WORLD);
MPI_Reduce(&localQC, &globalQC, 1, MPI_INT, MPI_SUM, 0,
MPI_COMM_WORLD);
...
Then I print out globalQC, it shows some pairs are missing.
The code for calQdd is same as before (struct data is known statically):
int calQdd ( rvec x[], rvec f[], int step, t_commrec *cr, t_inputrec *ir,
matrix box )
{
int Npairs = 0;
int i, ai, aj;
/* for domain decomposition ---------------*/
int localai, localaj;
int localcell;
/* for domain decomposition ---------------*/
real r_N, r_N2, dr, dr2 ;
rvec dx ;
/* Set the pbc ----------------------------*/
t_pbc pbc;
set_pbc(&pbc, ir->ePBC, box);
FILE *fp;
fp = fopen("pairs.dat","a");
if(fp == NULL){
fprintf(stderr, "Can't open input file !\n");
exit(1);
}
if ( NP > 0 )
{
for (i=0; i<NP; ++i)
{
ai = data.first[i] ;
if (ga2la_get_home(cr->dd->ga2la, ai, &localai))
{
aj = data.second[i] ;
if (ga2la_get(cr->dd->ga2la, aj, &localaj, &localcell))
{
fprintf(fp, "%d\t%d\n", ai, aj);
r_N = data.r_native[i] ;
r_N2 = r_N * r_N;
pbc_dx(&pbc, x[localai], x[localaj], dx) ;
dr2 = iprod( dx, dx ) ;
if (dr2 < r_N2)
{
Npairs += 1;
}
}
}
}
}
fclose(fp);
return Npairs;
}
As you can see, here I also print out the pairs recognized. Though I
increase rlist=1.5 still some pairs are missing (though their mutual
distance is less than 6 angstroms).
Thanks,
Xingcheng
On Sat, Aug 29, 2015 at 5:00 AM, <
gromacs.org_gmx-developers-request at maillist.sys.kth.se> wrote:
> Send gromacs.org_gmx-developers mailing list submissions to
> gromacs.org_gmx-developers at maillist.sys.kth.se
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>
> or, via email, send a message with subject or body 'help' to
> gromacs.org_gmx-developers-request at maillist.sys.kth.se
>
> You can reach the person managing the list at
> gromacs.org_gmx-developers-owner at maillist.sys.kth.se
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of gromacs.org_gmx-developers digest..."
>
>
> Today's Topics:
>
> 1. Re: Increase the scope of dd domains (Berk Hess)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 28 Aug 2015 20:35:50 +0200
> From: Berk Hess <hess at kth.se>
> To: gmx-developers at gromacs.org
> Subject: Re: [gmx-developers] Increase the scope of dd domains
> Message-ID: <55E0AA06.8020404 at kth.se>
> Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>
> Hi,
>
> If rlist=1, you should have access to all pairs up to 1 nm.
>
> One more question:
> You do check/print the pairs on all ranks, not only on the master rank?
>
> Cheers,
>
> Berk
>
> On 08/28/2015 08:15 PM, Xingcheng Lin wrote:
> > Hi Berk,
> >
> > Yes, I think I was calling that after the coordinate communication.
> > The place where I code it in is after the command:
> >
> > if (bDoForces)
> > {
> > post_process_forces(fplog, cr, step, nrnb, wcycle,
> > top, box, x, f, vir_force, mdatoms, graph, fr, vsite,
> > flags);
> > }
> >
> > After that I use a
> > if(bNS){
> > if (DOMAINDECOMP(cr)){
> >
> > to make sure it is done every neighboring list step.
> >
> > Is this correct? BTW, I was coding that in gromacs4.6.3 version (this
> > is for historical reason because I have coded other unrelated things
> > in other files).
> >
> > I have checked that the ga2la_get_home in the first loop find all ai
> > atoms, but the second loop ga2la_get missed some aj atoms. One of the
> > reasons I guess is because the missing atom pairs are too far apart
> > (but less than 6 angstroms) in Cartesian space from each other, so
> > that even by including all cells of the home node it still could not
> > find it. That could be a problem, but I don't know exactly how -dd
> > find this pair partner in its neighboring cells.
> >
> > Thank you,
> > Xingcheng
> >
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150828/0da41613/attachment-0001.html
> >
>
> ------------------------------
>
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> or send a mail to gmx-developers-request at gromacs.org.
>
> End of gromacs.org_gmx-developers Digest, Vol 136, Issue 25
> ***********************************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150829/80175e7b/attachment.html>
More information about the gromacs.org_gmx-developers
mailing list