[gmx-developers] Increase the scope of dd domains
Xingcheng Lin
linxingcheng50311 at gmail.com
Fri Aug 28 17:46:00 CEST 2015
Hi Berk,
The 1D dd where I used 2 MPI processes with -dd 1 1 2 can only work if I do
not turn on the -dlb yes. It seems the problem will always exist if I have
atom pairs separated in two domains.
You mentioned I could get all pairs by only looking at home atoms for the
i-atoms. Does it mean I can even make the code like this, by changing
ga2la_get for aj to ga2la_get_home?
// NP is number of tested pairs recorded in an array;
// If atom ai is one of the home atoms in a node, we start testing if aj is
available by the same node;
// rlist = 1.0nm, Coulombtype = Cut-off, cutoff-scheme = Verlet, ns_type
= grid, pbc = xyz;
int Npairs = 0;
for (i=0; i<NP; ++i)
{
int ai = datai[i] ;
int localai;
// Examine if ai it is within the home atoms index;
if (ga2la_get_home(cr->dd->ga2la, ai, &localai))
{
aj = dataj[i] ;
// Examine if aj is available by local node;;
if (ga2la_get_home(cr->dd->ga2la, aj, &localaj))
{
FILE *fp;
fp = fopen("pairs.dat","a");
fprintf(fp, "%d\t%d\n", ai, aj);
fclose(fp);
r_N = 0.6 ;
pbc_dx(&pbc, x[localai], x[localaj], dx) ;
dr2 = iprod( dx, dx ) ;
dr = sqrt( dr2 ) ;
if (dr < r_N)
{
Npairs += 1;
}
}
}
}
On Fri, Aug 28, 2015 at 2:22 AM, <
gromacs.org_gmx-developers-request at maillist.sys.kth.se> wrote:
> Send gromacs.org_gmx-developers mailing list submissions to
> gromacs.org_gmx-developers at maillist.sys.kth.se
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>
> or, via email, send a message with subject or body 'help' to
> gromacs.org_gmx-developers-request at maillist.sys.kth.se
>
> You can reach the person managing the list at
> gromacs.org_gmx-developers-owner at maillist.sys.kth.se
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of gromacs.org_gmx-developers digest..."
>
>
> Today's Topics:
>
> 1. Increase the scope of dd domains (Xingcheng Lin)
> 2. Re: energy group computation (Berk Hess)
> 3. Re: dhdl expanded ensemble (Berk Hess)
> 4. Re: Increase the scope of dd domains (Berk Hess)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 27 Aug 2015 23:35:30 -0500
> From: Xingcheng Lin <linxingcheng50311 at gmail.com>
> To: gromacs.org_gmx-developers at maillist.sys.kth.se
> Subject: [gmx-developers] Increase the scope of dd domains
> Message-ID:
> <CAPugp-HQjXb8hYKJO9i-1fMSfpzQxe=
> qZCOo8tPwPQrFraKEmw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
>
> I am trying to encode a "on the fly" calculation for the number of atom
> pairs within some distance (for example here, 6 angstroms).
>
> The problem happens when doing parallel domain decomposition. Specifically,
> when two particles reside in different domains, gromacs is not able to
> recognize this pair. I am trying to use ga2la_get_home and ga2la_get for
> calculation, here is the code:
>
> // NP is number of tested pairs recorded in an array;
> // If atom ai is one of the home atoms in a node, we start testing if aj is
> available by the same node;
> // rlist = 1.0nm, Coulombtype = Cut-off, cutoff-scheme = Verlet, ns_type
> = grid, pbc = xyz;
>
> int Npairs = 0;
> for (i=0; i<NP; ++i)
> {
> int ai = datai[i] ;
> int localai;
> // Examine if ai it is within the home atoms index;
> if (ga2la_get_home(cr->dd->ga2la, ai, &localai))
> {
> aj = dataj[i] ;
> // Examine if aj is available by local node;;
> if (ga2la_get(cr->dd->ga2la, aj, &localaj, &localcell))
> {
> FILE *fp;
> fp = fopen("pairs.dat","a");
>
> fprintf(fp, "%d\t%d\n", ai, aj);
> fclose(fp);
>
> r_N = 0.6 ;
> pbc_dx(&pbc, x[localai], x[localaj], dx) ;
> dr2 = iprod( dx, dx ) ;
> dr = sqrt( dr2 ) ;
> if (dr < r_N)
> {
> Npairs += 1;
> }
> }
> }
> }
>
> The results show some atom pairs, while atoms less than 6 angstroms apart,
> are missing here. I increase rlist, but it doesn't help. Is there any way
> to increase the scope of atoms available in each domain so that the missing
> atoms can be detected?
>
> Or are there any mistakes out of my notice in this code?
>
> Thank you,
> Xingcheng
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150827/208ffc05/attachment.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Fri, 28 Aug 2015 09:04:56 +0200
> From: Berk Hess <hess at kth.se>
> To: gmx-developers at gromacs.org
> Subject: Re: [gmx-developers] energy group computation
> Message-ID: <55E00818.1060601 at kth.se>
> Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>
> Hi,
>
> This is a use, not a development question. Please post such questions to
> gmx-users.
>
> But the quick answer is that although it's technically possible to
> calculate this number, this number is useless (and indeed high).
>
> Cheers,
>
> Berk
>
> On 2015-08-27 19:00, ANTHONY C MANSON wrote:
> > Hello:
> >
> > I am trying to estimate the polar solvation energy of a 76 residue
> > protein at standard conditions. I am using the gromacs 4.5.3 dp release.
> > I have set up a simulation with the solvated (using spce water) protein.
> > I have defined two energygrps: Protein SOL in the *.mdp file.
> > I have run the simulation and have then run g_energy on the *.edr file.
> > I get the following choices from g_energy:
> >
> > 1 Bond 2 Angle 3 Proper-Dih. 4 Improper-Dih.
> > 5 LJ-14 6 Coulomb-14 7 LJ-(SR) 8 Disper.-corr.
> > 9 Coulomb-(SR) 10 Coul.-recip. 11 Potential 12
> > Kinetic-En.
> > 13 Total-Energy 14 Temperature 15 Pres.-DC 16 Pressure
> > 17 Vir-XX 18 Vir-XY 19 Vir-XZ 20 Vir-YX
> > 21 Vir-YY 22 Vir-YZ 23 Vir-ZX 24 Vir-ZY
> > 25 Vir-ZZ 26 Pres-XX 27 Pres-XY 28 Pres-XZ
> > 29 Pres-YX 30 Pres-YY 31 Pres-YZ 32 Pres-ZX
> > 33 Pres-ZY 34 Pres-ZZ 35 #Surf*SurfTen 36 Mu-X
> > 37 Mu-Y 38 Mu-Z
> > 39 Coul-SR:Protein-Protein 40 LJ-SR:Protein-Protein
> > 41 Coul-14:Protein-Protein 42 LJ-14:Protein-Protein
> > 43 Coul-SR:Protein-SOL 44 LJ-SR:Protein-SOL
> > 45 Coul-14:Protein-SOL 46 LJ-14:Protein-SOL
> > 47 Coul-SR:SOL-SOL 48 LJ-SR:SOL-SOL
> > 49 Coul-14:SOL-SOL 50 LJ-14:SOL-SOL
> > 51 T-System
> >
> > I pick Coul-SR:Protein-SOL as a component of the polar solvation energy.
> > I note that the energy magnitude seems extremely high:
> >
> > Energy Average Err.Est. RMSD Tot-Drift
> >
> -------------------------------------------------------------------------------
> > Coul-SR:Protein-SOL -8343.86 46 193.457 -192.93
> > (kJ/mol)
> >
> > This seems unreasonable to me. I wanted to understand how mdrun
> > was calculating this value. Perhaps it is accumulating energies over
> > many frames. What part of the code (force.c ...) handles this
> > calculation so I can verify its action.
> >
> > Also, I tried running g_enemat on this with the following groups.dat
> file.
> >
> > 2
> > Protein
> > SOL
> >
> > I use the command:
> >
> > g_enemat -f ub_short.edr -e 100
> >
> > and get:
> >
> > Opened ub_short.edr as double precision energy file
> > Will read groupnames from in! putfile
> > Read 2 groups
> > group 0WARNING! could not find group (null):Protein-Protein (0,0)in
> > energy file
> > WARNING! could not find group (null):Protein-SOL (0,1)in energy file
> > group 1WARNING! could not find group (null):SOL-SOL (1,1)in energy file
> >
> > Will select half-matrix of energies with 6 elements
> > Read frame: 1000, Time: 100.000e 100.000
> > Will build energy half-matrix of 2 groups, 6 elements, over 1001 frames
> > Segmentation fault
> >
> >
> > Cheers
> > Anthony Manson Ph.D.
> >
> >
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150828/17c227f6/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Fri, 28 Aug 2015 09:09:49 +0200
> From: Berk Hess <hess at kth.se>
> To: gmx-developers at gromacs.org, "michael.shirts at virginia.edu"
> <michael.shirts at virginia.edu>
> Subject: Re: [gmx-developers] dhdl expanded ensemble
> Message-ID: <55E0093D.2000804 at kth.se>
> Content-Type: text/plain; charset=windows-1252; format=flowed
>
> No. To be more precise dhdl is only (fully) calculated at each step
> where free energies are calculated, i.e. every nstdhdl steps.
> To find where the final values are stored, it's often easiest to look in
> mdebin.c, in this case at line 1244.
>
> Cheers,
>
> Berk
>
> On 2015-08-28 03:52, Shirts, Michael R. (mrs5pt) wrote:
> > Dhdl is automatically calculated at every step if free energies are
> > calculated. Look in sum_dhdl -- difference components are stored in
> > enerd->term.
> >
> > Unless it's a new development you would rather keep private for now, you
> > should consider posting the proposed enhancement on redmine. You would
> > likely get some tips about how to do it in a way that could be more
> easily
> > included in later gromcs versions.
> >
> > Best,
> > ~~~~~~~~~~~~
> > Michael Shirts
> > Associate Professor
> > Department of Chemical Engineering
> > University of Virginia
> > michael.shirts at virginia.edu
> > (434) 243-1821
> >
> >
> > From: <Mirabzadeh>, "Christopher (mira2978 at vandals.uidaho.edu)"
> > <mira2978 at vandals.uidaho.edu>
> > Reply-To: "gmx-developers at gromacs.org" <gmx-developers at gromacs.org>
> > Date: Thursday, August 27, 2015 at 12:20 PM
> > To: "gromacs.org_gmx-developers at maillist.sys.kth.se"
> > <gromacs.org_gmx-developers at maillist.sys.kth.se>
> > Subject: [gmx-developers] dhdl expanded ensemble
> >
> >
> >
> > Hello,
> >
> > What I'm trying to do;
> >
> > I'm attempting to add functionality to expanded ensemble.
> >
> >
> > What I've done so far;
> >
> > I've added functionality to accept my mdp options and found pre-written
> > functions to take advantage of.
> >
> > Where I'm stuck;
> >
> > I need the derivative of the Hamiltonian with respect to lambda at every
> > md step. I'm having a hard time narrowing down this variable. I know it's
> > calculated and I see in mdebin.c there is a line that says "/* store_dh
> is
> > dE */". Is the derivative "store_dhdl[]=enerd->term[F_DVDL]"?
> > Or, is this already being called somewhere in expanded.c that I can
> take
> > advantage of, maybe dfhist?
> >
> > Is the dhdl automatically calculated? Where is it stored? If dhdl isn?t
> > being calculated automatically, is there an mdp option or mdrun flag that
> > I need?
> >
> > I appreciate any advice that can be offered.
> >
> >
> > -ChrisM
> >
> > Mirabzadeh, Christopher
> > Graduate Research Assistant/Physics Instructor
> > Department of Physics
> > University of Idaho
> > Moscow, Id
> > (509)339-5647
> >
> >
> >
> >
> >
>
>
>
> ------------------------------
>
> Message: 4
> Date: Fri, 28 Aug 2015 09:20:44 +0200
> From: Berk Hess <hess at kth.se>
> To: gmx-developers at gromacs.org
> Subject: Re: [gmx-developers] Increase the scope of dd domains
> Message-ID: <55E00BCC.4040506 at kth.se>
> Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>
> Hi,
>
> Gromacs uses a neutral-territory eighth-shell domain decomposition
> method. So you can get all pairs by only looking at home-atoms for the
> i-atoms. You code should work though for 1D decomposition, is that the
> case?
>
> To get this to work with 2D or 3D decomposition, you need to replace
> ga2la_get_home by ga2la_get and then only consider i-j pairs where the
> cell values are in the cell pair list given by dd->comm->zones. cell_i
> should be < zones->nizone and zones->izone[cell_i].j0 <= cell_j <
> zones->izone[cell_j].j1 When the DD cells get smaller than the cutoff,
> you need to use cell % zones->n.
>
> Note that since the ga2la_get calls are expensive, such an approach is
> only efficient when you have only few atoms / atom pairs.
> For improving performance, it can help to first lookup locala and cell
> for all atoms and store them and then do the pair double loop after
> that. And instead of using sqrt, check dr2 < r_N*r_N.
>
> Cheers,
>
> Berk
>
> On 2015-08-28 06:35, Xingcheng Lin wrote:
> > Hi,
> >
> > I am trying to encode a "on the fly" calculation for the number of
> > atom pairs within some distance (for example here, 6 angstroms).
> >
> > The problem happens when doing parallel domain decomposition.
> > Specifically, when two particles reside in different domains, gromacs
> > is not able to recognize this pair. I am trying to use ga2la_get_home
> > and ga2la_get for calculation, here is the code:
> >
> > // NP is number of tested pairs recorded in an array;
> > // If atom ai is one of the home atoms in a node, we start testing if
> > aj is available by the same node;
> > // rlist = 1.0nm, Coulombtype = Cut-off, cutoff-scheme = Verlet,
> > ns_type = grid, pbc = xyz;
> >
> > int Npairs = 0;
> > for (i=0; i<NP; ++i)
> > {
> > int ai = datai[i] ;
> > int localai;
> > // Examine if ai it is within the home atoms index;
> > if (ga2la_get_home(cr->dd->ga2la, ai, &localai))
> > {
> > aj = dataj[i] ;
> > // Examine if aj is available by local node;;
> > if (ga2la_get(cr->dd->ga2la, aj, &localaj, &localcell))
> > {
> > FILE *fp;
> > fp = fopen("pairs.dat","a");
> >
> > fprintf(fp, "%d\t%d\n", ai, aj);
> > fclose(fp);
> >
> > r_N = 0.6 ;
> > pbc_dx(&pbc, x[localai], x[localaj], dx) ;
> > dr2 = iprod( dx, dx ) ;
> > dr = sqrt( dr2 ) ;
> > if (dr < r_N)
> > {
> > Npairs += 1;
> > }
> > }
> > }
> > }
> >
> > The results show some atom pairs, while atoms less than 6 angstroms
> > apart, are missing here. I increase rlist, but it doesn't help. Is
> > there any way to increase the scope of atoms available in each domain
> > so that the missing atoms can be detected?
> >
> > Or are there any mistakes out of my notice in this code?
> >
> > Thank you,
> > Xingcheng
> >
> >
> >
> >
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150828/682fbb3e/attachment.html
> >
>
> ------------------------------
>
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> or send a mail to gmx-developers-request at gromacs.org.
>
> End of gromacs.org_gmx-developers Digest, Vol 136, Issue 21
> ***********************************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150828/1d99e7f4/attachment-0001.html>
More information about the gromacs.org_gmx-developers
mailing list