[gmx-users] Parallel pulling with Gromacs 4.0.7: COMM mode problem

Chris Neale chris.neale at utoronto.ca
Tue Mar 30 19:29:49 CEST 2010


Dear Aykut:

1. Did you see the log file message:

"comm-mode angular will give incorrect results when the comm group 
partially crosses a periodic boundary"

2. You say "Actually you might be right about the domain decomposition", 
but it seems like you didn't run it on gmx 4 in serial or with particle 
decomposition.

I wish you the best of luck, I'm out of ideas here.

Chris.

-- original message --

Hi

Actually you might be right about the domain decomposition


G3 pull.pdo output file on single machine

focus on the 2nd and 3rd columns which are x and y positions of the
surface: almost *unchanged* as expected for COMM_grps=surface option
*************
20000.000000 3.149521 1.576811 5.770928
7.149521 1.874820 1.676811
20000.201172 3.149521 1.576812 5.761463
7.149541 1.880746 1.676812
20000.400391 3.149520 1.576813 5.771702
7.149560 1.867692 1.676813
20000.601562 3.149519 1.576813 5.797871
7.149579 1.879650 1.676813
20000.800781 3.149518 1.576812 5.794115
7.149598 1.887728 1.676812
20001.000000 3.149517 1.576813 5.778761
7.149617 1.870823 1.676813
20001.201172 3.149518 1.576815 5.783334
7.149638 1.849283 1.676815
20001.400391 3.149517 1.576815 5.780031
7.149658 1.877158 1.676815
.....
.....
39999.402344 3.149799 1.576911 2.249830
9.149739 1.604563 1.676911
39999.601562 3.149797 1.576911 2.209385
9.149757 1.622380 1.676911
39999.800781 3.149792 1.576911 2.215503
9.149773 1.653246 1.676911
40000.000000 3.149791 1.576912 2.221903
9.149791 1.659781 1.676912



G4 pull.xvg output (in parellel), 2nd and 3rd columns which are x and y
positions of the surface: *changing*, contradiction to COMM_grps=surface
option

*********
0.4000 3.1498 2.997 -0.391131 -0.331925
0.8000 3.14903 2.99499 -0.391976 -0.346309
1.2000 3.14753 2.99846 -0.372158 -0.407621
1.6000 3.14635 3.00695 -0.337084 -0.422437
2.0000 3.14465 3.00585 -0.306999 -0.474991
2.4000 3.14365 3.00408 -0.30164 -0.48047
2.8000 3.14338 3.00447 -0.285076 -0.483861
3.2000 3.14361 3.00119 -0.226717 -0.460955
........
..........
2838.0000 3.20024 0.662325 1.7185 0.986139
2838.4000 3.19435 0.661913 1.74023 1.0404
2838.8000 3.18835 0.666171 1.8073 1.02766
2839.2000 3.18264 0.658261 1.81687 0.999429
2839.6000 3.17766 0.668439 1.82782 1.05693


here is the log file for G4 (pulling) run in parallel

********************************
Initializing Domain Decomposition on 32 nodes
Dynamic load balancing: auto
Will sort the charge groups at every domain (re)decomposition
Initial maximum inter charge-group distances:
two-body bonded interactions: 0.507 nm, LJ-14, atoms 5186 5197
multi-body bonded interactions: 0.507 nm, Proper Dih., atoms 5186 5197
Minimum cell size due to bonded interactions: 0.557 nm
Maximum distance for 5 constraints, at 120 deg. angles, all-trans: 0.200 nm
Estimated maximum distance required for P-LINCS: 0.200 nm
Guess for relative PME load: 0.20
Will use 24 particle-particle and 8 PME only nodes
This is a guess, check the performance at the end of the log file
Using 8 separate PME nodes
Scaling the initial minimum size with 1/0.8 (option -dds) = 1.25
Optimizing the DD grid for 24 cells with a minimum initial size of 0.697 nm
The maximum allowed number of cells is: X 9 Y 4 Z 9
Domain decomposition grid 4 x 2 x 3, separate PME nodes 8

comm-mode angular will give incorrect results when the comm group
partially crosses a periodic boundary
Interleaving PP and PME nodes
This is a particle-particle only node

Domain decomposition nodeid 0, coordinates 0 0 0

Table routines are used for coulomb: TRUE
Table routines are used for vdw: FALSE
Will do PME sum in reciprocal space.

-------- -------- --- Thank You --- -------- --------
Using a Gaussian width (1/beta) of 0.25613 nm for Ewald
Cut-off's: NS: 0.8 Coulomb: 0.8 LJ: 0.8
System total charge: -0.000
Generated table with 3600 data points for Ewald.
Tabscale = 2000 points/nm
Generated table with 3600 data points for LJ6.
Tabscale = 2000 points/nm
Generated table with 3600 data points for LJ12.
Tabscale = 2000 points/nm
Generated table with 3600 data points for 1-4 COUL.
Tabscale = 2000 points/nm
Generated table with 3600 data points for 1-4 LJ6.
Tabscale = 2000 points/nm
Generated table with 3600 data points for 1-4 LJ12.
Tabscale = 2000 points/nm

Enabling SPC water optimization for 3021 molecules.

Configuring nonbonded kernels...


Removing pbc first time

Will apply umbrella COM pulling in geometry 'position'
between a reference group and 1 group
Pull group 0: 5181 atoms, mass 56947.551
Pull group 1: 13 atoms, mass 116.120

Initializing Parallel LINear Constraint Solver



Linking all bonded interactions to atoms
There are 85833 inter charge-group exclusions,
will use an extra communication step for exclusion forces for PME

The initial number of communication pulses is: X 1 Y 1 Z 1
The initial domain decomposition cell size is: X 1.58 nm Y 1.58 nm Z 2.23 nm

The maximum allowed distance for charge groups involved in interactions is:
non-bonded interactions 0.800 nm
(the following are initial values, they could change due to box deformation)
two-body bonded interactions (-rdd) 0.800 nm
multi-body bonded interactions (-rdd) 0.800 nm
atoms separated by up to 5 constraints (-rcon) 1.575 nm

When dynamic load balancing gets turned on, these settings will change to:
The maximum number of communication pulses is: X 1 Y 1 Z 1
The minimum size for domain decomposition cells is 0.800 nm
The requested allowed shrink of DD cells (option -dds) is: 0.80
The allowed shrink of domain decomposition cells is: X 0.51 Y 0.51 Z 0.36
The maximum allowed distance for charge groups involved in interactions is:
non-bonded interactions 0.800 nm
two-body bonded interactions (-rdd) 0.800 nm
multi-body bonded interactions (-rdd) 0.800 nm
atoms separated by up to 5 constraints (-rcon) 0.800 nm


Making 3D domain decomposition grid 4 x 2 x 3, home cell index 0 0 0

Center of mass motion removal mode is Angular
We have the following groups for center of mass motion removal:
0: DIAM




More information about the gromacs.org_gmx-users mailing list