[gmx-users] FEP and loss of performance

Justin A. Lemkul jalemkul at vt.edu
Mon Apr 4 18:33:55 CEST 2011



Luca Bellucci wrote:
> Dear Chris and Justin
> Thank you for your precious suggestions 
> This is a test that i perform in a single machine with 8 cores 
> and gromacs 4.5.4.
> 
> I am trying  to enhance the  sampling of a protein using the decoupling scheme 
> of the free energy module of gromacs.  However when i decouple only the 
> protein, the protein collapsed. Because i simulated in NVT i thought that 
> this was an effect of the solvent. I was trying to decouple also the solvent 
> to understand the system behavior.
> 

Rather than suspect that the solvent is the problem, it's more likely that 
decoupling an entire protein simply isn't stable.  I have never tried anything 
that enormous, but the volume change in the system could be unstable, along with 
any number of factors, depending on how you approach it.

If you're looking for better sampling, REMD is a much more robust approach than 
trying to manipulate the interactions of huge parts of your system using the 
free energy code.

-Justin

>  I expected a loss of performance, but not so drastic. 
> Luca 
> 
>> Load balancing problems I can understand, but why would it take longer
>> in absolute time? I would have thought that some nodes would simple be
>> sitting idle, but this should not cause an increase in the overall
>> simulation time (15x at that!).
>>
>> There must be some extra communication?
>>
>> I agree with Justin that this seems like a strange thing to do, but
>> still I think that there must be some underlying coding issue (probably
>> one that only exists because of a reasonable assumption that nobody
>> would annihilate the largest part of their system).
>>
>> Chris.
>>
>> Luca Bellucci wrote:
>>> /  Hi Chris,
>> />/  thank for the suggestions,
>> />/  in the previous mail there is a mistake because
>> />/  couple-moltype = SOL (for solvent) and not "Protein_chaim_P".
>> />/  Now the problem of the load balance seems reasonable, because
>> />/  the water box is large ~9.0 nm.
>> /
>> Now your outcome makes a lot more sense.  You're decoupling all of the
>> solvent? I don't see how that is going to be physically stable or terribly
>> meaningful, but it explains your performance loss.  You're annihilating a
>> significant number of interactions (probably the vast majority of all the
>> nonbonded interactions in the system), which I would expect would cause
>> continuous load balancing issues.
>>
>> -Justin
>>
>>> /  However the problem exist and the performance loss is very high, so I
>>> have
>> />/  redone calculations with this command:
>> />/
>> />/  grompp -f
>> />/  md.mdp -c ../Run-02/confout.gro -t ../Run-02/state.cpt -p ../topo.top
>> -n ../index.ndx -o />/  md.tpr -maxwarn 1
>> />/
>> />/  mdrun -s md.tpr -o md
>> />/
>> />/  this is part of the md.mdp file:
>> />/
>> />/  ; Run parameters
>> />/  ; define          = -DPOSRES
>> />/  integrator	= md		;
>> />/  nsteps		= 1000 	;
>> />/  dt		= 0.002		;
>> />/  [..]
>> />/  free_energy    = yes     ; /no
>> />/  init_lambda    = 0.9
>> />/  delta_lambda   = 0.0
>> />/  couple-moltype = SOL    ; solvent water
>> />/  couple-lambda0 = vdw-q
>> />/  couple-lambda1 = none
>> />/  couple-intramol= yes
>> />/
>> />/  Result for free energy calculation
>> />/   Computing:         Nodes     Number     G-Cycles    Seconds     %
>> />/ 
>> ----------------------------------------------------------------------- />/
>>   Domain decomp.       8        126       22.050        8.3     0.1 />/  
>> DD comm. load          8         15        0.009        0.0     0.0 />/  
>> DD comm. bounds     8         12        0.031        0.0     0.0 />/  
>> Comm. coord.            8       1001       17.319        6.5     0.0 />/  
>> Neighbor search        8        127      436.569      163.7     1.1 />/  
>> Force                           8       1001    34241.576    12840.9   
>> 87.8 />/   Wait + Comm. F        8       1001       19.486        7.3    
>> 0.0 />/   PME mesh                  8       1001     4190.758     1571.6   
>> 10.7 />/   Write traj.                  8          7        1.827       
>> 0.7     0.0 />/   Update                      8       1001       12.557    
>>    4.7     0.0 />/   Constraints               8       1001       26.496   
>>     9.9     0.1 />/   Comm. energies      8       1002       10.710       
>> 4.0     0.0 />/   Rest                   8                  25.142       
>> 9.4     0.1 />/ 
>> ----------------------------------------------------------------------- />/
>>   Total                  8               39004.531    14627.1   100.0 />/ 
>> ----------------------------------------------------------------------- />/
>>  -----------------------------------------------------------------------
>> />/   PME redist. X/F          8       3003     3479.771     1304.9     8.9
>> />/   PME spread/gather   8       4004      277.574      104.1     0.7 />/ 
>>  PME 3D-FFT               8       4004      378.090      141.8     1.0 />/ 
>>  PME solve                  8       2002       55.033       20.6     0.1
>> />/ 
>> ----------------------------------------------------------------------- />/
>>  	Parallel run - timing based on wallclock.
>> />/
>> />/                 NODE (s)   Real (s)      (%)
>> />/         Time:   1828.385   1828.385    100.0
>> />/                         30:28
>> />/                               (Mnbf/s)   (GFlops)   (ns/day)  (hour/ns)
>> />/  Performance:      3.115      3.223      0.095    253.689
>> />/
>> />/   I Switched off only the free_energy keyword and I redone the
>> calculation />/  I have:
>> />/   Computing:         Nodes     Number     G-Cycles    Seconds     %
>> />/ 
>> ----------------------------------------------------------------------- />/
>>   Domain decomp.      8         77       10.975        4.1     0.6 />/   DD
>> comm. load         8          1        0.001        0.0     0.0 />/   Comm.
>> coord.           8       1001       14.480        5.4     0.8 />/  
>> Neighbor search       8         78      136.479       51.2     7.3 />/  
>> Force                         8       1001     1141.115      427.9    61.3
>> />/   Wait + Comm. F      8       1001       17.845        6.7     1.0 />/ 
>>  PME mesh                8       1001      484.581      181.7    26.0 />/  
>> Write traj.               8          5        1.221        0.5     0.1 />/ 
>>  Update                   8       1001        9.976        3.7     0.5 />/ 
>>  Constraints            8       1001       20.275        7.6     1.1 />/  
>> Comm. energies     8        992        5.933        2.2     0.3 />/   Rest 
>>                        8                  19.670        7.4     1.1 />/ 
>> ----------------------------------------------------------------------- />/
>>   Total                  8                1862.552      698.5   100.0 />/ 
>> ----------------------------------------------------------------------- />/
>>  -----------------------------------------------------------------------
>> />/   PME redist. X/F        8       2002       92.204       34.6     5.0
>> />/   PME spread/gather      8       2002      192.337       72.1    10.3
>> />/   PME 3D-FFT             8       2002      177.373       66.5     9.5
>> />/   PME solve              8       1001       22.512        8.4     1.2
>> />/ 
>> ----------------------------------------------------------------------- />/
>>  	Parallel run - timing based on wallclock.
>> />/
>> />/                 NODE (s)   Real (s)      (%)
>> />/         Time:     87.309     87.309    100.0
>> />/                         1:27
>> />/                           (Mnbf/s)   (GFlops)   (ns/day)  (hour/ns)
>> />/  Performance:    439.731     23.995      1.981     12.114
>> />/  Finished mdrun on node 0 Mon Apr  4 16:52:04 2011
>> />/
>> />/  Luca
>> />/
>> />/
>> />/
>> />/
>> />>/  If we accept your text at face value, then the simulation slowed down
>> />>/  by a factor of 1500%, certainly not the 16% of the load balancing.
>> />>/
>> />>/  Please let us know what version of gromacs and cut and paste your
>> />>/  cammands that you used to run gromacs (so we can verify that you ran
>> />>/  on the same number of processors) and cut and paste a diff of the
>> .mdp />>/  files (so that we can verify that you ran for the same number of
>> steps). />>/
>> />>/  You might be correct about the slowdown, but let's rule out some
>> other />>/  more obvious problems first.
>> />>/
>> />>/  Chris.
>> />>/
>> />>/  -- original message --
>> />>/
>> />>/
>> />>/  Dear all,
>> />>/  when I run a single free energy simulation
>> />>/  i noticed that there is a loss of performace with respect to
>> />>/  the normal MD
>> />>/
>> />>/  free_energy    = yes
>> />>/  init_lambda    = 0.9
>> />>/  delta_lambda   = 0.0
>> />>/  couple-moltype = Protein_Chain_P
>> />>/  couple-lambda0 = vdw-q
>> />>/  couple-lambda0 = none
>> />>/  couple-intramol= yes
>> />>/
>> />>/      Average load imbalance: 16.3 %
>> />>/      Part of the total run time spent waiting due to load imbalance:
>> 12.2 % />>/      Steps where the load balancing was limited by -rdd, -rcon
>> and/or -dds: />>/  X0 % Time:   1852.712   1852.712    100.0
>> />>/
>> />>/  free_energy    = no
>> />>/      Average load imbalance: 2.7 %
>> />>/      Part of the total run time spent waiting due to load imbalance:
>> 1.7 % />>/      Time:    127.394    127.394    100.0
>> />>/
>> />>/  It seems that the loss of performace is due in part to in the load
>> />>/  imbalance in the domain decomposition, however I tried to change
>> />>/  these keywords without benefit
>> />>/  Any comment is welcome.
>> /
> 
> 

-- 
========================================

Justin A. Lemkul
Ph.D. Candidate
ICTAS Doctoral Scholar
MILES-IGERT Trainee
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin

========================================



More information about the gromacs.org_gmx-users mailing list