[gmx-users] Free energy calculations and speed problems

[Justin A. Lemkul]

Hi all,

I'm running some free energy calculations (in vacuo and in solution) for a set
of molecules I am trying to parameterize.  All the molecules are fairly small,
about 30-35 atoms in each.  The solvated calculations go smoothly, and rather
quickly (~5 hr/ns on two nodes).  I am running into a problem when I do the in
vacuo calculations.  Since I am constraining all bonds, I am running things on
a local machine in the lab (single core AMD64 Opteron).  I get as far as NVT
equilibration (removing charges only) before I give up and kill it, since it
takes so long.  A 10-ps NVT run takes about 20 minutes, but that adds up to
over 5 days for my 5-ns production run!  The machine is no slouch; I have used
it to run other small NVT/NPT/MD runs with good performance.  I get similar
performance on another machine that is even faster (a single CPU of the 2-CPU
node on our supercomputer).  The installations of Gromacs on both are fine, as
far as I can tell, and both have been used extensively.

I am using Gromacs version 3.3.3, and I have attached my .mdp file and the tail
end of my log file for reference.  If anyone notices anything obvious that I'm
doing wrong, I would be grateful if you pointed it out!  I see in the log of
the slow run (in vacuo) that the 3D-FFT calculation is dominating the CPU
usage.  Is that indicative of any problem?

Thanks for your attention.

-Justin

---My nvt.mdp file---

; NVT equilibration for free energy calculation
title           = 10-ps NVT equilibration for free energy calculation
cpp             = /usr/bin/cpp
include         = -I/home/jalemkul/Gromacs/Free_Energy/Topologies/
; Run control parameters
integrator      = sd
; start time and timestep in ps
tinit           = 0
dt              = 0.002         ; 0.002 ps = 2 fs
nsteps          = 5000          ; 10 ps total
; COM motion removal
nstcomm         = 100
; Output frequency
nstlog          = 500
nstenergy       = 100
nstxout         = 0
nstvout         = 0
nstfout         = 0
; Output for xtc file
nstxtcout       = 5000
xtc-precision   = 1000
; Neighborsearching parameters
nstlist         = 5
ns_type         = grid
; Temperature coupling
tcoupl          = Berendsen
tc_grps         = system
tau_t           = 0.1
ref_t           = 298
; Constraint options
constraints     = all-bonds
constraint-algorithm     = lincs
unconstrained-start      = no
; Relative tolerance of shake
shake-tol       = 0.0001
; LINCS parameters
lincs-order     = 12
lincs-warnangle = 30
; Periodic boundary conditions
pbc             = xyz
; nblist cut-off
rlist           = 0.9
; Electrostatics
coulombtype     = pme
rcoulomb        = 0.9
; vdw cut-off lengths
rvdw            = 1.4
; FFT parameters
fourierspacing  = 0.12
fourier_nx      = 0
fourier_ny      = 0
fourier_nz      = 0
; PME parameters
pme_order       = 4
ewald_rtol      = 1e-5
optimize_fft    = yes
; Generate velocities is on
gen_vel         = yes
gen_temp        = 298
gen_seed        = -1
; Free energy control stuff
free_energy     = yes
init_lambda     = 0.0
delta_lambda    = 0
sc_alpha        = 0
sc_power        = 1.0
sc_sigma        = 0

---end of nvt.mdp file---

---Tail of in vacuo log file---

       M E G A - F L O P S   A C C O U N T I N G

   RF=Reaction-Field  FE=Free Energy  SCFE=Soft-Core/Free Energy
   T=Tabulated        W3=SPC/TIP3p    W4=TIP4p (single or pairs)
   NF=No Forces

 Computing:                        M-Number         M-Flops  % of Flops
-----------------------------------------------------------------------
 LJ                                0.005001        0.165033     0.0
 Free energy innerloop             1.504697      225.704550     0.0
 Outer nonbonded loop              0.371929        3.719290     0.0
 1,4 nonbonded interactions        0.345069       31.056210     0.0
 Spread Q Bspline                 20.484096       40.968192     0.0
 Gather F Bspline                 20.484096      245.809152     0.0
 3D-FFT                       227370.585024  1818964.680192    90.5
 Solve PME                      2964.112704   189703.213056     9.4
 NS-Pairs                          0.078078        1.639638     0.0
 Reset In Box                      0.032032        0.288288     0.0
 Shift-X                           0.320064        1.920384     0.0
 CG-CoM                            0.012012        0.348348     0.0
 Angles                            0.255051       41.573313     0.0
 Propers                           0.055011       12.597519     0.0
 Impropers                         0.135027       28.085616     0.0
 Virial                            0.385077        6.931386     0.0
 Update                            0.320064        9.921984     0.0
 Stop-CM                           0.001600        0.016000     0.0
 Calc-Ekin                         0.160064        4.321728     0.0
 Lincs                             0.340136       20.408160     0.0
 Lincs-Mat                        14.285712       57.142848     0.0
 Constraint-Vir                    0.170102        4.082448     0.0
-----------------------------------------------------------------------
 Total                                       2009404.593335   100.0
-----------------------------------------------------------------------

               NODE (s)   Real (s)      (%)
       Time:   1142.590   1143.000    100.0
                       19:02
               (Mnbf/s)   (GFlops)   (ns/day)  (hour/ns)
Performance:      0.000      1.759      0.756     31.739

---end of log file---

========================================

Justin A. Lemkul
Graduate Research Assistant
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul at vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin/

========================================