[gmx-users] GPU crashes

[Justin A. Lemkul]

Hi All,

I'm wondering if anyone has experienced what I'm seeing with Gromacs 4.5.5 on 
GPU.  It seems that certain systems fail inexplicably.  The system I am working 
with is a heterodimeric protein complex bound to DNA.  After about 1 ns of 
simulation time using mdrun-gpu, all the energies become NaN.  The simulations 
don't stop, they just carry on merrily producing nonsense.  I would love to see 
some action regarding http://redmine.gromacs.org/issues/941 for this reason ;)

I ran simulations of each of the components of the system individually - each 
protein alone, and DNA - to try to track down what might be causing this 
problem.  The DNA simulation is perfectly stable out to 10 ns, but each protein 
fails within 2 ns.  Each protein has two domains with a flexible linker, and it 
seems that as soon as the linker flexes a bit, the simulations go poof. 
Well-behaved proteins like lysozyme and DHFR (from the benchmark set) seem fine, 
but anything that twitches even a small amount fails.  This is very unfortunate 
for us, as we are hoping to see domain motions on a feasible time scale using 
implicit solvent on GPU hardware.

Has anyone seen anything like this?  Our Gromacs implementation is being run on 
an x86_64 Linux system with Tesla S2050 GPU cards.  The CUDA version is 3.1 and 
Gromacs is linked against OpenMM-2.0.  An .mdp file is appended below.  I have 
also tested finite values for cutoffs, but the results were worse (failures 
occurred more quickly).

I have not been able to use the latest git version of Gromacs to test whether 
anything has been fixed, but will post separately to gmx-developers regarding 
the reasons for that soon.

-Justin

=== md.mdp ===

title           = Implicit solvent test
; Run parameters
integrator      = sd
dt                  = 0.002
nsteps          = 5000000   ; 10000 ps (10 ns)
nstcomm         = 1
comm_mode   = angular       ; non-periodic system
; Output parameters
nstxout         = 0
nstvout         = 0
nstfout         = 0
nstxtcout   = 1000          ; every 2 ps
nstlog      = 5000          ; every 10 ps
nstenergy   = 1000          ; every 2 ps
; Bond parameters
constraint_algorithm    = lincs
constraints             = all-bonds
continuation    = no            ; starting up
; required cutoffs for implicit
nstlist         = 0
ns_type         = grid
rlist           = 0
rcoulomb        = 0
rvdw            = 0
; cutoffs required for qq and vdw
coulombtype     = cut-off
vdwtype     = cut-off
; temperature coupling
tc_grps         = System
tau_t           = 1.0   ; inverse friction coefficient for Langevin (ps^-1)
ref_t           = 310
; Pressure coupling is off
Pcoupl          = no
; Generate velocities is on
gen_vel         = yes
gen_temp        = 310
gen_seed        = 173529
; Periodic boundary conditions are off for implicit
pbc                 = no
; Free energy must be off to use all-vs-all kernels
free_energy = no    ; default, but just for the sake of being pedantic
; Implicit
implicit_solvent    = GBSA
gb_algorithm        = OBC
rgbradii            = 0

-- 
========================================

Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin

========================================