[gmx-users] low performance 2 GTX 980+ Intel CPU Core i7-5930K 3.5 GHz (2011-3)

[Carlos Navarro Retamal]

Dear gromacs users,  
I just recently bought a workstation that posees two GTX 980 plus an i7 (Intel CPU Core i7-5930K 3.5 GHz (2011-3)).
In order to test it, i run a MD simulation of a system containing ~90k atoms.
These are the performances:

2 GPU’s (1 job):
34ns/day (each cards were working about ~40%)

1 GPU (Nº1)  (1 job):
37ns/day (~65% of performance)

1 GPU (Nº2) (1 job):
36ns/day (~65% of performance)

2 GPU’s (2 jobs simultaneously )
16ns/day and 16ns/day respectively. (~20% of performance each)

With respect to the last test, the .log file show the next message:

Force evaluation time GPU/CPU: 3.177 ms/5.804 ms = 0.547
For optimal performance this ratio should be close to 1!


NOTE: The GPU has >25% less load than the CPU. This imbalance cause
      performance loss.


So probably, since all the cpu is splitting between each job, the ratio GPU/CPU will be worse.

Is there a way i can solve this issue (is kind of sad that i’m getting a better performance with one GPU instead of two, since i saw that when i add a third or even a fourth one the performance start to decrease).
Here’s my .mdp file:

> title       = Protein-ligand complex MD simulation  
> ; Run parameters
> integrator  = md        ; leap-frog integrator
> nsteps      = 15000000    ; 2 * 1500000 = 30000 ps (30ns)
> dt          = 0.002     ; 2 fs
> ; Output control
> nstxout     = 0         ; suppress .trr output  
> nstvout     = 0         ; suppress .trr output
> nstenergy   = 10000      ; save energies every 2 ps
> nstlog      = 10000      ; update log file every 2 ps
> nstxtcout   = 15000      ; write .xtc trajectory every 2 ps
> energygrps  = Protein non-Protein
> ; Bond parameters
> continuation    = yes           ; first dynamics run
> constraint_algorithm = lincs    ; holonomic constraints  
> constraints     = all-bonds     ; all bonds (even heavy atom-H bonds) c
> lincs_iter      = 1             ; accuracy of LINCS
> lincs_order     = 4             ; also related to accuracy
> ; Neighborsearching
> ns_type     = grid      ; search neighboring grid cells
> nstlist     = 10         ; 10 fs
> cutoff-scheme = Verlet
> rlist       = 1.0       ; short-range neighborlist cutoff (in nm)
> rcoulomb    = 1.0       ; short-range electrostatic cutoff (in nm)
> rvdw        = 1.0       ; short-range van der Waals cutoff (in nm)
> ; Electrostatics
> coulombtype     = PME       ; Particle Mesh Ewald for long-range electr
> pme_order       = 4         ; cubic interpolation
> fourierspacing  = 0.16      ; grid spacing for FFT
> ; Temperature coupling
> tcoupl      = V-rescale                     ; modified Berendsen thermo
> tc-grps     = Protein non-Protein    ; two coupling groups - more accur
> tau_t       = 0.1   0.1                       ; time constant, in ps
> ref_t       = 300   300                       ; reference temperature,  
> ; Pressure coupling  
> pcoupl      = Parrinello-Rahman             ; pressure coupling is on f
> pcoupltype  = isotropic                     ; uniform scaling of box ve
> tau_p       = 2.0                           ; time constant, in ps
> ref_p       = 1.0                           ; reference pressure, in ba
> compressibility = 4.5e-5                    ; isothermal compressibilit
> ; Periodic boundary conditions
> pbc         = xyz       ; 3-D PBC
> ; Dispersion correction
> DispCorr    = EnerPres  ; account for cut-off vdW scheme
> ; Velocity generation
> gen_vel     = no        ; assign velocities from Maxwell distribution
>  

  
Kind regards,
Carlos
--  
Carlos Navarro Retamal
Bioinformatic engineer
Ph.D(c) in Applied Science, Universidad de Talca, Chile
Center of Bioinformatics and Molecular Simulations (CBSM)
Universidad de Talca
2 Norte 685, Casilla 721, Talca - Chile   
Teléfono: 56-71-201 798,  
Fax: 56-71-201 561
Email: carlos.navarro87 at gmail.com or cnavarro at utalca.cl