[gmx-users] GROMACS 5.1.2 performance

[Neha Gandhi]

Dear list,

I think the question on gromacs performance on cpu cluster has been raised
many times in the mailing list. My apologies for reiterating the question.
I am using a system ~80000 atoms with virtual sites (hence timestep of 4
fs). The job hasn't completed yet but it seems that the jobs are running
slower than my experience running with older versions of GROMACS.

Here is the job script
#!/bin/bash -l
#PBS -N sgk
#PBS -l walltime=24:00:00
#PBS -l select=6:ncpus=16:mpiprocs=16:mem=20gb
#PBS -j oe

export OMP_NUM_THREADS=1

module purge
module load gromacs/5.1.2-foss-2016a-hybrid
mpirun -np 96 gmx_mpi mdrun -v -deffnm npt


and the output log indicates load imbalance :

Number of logical cores detected (48) does not match the number reported by
OpenMP (10).
Consider setting the launch configuration manually!

Running on 6 nodes with total 248 logical cores
  Logical cores per node:   40 - 48
Hardware detected on host cl3n073 (the node of MPI rank 0):
  CPU info:
    Vendor: GenuineIntel
    Brand:  Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
    Family:  6  model: 63  stepping:  2
    CPU features: apic avx avx2 clfsh cmov cx8 cx16 f16c fma htt lahf_lm
mmx msr nonstop_tsc pcid pclmuldq pdcm pdpe1gb popcnt pse rdrnd rdtscp sse2
sse3 sse4.1 sse4.2 ssse3 tdt x2apic
    SIMD instructions most likely to fit this hardware: AVX_256 - AVX2_256
    SIMD instructions selected at GROMACS compile time: AVX_256


DD  step 34999  vol min/aver 0.553  load imb.: force 46.2%  pme mesh/force
0.506

           Step           Time         Lambda
          35000       70.00000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    9.21820e+03    6.85434e+03    2.03852e+03   -4.52887e+01    8.24856e+04
        LJ (SR)  Disper. corr.   Coulomb (SR)   Coul. recip.      Potential
    1.76075e+05   -4.85764e+03   -1.52098e+06    3.15077e+03   -1.24606e+06
    Kinetic En.   Total Energy    Temperature Pres. DC (bar) Pressure (bar)
    2.19009e+05   -1.02705e+06    2.99843e+02   -8.61175e+01   -1.48459e+02
   Constr. rmsd
    2.75565e-05

DD  step 39999  vol min/aver 0.475  load imb.: force 89.0%  pme mesh/force
0.948

           Step           Time         Lambda
          40000       80.00000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    9.32970e+03    6.83305e+03    2.07167e+03   -2.67360e+01    8.24534e+04
        LJ (SR)  Disper. corr.   Coulomb (SR)   Coul. recip.      Potential
    1.78434e+05   -4.86516e+03   -1.52332e+06    3.15600e+03   -1.24593e+06
    Kinetic En.   Total Energy    Temperature Pres. DC (bar) Pressure (bar)
    2.19647e+05   -1.02628e+06    3.00717e+02   -8.63844e+01    7.45115e+01
   Constr. rmsd
    2.80701e-05

DD  step 44999  vol min/aver 0.704  load imb.: force 106.3%  pme mesh/force
0.720

           Step           Time         Lambda
          45000       90.00000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    9.30472e+03    6.53990e+03    2.08250e+03    1.40282e+01    8.27162e+04
        LJ (SR)  Disper. corr.   Coulomb (SR)   Coul. recip.      Potential
    1.77677e+05   -4.87223e+03   -1.52383e+06    3.05237e+03   -1.24731e+06
    Kinetic En.   Total Energy    Temperature Pres. DC (bar) Pressure (bar)
    2.19610e+05   -1.02770e+06    3.00666e+02   -8.66354e+01    4.81709e+01
   Constr. rmsd
    2.77685e-05

Writing checkpoint, step 46075 at Wed Dec 14 21:28:11 2016


DD  step 49999  vol min/aver 0.543  load imb.: force 16.6%  pme mesh/force
0.848

           Step           Time         Lambda
          50000      100.00000        0.00000

   Energies (kJ/mol)
       G96Angle    Proper Dih.  Improper Dih.          LJ-14     Coulomb-14
    9.09212e+03    6.76222e+03    1.95816e+03   -6.36247e+01    8.23450e+04
        LJ (SR)  Disper. corr.   Coulomb (SR)   Coul. recip.      Potential
    1.77732e+05   -4.87147e+03   -1.52371e+06    3.15838e+03   -1.24760e+06
    Kinetic En.   Total Energy    Temperature Pres. DC (bar) Pressure (bar)
    2.18646e+05   -1.02895e+06    2.99346e+02   -8.66085e+01    1.49550e+01
   Constr. rmsd
    2.95505e-05

I appreciate your feedback.

Regards,
Neha