[gmx-users] CPU running doesn't match command line

[Albert]

Hello:

Here is my MDP file:

define                  = -DREST_ON -DSTEP6_4
integrator              = md
dt                      = 0.002
nsteps                  = 1000000
nstlog                  = 1000
nstxout                 = 0
nstvout                 = 0
nstfout                 = 0
nstcalcenergy           = 100
nstenergy               = 1000
nstxout-compressed      = 10000
;
cutoff-scheme           = Verlet
nstlist                 = 20
rlist                   = 1.0
coulombtype             = pme
rcoulomb                = 1.0
vdwtype                 = Cut-off
vdw-modifier            = Force-switch
rvdw_switch             = 0.9
rvdw                    = 1.0
;
tcoupl                  = berendsen
tc_grps                 = PROT   MEMB   SOL_ION
tau_t                   = 1.0    1.0    1.0
ref_t                   = 310   310   310
;
pcoupl                  = berendsen
pcoupltype              = semiisotropic
tau_p                   = 5.0
compressibility         = 4.5e-5  4.5e-5
ref_p                   = 1.0     1.0
;
constraints             = h-bonds
constraint_algorithm    = LINCS
continuation            = yes
;
nstcomm                 = 100
comm_mode               = linear
comm_grps               = PROT   MEMB   SOL_ION
;
refcoord_scaling        = com


I compiled Gromacs with the following settings, using Intel MPI:

env CC=mpicc CXX=mpicxx F77=mpif90 FC=mpif90 LDF90=mpif90 
CMAKE_PREFIX_PATH=/soft/gromacs/fftw-3.3.4:/soft/intel/impi/5.1.3.223 
cmake .. -DBUILD_SHARED_LIB=OFF -DBUILD_TESTING=OFF 
-DCMAKE_INSTALL_PREFIX=/soft/gromacs/5.1.3_intel -DGMX_MPI=ON 
-DGMX_GPU=ON -DGMX_PREFER_STATIC_LIBS=ON 
-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda


I tried it again with one of the job with options:

-ntomp 8 -pin on -pinoffset 8


The two submitted jobs can still only use 8 CPU and the speed is 
extremely slow (10ns/day)....when I remove option "-pin on" from one of 
the job, it fasten a lot (32ns/day) and 16 CPU were used..... If I only 
submit one job with option "-pin on", I can obtain 52ns/day..........


thx a lot


On 08/16/2016 04:59 PM, Szilárd Páll wrote:
> Hi,
>
> Without log and hw configs, I it's hard to tell what's happening.
>
> By turning off pinning the OS is free to move threads around and it
> will try to ensure cores are utilized. However, by leaving threads
> up-pinned you risk taking a significant performance hit. So I'd
> recommend that you run with correct settings.
>
> If you start with "-ntomp 8 -pin on -pioffset 8" (and you indeed have
> 16 cores no HT), you should be able to see in htop the first eight
> cores empty while the last eight occupied.
>
> Cheers,
> --
> Szilárd