[gmx-users] 1 gpu vs 2 gpu speedup

[Harry Mark Greenblatt]

BS"D

Dear Szilárd,

Thanks very much for your reply.


This compiler is very outdated, you should use at least gcc 4.7 or 4.8
for best performance - especially the CPU-only runs should get quite a
bit faster.

Thanks, point taken.  The person who compiled this on the test machine just used what RedHat 6.5 had to offer.


Setup Wall Time (s) ns/day speedup (relative to 1st result)

6 cores, no gpu 12,996 6.65
12 cores, no gpu 7037 12.3 1.85
6 cores, 1 gpu 1853 46.6 7.01
2 x 6 cores, 2 gpu 1342 64.4 9.68

Is that really 7x speedup wrt 6 cores? That should be more like 3-4x,
so I suspect your CPU-only performance is 1.5-2x off.

The speedup would be relative to the CPU.  This is a Sandy Bridge CPU running at 2.3GHz, not an Ivy Bridge v2 model.  Perhaps you've gotten used to later CPU's.

The job certainly ran on 2 MPI ranks, using 12 cores and 2 GPU's (~75% usage each).  I'm wondering if I didn't specify the proper mdrun command line arguments (based on log file, see below, where it complains about too few total ranks), or perhaps my system is not amenable to efficient GPU acceleration beyond one GPU.

Comments?  Below is some of the last log file:
...
Number of hardware threads detected (12) does not match the number reported by OpenMP (6).

This does not look good, I think it means that your job scheduler is
expecting you to use 6 cores. You should make sure that thread
affinities are set correctly and getting rid of the above could help
too.


I suspect my mdrun arguments (or lack thereof) might be at fault.  I did try "-pin on" but it had no effect.


...

You may want to try using multiple ranks per GPU, e.g.
mpirun -np 4 mdrun -gpu_id 0011
mpirun -np 6 mdrun -gpu_id 000111


Ok, will see what happens.
Thanks very much

Harry


-------------------------------------------------------------------------

Harry M. Greenblatt

Associate Staff Scientist

Dept of Structural Biology

Weizmann Institute of Science        Phone:  972-8-934-3625

234 Herzl St.                        Facsimile:   972-8-934-4159

Rehovot, 76100

Israel


Harry.Greenblatt at weizmann.ac.il<mailto:Harry.Greenblatt at weizmann.ac.il>