[gmx-users] 1 gpu vs 2 gpu speedup

[Szilárd Páll]

On Wed, Jul 9, 2014 at 2:27 PM, Harry Mark Greenblatt
<harry.greenblatt at weizmann.ac.il> wrote:
> BS"D
>
> Dear Szilárd,
>
> Thanks very much for your reply.
>
>
> This compiler is very outdated, you should use at least gcc 4.7 or 4.8
> for best performance - especially the CPU-only runs should get quite a
> bit faster.
>
> Thanks, point taken.  The person who compiled this on the test machine just used what RedHat 6.5 had to offer.
>
>
> Setup Wall Time (s) ns/day speedup (relative to 1st result)
>
> 6 cores, no gpu 12,996 6.65
> 12 cores, no gpu 7037 12.3 1.85
> 6 cores, 1 gpu 1853 46.6 7.01
> 2 x 6 cores, 2 gpu 1342 64.4 9.68
>
> Is that really 7x speedup wrt 6 cores? That should be more like 3-4x,
> so I suspect your CPU-only performance is 1.5-2x off.
>
> The speedup would be relative to the CPU.  This is a Sandy Bridge CPU running at 2.3GHz, not an Ivy Bridge v2 model.  Perhaps you've gotten used to later CPU's.

The Sandy Bridge-Ivy Bridge difference at identical clock speed is
quite small (<5%, I think). So your CPU-only performance is poor
(which I'm guessing based on the unusually high GPU speedup) probably
because of the old compiler.

> The job certainly ran on 2 MPI ranks, using 12 cores and 2 GPU's (~75% usage each).  I'm wondering if I didn't specify the proper mdrun command line arguments (based on log file, see below, where it complains about too few total ranks), or perhaps my system is not amenable to efficient GPU acceleration beyond one GPU.
>
> Comments?  Below is some of the last log file:
> ...
> Number of hardware threads detected (12) does not match the number reported by OpenMP (6).
>
> This does not look good, I think it means that your job scheduler is
> expecting you to use 6 cores. You should make sure that thread
> affinities are set correctly and getting rid of the above could help
> too.
>
>
> I suspect my mdrun arguments (or lack thereof) might be at fault.  I did try "-pin on" but it had no effect.

The mdrun arguments can't influence the CPU count the OpenMP library
reports, that's something typically environment variables control.

>
> ...
>
> You may want to try using multiple ranks per GPU, e.g.
> mpirun -np 4 mdrun -gpu_id 0011
> mpirun -np 6 mdrun -gpu_id 000111
>
>
> Ok, will see what happens.
> Thanks very much
>
> Harry
>
>
> -------------------------------------------------------------------------
>
> Harry M. Greenblatt
>
> Associate Staff Scientist
>
> Dept of Structural Biology
>
> Weizmann Institute of Science        Phone:  972-8-934-3625
>
> 234 Herzl St.                        Facsimile:   972-8-934-4159
>
> Rehovot, 76100
>
> Israel
>
>
> Harry.Greenblatt at weizmann.ac.il<mailto:Harry.Greenblatt at weizmann.ac.il>
>
>
>
>
>
>
>
> --
> Gromacs Users mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.