[gmx-users] Number of Xeon cores per GTX 1080Ti

[Szilárd Páll]

On Tue, Apr 3, 2018 at 3:41 PM, Jochen Hub <jhub at gwdg.de> wrote:
>
>
> Am 29.03.18 um 20:57 schrieb Szilárd Páll:
>
>> Hi Jochen,
>>
>> For that particular benchmark I only measured performance with
>> 1,2,4,8,16 cores with a few different kinds of GPUs. It would be easy
>> to do the runs on all possible core counts with increments of 1, but
>> that won't tell a whole lot more than what the performance is of a run
>> using a E5-2620 v4 CPU (with some GPUs) on a certain core count. Even
>> extrapolating from that 2620 to a E5-2630 v4 and expecting to get a
>> good estimate is tricky (given that the latter has 25% more cores for
>> the same TDP!), let alone to any 26xxv4 CPU or the current-gen Skylake
>> chips which have different performance characteristics.
>>
>> As Mark notes, there are some mdp option as well as some system
>> charateristics that will have a strong influence on performance -- if
>> tens of % is something you consider "strong" (some users are fine to
>> be within a 2x ballpark :).
>>
>> What's worth considering is to try to avoid ending up strongly CPU or
>> GPU bound from start. That may admittedly could be a difficult task
>> you would run e.g. both biased MD with large pull groups and all-bonds
>> constraints with Amber FF on large-ish (>100k) systems as well as
>> vanilla MD with CHARMM FF with small-ish (<25k) systems. On the same
>> hardware the former will be more prone to be CPU-bound while the
>> latter will have relatively more GPU-heavy workload.
>>
>> There are many factors that influence the performance of a run and
>> therefore giving a the one right answer to your question is not really
>> possible. What can say is that 7-10 "core-GHz" per fast Pascal GPU is
>> generally sufficient for "typical" protein simulations to run at >=85%
>> of peak.
>
>
> Hi Szilárd,
>
> many thanks, this alrady helps me a lot. Just to get it 100% clear what you
> mean with core-GHz: A 10-core E5-2630v4 with 2.2 GHz would have 22 core-GHz,
> right?

Yes, that's what I was referring to; note that a 2630v4 won't be
running at a 2.2 GHz base clock if you run AVX code ;)

> Thanks,
> Jochen
>
>
>>
>> Cheers,
>> --
>> Szilárd
>>
>>
>> On Wed, Mar 28, 2018 at 4:31 AM, Mark Abraham <mark.j.abraham at gmail.com>
>> wrote:
>>>
>>> Hi,
>>>
>>> On Tue, Mar 27, 2018 at 6:43 PM Jochen Hub <jhub at gwdg.de> wrote:
>>>
>>>> Dear Gromacs community, dear Mark,
>>>>
>>>> Mark showed in the webinar today that having more than 8 Xeon E5-26XXv4
>>>> cores does not help when using a GTX 1080Ti and PME on the GPU.
>>>>
>>>
>>> ... for that particular simulation system.
>>>
>>>
>>>> Unfortunately, there were no data points between 4 and 8 CPU cores,
>>>> hence it was not clear at which #cores the performance actually levels
>>>> off. With a GTX 1080 (not Ti) I once found that having more than 5 Xeon
>>>> cores does not help, if not having UB potentials, but I don't have a
>>>> 1080Ti at hand to test for that.
>>>>
>>>
>>> Those data points may not have been run. Szilard might have the data -
>>> this
>>> was GLIC 2fs comparing 1080 with 1080Ti from the recent plots he shared.
>>>
>>>
>>>> So my questions are:
>>>>
>>>> - At which number of E5-26XXv4 cores does the performance for common
>>>> systems level off with a 1080Ti for your test system (GLIC)?
>>>>
>>>> - Does the answer depend strongly on the mdp settings (in particular on
>>>> the LJ cutoff)?
>>>>
>>>
>>> Longer LJ cutoff (e.g. from different forcefields) will certainly require
>>> more non-bonded work, so then fewer CPU cores would be needed to do the
>>> remaining non-offloaded work. However any sweet spot for a particular
>>> .tpr
>>> would be highly dependent on other effects, such as the ratio of solvent
>>> (which typically has less LJ and simpler update) to solute, or the
>>> density
>>> of dihedral or U-B interactions. And doing pulling or FEP is very
>>> different
>>> again. The sweet spot for the next project will be elsewhere, sadly.
>>>
>>> This would help us a lot when choosing the appropriate CPU for a 1080Ti.
>>>>
>>>>
>>>> Many thanks for any suggestions,
>>>> Jochen
>>>>
>>>> --
>>>> ---------------------------------------------------
>>>> Dr. Jochen Hub
>>>> Computational Molecular Biophysics Group
>>>> Institute for Microbiology and Genetics
>>>> Georg-August-University of Göttingen
>>>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany
>>>>
>>>> <https://maps.google.com/?q=Justus-von-Liebig-Weg+11,+37077+G%C3%B6ttingen,+Germany&entry=gmail&source=g>
>>>> .
>>>> Phone: +49-551-39-14189 <+49%20551%203914189>
>>>> http://cmb.bio.uni-goettingen.de/
>>>> ---------------------------------------------------
>>>> --
>>>> Gromacs Users mailing list
>>>>
>>>> * Please search the archive at
>>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>>>> posting!
>>>>
>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>>
>>>> * For (un)subscribe requests visit
>>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>>>> send a mail to gmx-users-request at gromacs.org.
>>>
>>> --
>>> Gromacs Users mailing list
>>>
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>>>
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>> * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>>> send a mail to gmx-users-request at gromacs.org.
>
>
> --
> ---------------------------------------------------
> Dr. Jochen Hub
> Computational Molecular Biophysics Group
> Institute for Microbiology and Genetics
> Georg-August-University of Göttingen
> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
> Phone: +49-551-39-14189
>
> http://cmb.bio.uni-goettingen.de/
> ---------------------------------------------------
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a
> mail to gmx-users-request at gromacs.org.