[gmx-users] Number of Xeon cores per GTX 1080Ti

[Szilárd Páll]

On Tue, Apr 3, 2018 at 5:10 PM, Jochen Hub <jhub at gwdg.de> wrote:
>
>
> Am 03.04.18 um 16:26 schrieb Szilárd Páll:
>
>> On Tue, Apr 3, 2018 at 3:41 PM, Jochen Hub <jhub at gwdg.de> wrote:
>>>
>>>benchmar
>>>
>>> Am 29.03.18 um 20:57 schrieb Szilárd Páll:
>>>
>>>> Hi Jochen,
>>>>
>>>> For that particular benchmark I only measured performance with
>>>> 1,2,4,8,16 cores with a few different kinds of GPUs. It would be easy
>>>> to do the runs on all possible core counts with increments of 1, but
>>>> that won't tell a whole lot more than what the performance is of a run
>>>> using a E5-2620 v4 CPU (with some GPUs) on a certain core count. Even
>>>> extrapolating from that 2620 to a E5-2630 v4 and expecting to get a
>>>> good estimate is tricky (given that the latter has 25% more cores for
>>>> the same TDP!), let alone to any 26xxv4 CPU or the current-gen Skylake
>>>> chips which have different performance characteristics.
>>>>
>>>> As Mark notes, there are some mdp option as well as some system
>>>> charateristics that will have a strong influence on performance -- if
>>>> tens of % is something you consider "strong" (some users are fine to
>>>> be within a 2x ballpark :).
>>>>
>>>> What's worth considering is to try to avoid ending up strongly CPU or
>>>> GPU bound from start. That may admittedly could be a difficult task
>>>> you would run e.g. both biased MD with large pull groups and all-bonds
>>>> constraints with Amber FF on large-ish (>100k) systems as well as
>>>> vanilla MD with CHARMM FF with small-ish (<25k) systems. On the same
>>>> hardware the former will be more prone to be CPU-bound while the
>>>> latter will have relatively more GPU-heavy workload.
>>>>
>>>> There are many factors that influence the performance of a run and
>>>> therefore giving a the one right answer to your question is not really
>>>> possible. What can say is that 7-10 "core-GHz" per fast Pascal GPU is
>>>> generally sufficient for "typical" protein simulations to run at >=85%
>>>> of peak.
>>>
>>>
>>>
>>> Hi Szilárd,
>>>
>>> many thanks, this alrady helps me a lot. Just to get it 100% clear what
>>> you
>>> mean with core-GHz: A 10-core E5-2630v4 with 2.2 GHz would have 22
>>> core-GHz,
>>> right?
>>
>>
>> Yes, that's what I was referring to; note that a 2630v4 won't be
>> running at a 2.2 GHz base clock if you run AVX code ;)
>
>
> Okay, I didn't know this. What would be the base clock instead with AVX
> code?


Short version: It's not easy to out details as Intel conveniently
omits it from the specs, but it's AFAIK 3-400 MHz lower; also note
that "turbo bins" change as a function of cores used (so you can't
just benchmark on a few cores leaving the rest idle). Also, the actual
clock speed (and overall performance) depend on other factors too so
benchmarking and extrapolation might require consider other factors
too.

Let me know if you are interested in more details.

--
Szilárd

>
>>
>>> Thanks,
>>> Jochen
>>>
>>>
>>>>
>>>> Cheers,
>>>> --
>>>> Szilárd
>>>>
>>>>
>>>> On Wed, Mar 28, 2018 at 4:31 AM, Mark Abraham <mark.j.abraham at gmail.com>
>>>> wrote:
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> On Tue, Mar 27, 2018 at 6:43 PM Jochen Hub <jhub at gwdg.de> wrote:
>>>>>
>>>>>> Dear Gromacs community, dear Mark,
>>>>>>
>>>>>> Mark showed in the webinar today that having more than 8 Xeon
>>>>>> E5-26XXv4
>>>>>> cores does not help when using a GTX 1080Ti and PME on the GPU.
>>>>>>
>>>>>
>>>>> ... for that particular simulation system.
>>>>>
>>>>>
>>>>>> Unfortunately, there were no data points between 4 and 8 CPU cores,
>>>>>> hence it was not clear at which #cores the performance actually levels
>>>>>> off. With a GTX 1080 (not Ti) I once found that having more than 5
>>>>>> Xeon
>>>>>> cores does not help, if not having UB potentials, but I don't have a
>>>>>> 1080Ti at hand to test for that.
>>>>>>
>>>>>
>>>>> Those data points may not have been run. Szilard might have the data -
>>>>> this
>>>>> was GLIC 2fs comparing 1080 with 1080Ti from the recent plots he
>>>>> shared.
>>>>>
>>>>>
>>>>>> So my questions are:
>>>>>>
>>>>>> - At which number of E5-26XXv4 cores does the performance for common
>>>>>> systems level off with a 1080Ti for your test system (GLIC)?
>>>>>>
>>>>>> - Does the answer depend strongly on the mdp settings (in particular
>>>>>> on
>>>>>> the LJ cutoff)?
>>>>>>
>>>>>
>>>>> Longer LJ cutoff (e.g. from different forcefields) will certainly
>>>>> require
>>>>> more non-bonded work, so then fewer CPU cores would be needed to do the
>>>>> remaining non-offloaded work. However any sweet spot for a particular
>>>>> .tpr
>>>>> would be highly dependent on other effects, such as the ratio of
>>>>> solvent
>>>>> (which typically has less LJ and simpler update) to solute, or the
>>>>> density
>>>>> of dihedral or U-B interactions. And doing pulling or FEP is very
>>>>> different
>>>>> again. The sweet spot for the next project will be elsewhere, sadly.
>>>>>
>>>>> This would help us a lot when choosing the appropriate CPU for a
>>>>> 1080Ti.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Many thanks for any suggestions,
>>>>>> Jochen
>>>>>>
>>>>>> --
>>>>>> ---------------------------------------------------
>>>>>> Dr. Jochen Hub
>>>>>> Computational Molecular Biophysics Group
>>>>>> Institute for Microbiology and Genetics
>>>>>> Georg-August-University of Göttingen
>>>>>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany
>>>>>>
>>>>>>
>>>>>> <https://maps.google.com/?q=Justus-von-Liebig-Weg+11,+37077+G%C3%B6ttingen,+Germany&entry=gmail&source=g>
>>>>>> .
>>>>>> Phone: +49-551-39-14189 <+49%20551%203914189>
>>>>>> http://cmb.bio.uni-goettingen.de/
>>>>>> ---------------------------------------------------
>>>>>> --
>>>>>> Gromacs Users mailing list
>>>>>>
>>>>>> * Please search the archive at
>>>>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>>>>>> posting!
>>>>>>
>>>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>>>>
>>>>>> * For (un)subscribe requests visit
>>>>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>>>>>> send a mail to gmx-users-request at gromacs.org.
>>>>>
>>>>>
>>>>> --
>>>>> Gromacs Users mailing list
>>>>>
>>>>> * Please search the archive at
>>>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>>>>> posting!
>>>>>
>>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>>>
>>>>> * For (un)subscribe requests visit
>>>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>>>>> send a mail to gmx-users-request at gromacs.org.
>>>
>>>
>>>
>>> --
>>> ---------------------------------------------------
>>> Dr. Jochen Hub
>>> Computational Molecular Biophysics Group
>>> Institute for Microbiology and Genetics
>>> Georg-August-University of Göttingen
>>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
>>> Phone: +49-551-39-14189
>>>
>>> http://cmb.bio.uni-goettingen.de/
>>> ---------------------------------------------------
>>> --
>>> Gromacs Users mailing list
>>>
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>>> posting!
>>>
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>> * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>>> send a
>>> mail to gmx-users-request at gromacs.org.
>
>
> --
> ---------------------------------------------------
> Dr. Jochen Hub
> Computational Molecular Biophysics Group
> Institute for Microbiology and Genetics
> Georg-August-University of Göttingen
> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
> Phone: +49-551-39-14189
> http://cmb.bio.uni-goettingen.de/
> ---------------------------------------------------
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a
> mail to gmx-users-request at gromacs.org.