[gmx-users] Number of Xeon cores per GTX 1080Ti

Thu Apr 5 10:22:21 CEST 2018

Am 03.04.18 um 19:03 schrieb Szilárd Páll:
> On Tue, Apr 3, 2018 at 5:10 PM, Jochen Hub <jhub at gwdg.de> wrote:
>>
>>
>> Am 03.04.18 um 16:26 schrieb Szilárd Páll:
>>
>>> On Tue, Apr 3, 2018 at 3:41 PM, Jochen Hub <jhub at gwdg.de> wrote:
>>>>
>>>> benchmar
>>>>
>>>> Am 29.03.18 um 20:57 schrieb Szilárd Páll:
>>>>
>>>>> Hi Jochen,
>>>>>
>>>>> For that particular benchmark I only measured performance with
>>>>> 1,2,4,8,16 cores with a few different kinds of GPUs. It would be easy
>>>>> to do the runs on all possible core counts with increments of 1, but
>>>>> that won't tell a whole lot more than what the performance is of a run
>>>>> using a E5-2620 v4 CPU (with some GPUs) on a certain core count. Even
>>>>> extrapolating from that 2620 to a E5-2630 v4 and expecting to get a
>>>>> good estimate is tricky (given that the latter has 25% more cores for
>>>>> the same TDP!), let alone to any 26xxv4 CPU or the current-gen Skylake
>>>>> chips which have different performance characteristics.
>>>>>
>>>>> As Mark notes, there are some mdp option as well as some system
>>>>> charateristics that will have a strong influence on performance -- if
>>>>> tens of % is something you consider "strong" (some users are fine to
>>>>> be within a 2x ballpark :).
>>>>>
>>>>> What's worth considering is to try to avoid ending up strongly CPU or
>>>>> GPU bound from start. That may admittedly could be a difficult task
>>>>> you would run e.g. both biased MD with large pull groups and all-bonds
>>>>> constraints with Amber FF on large-ish (>100k) systems as well as
>>>>> vanilla MD with CHARMM FF with small-ish (<25k) systems. On the same
>>>>> hardware the former will be more prone to be CPU-bound while the
>>>>> latter will have relatively more GPU-heavy workload.
>>>>>
>>>>> There are many factors that influence the performance of a run and
>>>>> therefore giving a the one right answer to your question is not really
>>>>> possible. What can say is that 7-10 "core-GHz" per fast Pascal GPU is
>>>>> generally sufficient for "typical" protein simulations to run at >=85%
>>>>> of peak.
>>>>
>>>>
>>>>
>>>> Hi Szilárd,
>>>>
>>>> many thanks, this alrady helps me a lot. Just to get it 100% clear what
>>>> you
>>>> mean with core-GHz: A 10-core E5-2630v4 with 2.2 GHz would have 22
>>>> core-GHz,
>>>> right?
>>>
>>>
>>> Yes, that's what I was referring to; note that a 2630v4 won't be
>>> running at a 2.2 GHz base clock if you run AVX code ;)
>>
>>
>> Okay, I didn't know this. What would be the base clock instead with AVX
>> code?
> 
> 
> Short version: It's not easy to out details as Intel conveniently
> omits it from the specs, but it's AFAIK 3-400 MHz lower; also note
> that "turbo bins" change as a function of cores used (so you can't
> just benchmark on a few cores leaving the rest idle). Also, the actual
> clock speed (and overall performance) depend on other factors too so
> benchmarking and extrapolation might require consider other factors
> too.
> 
> Let me know if you are interested in more details.

Hi Szilárd,

many thanks, that helps a lot!

Best,
Jochen

> 
> --
> Szilárd
> 
>>
>>>
>>>> Thanks,
>>>> Jochen
>>>>
>>>>
>>>>>
>>>>> Cheers,
>>>>> --
>>>>> Szilárd
>>>>>
>>>>>
>>>>> On Wed, Mar 28, 2018 at 4:31 AM, Mark Abraham <mark.j.abraham at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> On Tue, Mar 27, 2018 at 6:43 PM Jochen Hub <jhub at gwdg.de> wrote:
>>>>>>
>>>>>>> Dear Gromacs community, dear Mark,
>>>>>>>
>>>>>>> Mark showed in the webinar today that having more than 8 Xeon
>>>>>>> E5-26XXv4
>>>>>>> cores does not help when using a GTX 1080Ti and PME on the GPU.
>>>>>>>
>>>>>>
>>>>>> ... for that particular simulation system.
>>>>>>
>>>>>>
>>>>>>> Unfortunately, there were no data points between 4 and 8 CPU cores,
>>>>>>> hence it was not clear at which #cores the performance actually levels
>>>>>>> off. With a GTX 1080 (not Ti) I once found that having more than 5
>>>>>>> Xeon
>>>>>>> cores does not help, if not having UB potentials, but I don't have a
>>>>>>> 1080Ti at hand to test for that.
>>>>>>>
>>>>>>
>>>>>> Those data points may not have been run. Szilard might have the data -
>>>>>> this
>>>>>> was GLIC 2fs comparing 1080 with 1080Ti from the recent plots he
>>>>>> shared.
>>>>>>
>>>>>>
>>>>>>> So my questions are:
>>>>>>>
>>>>>>> - At which number of E5-26XXv4 cores does the performance for common
>>>>>>> systems level off with a 1080Ti for your test system (GLIC)?
>>>>>>>
>>>>>>> - Does the answer depend strongly on the mdp settings (in particular
>>>>>>> on
>>>>>>> the LJ cutoff)?
>>>>>>>
>>>>>>
>>>>>> Longer LJ cutoff (e.g. from different forcefields) will certainly
>>>>>> require
>>>>>> more non-bonded work, so then fewer CPU cores would be needed to do the
>>>>>> remaining non-offloaded work. However any sweet spot for a particular
>>>>>> .tpr
>>>>>> would be highly dependent on other effects, such as the ratio of
>>>>>> solvent
>>>>>> (which typically has less LJ and simpler update) to solute, or the
>>>>>> density
>>>>>> of dihedral or U-B interactions. And doing pulling or FEP is very
>>>>>> different
>>>>>> again. The sweet spot for the next project will be elsewhere, sadly.
>>>>>>
>>>>>> This would help us a lot when choosing the appropriate CPU for a
>>>>>> 1080Ti.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Many thanks for any suggestions,
>>>>>>> Jochen
>>>>>>>
>>>>>>> --
>>>>>>> ---------------------------------------------------
>>>>>>> Dr. Jochen Hub
>>>>>>> Computational Molecular Biophysics Group
>>>>>>> Institute for Microbiology and Genetics
>>>>>>> Georg-August-University of Göttingen
>>>>>>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany
>>>>>>>
>>>>>>>
>>>>>>> <https://maps.google.com/?q=Justus-von-Liebig-Weg+11,+37077+G%C3%B6ttingen,+Germany&entry=gmail&source=g>
>>>>>>> .
>>>>>>> Phone: +49-551-39-14189 <+49%20551%203914189>
>>>>>>> http://cmb.bio.uni-goettingen.de/
>>>>>>> ---------------------------------------------------
>>>>>>> --
>>>>>>> Gromacs Users mailing list
>>>>>>>
>>>>>>> * Please search the archive at
>>>>>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>>>>>>> posting!
>>>>>>>
>>>>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>>>>>
>>>>>>> * For (un)subscribe requests visit
>>>>>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>>>>>>> send a mail to gmx-users-request at gromacs.org.
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Gromacs Users mailing list
>>>>>>
>>>>>> * Please search the archive at
>>>>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>>>>>> posting!
>>>>>>
>>>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>>>>
>>>>>> * For (un)subscribe requests visit
>>>>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>>>>>> send a mail to gmx-users-request at gromacs.org.
>>>>
>>>>
>>>>
>>>> --
>>>> ---------------------------------------------------
>>>> Dr. Jochen Hub
>>>> Computational Molecular Biophysics Group
>>>> Institute for Microbiology and Genetics
>>>> Georg-August-University of Göttingen
>>>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
>>>> Phone: +49-551-39-14189
>>>>
>>>> http://cmb.bio.uni-goettingen.de/
>>>> ---------------------------------------------------
>>>> --
>>>> Gromacs Users mailing list
>>>>
>>>> * Please search the archive at
>>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>>>> posting!
>>>>
>>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>>
>>>> * For (un)subscribe requests visit
>>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>>>> send a
>>>> mail to gmx-users-request at gromacs.org.
>>
>>
>> --
>> ---------------------------------------------------
>> Dr. Jochen Hub
>> Computational Molecular Biophysics Group
>> Institute for Microbiology and Genetics
>> Georg-August-University of Göttingen
>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
>> Phone: +49-551-39-14189
>> http://cmb.bio.uni-goettingen.de/
>> ---------------------------------------------------
>> --
>> Gromacs Users mailing list
>>
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a
>> mail to gmx-users-request at gromacs.org.

-- 
---------------------------------------------------
Dr. Jochen Hub
Computational Molecular Biophysics Group
Institute for Microbiology and Genetics
Georg-August-University of Göttingen
Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
Phone: +49-551-39-14189
http://cmb.bio.uni-goettingen.de/
---------------------------------------------------