[gmx-users] GPU-based workstation

Thu Aug 1 19:25:00 CEST 2013

I may be late with the reply, but here are my 2 cents.

If you need a single very fast machine (i.e. maximum single simulation
performance), you should get
- either a very fast desktop CPU: i7 3930 or for 2x more the 3970 -
which, BTW, I think is not worth it ($600-1000)
- or 1-2 fast Xeon E5-s - depending on how many and which these will
be $1k-2k each.

For a single CPU setup two Titans may be an overkill and (at least
with the current code) you may get very little extra performance from
using two iso one GPU. With a dual-socket machine (and decently fast
CPUs), if you have a large enough input system, two GPUs will work
nicely.

However, if you care about total simulation throughput and you have
multiple simulations to run, I'd suggest that you buy 2-3 machines
with the components that give the best ns/day/$: something like
i7-4670 or 4770 with GTX 680/770 (or 780).

--
Szilárd

On Thu, Jun 27, 2013 at 1:01 PM, James Starlight <jmsstarlight at gmail.com> wrote:
> Back to my question
> I want to build gpu-based workstation based onto 2 titans geforces.
>
> My current budget allow me only hight-end  6nodes core i 7-3930  and MB
> with 5 PCI-E (like Asus rampage IV series). Would this system be balanced
> with two GPUs ? Should I use two 6-8 nodes XEONS instead of i7?
>
> James
>
> 2013/5/29 James Starlight <jmsstarlight at gmail.com>
>
>> Dear Dr. Pall!
>>
>> Thank you for your suggestions!
>>
>> Asumming that I have budget of 5000 $ and I want to build gpu-based
>> desktop on this money.
>>
>> Previously I've used single 4 core i5 with GTX 670 and obtain average 10
>> ns\day performance for the 70k atoms systems (1.0 cutoffs, no virtual sites
>> , sd integrator).
>>
>> Now I'd like to build system based on 2 hight-end GeForces (e.g like
>> TITAN).
>> Should that system include 2 cpu's for good balancing? (e.g two 6 nodes
>> XEONS with faster clocks for instance could be better for simulations than
>> i7, couldnt it?)
>>
>> What addition properties to the MB should I consider for such system ?
>>
>> James
>>
>>
>> 2013/5/28 lloyd riggs <lloyd.riggs at gmx.ch>
>>
>>> Dear Dr. Pali,
>>>
>>> Thank you,
>>>
>>> Stephan Watkins
>>>
>>> *Gesendet:* Dienstag, 28. Mai 2013 um 19:50 Uhr
>>> *Von:* "Szilárd Páll" <szilard.pall at cbr.su.se>
>>>
>>> *An:* "Discussion list for GROMACS users" <gmx-users at gromacs.org>
>>> *Betreff:* Re: Re: [gmx-users] GPU-based workstation
>>> Dear all,
>>>
>>> As far as I understand, the OP is interested in hardware for *running*
>>> GROMACS 4.6 rather than developing code. or running LINPACK.
>>>
>>>
>>> To get best performance it is important to use a machine with hardware
>>> balanced for GROMACS' workloads. Too little GPU resources will result
>>> in CPU idling; too much GPU resources will lead to the runs being CPU
>>> or multi-GPU scaling bound and above a certain level GROMACS won't be
>>> able to make use of additional GPUs.
>>>
>>> Of course, the balance will depend both on hardware and simulation
>>> settings (mostly the LJ cut-off used).
>>>
>>> An additional factor to consider is typical system size. To reach near
>>> peak pair-force throughput on GPUs you typically need >20k-40k
>>> particles/GPU (depends on the architecture) and throughput drops below
>>> these values. Hence, in most cases it is preferred to use fewer and
>>> faster GPUs rather than more.
>>>
>>> Without knowing the budgdet and indented use of the machine it is hard
>>> to make suggestions, but I would say for a budget desktop box a
>>> quad-core Intel Ivy Bridge or the top-end AMD Piledriver CPU with a
>>> fast Kepler GTX card (e.g. GTX 680 or GTX 770/780) should work well.
>>> If you're considering dual-socket workstations, I suggest you go with
>>> the higher core-count and higher frequency Intel CPUs (6+ cores >2.2
>>> GHz), otherwise you may not see as much benefit as you would expect
>>> based on the insane price tag (especially if you compare to an i7
>>> 3939K or its IVB successor).
>>>
>>> Cheers,
>>> --
>>> Szilárd
>>>
>>>
>>> On Sat, May 25, 2013 at 1:02 PM, lloyd riggs <lloyd.riggs at gmx.ch> wrote:
>>> > More RAM the better, and the best I have seen is 4 GPU work station. I
>>> can
>>> > use/have used 4. The GPU takes 2 slots though, so a 7-8 PCIe board is
>>> > really 3-4 GPU, except the tyan mentioned (there designed as blades so
>>> an 8
>>> > or 10 slot board really holds 8 or 10 GPU's). There's cooling problems
>>> > though with GPU's, as on a board there packed, so extra cooling things
>>> may
>>> > help not blow a GPU, but I would look for good ones (ask around), as
>>> its a
>>> > video game market and they go for looks even though its in casing? The
>>> > external RAM (not onboard GPU RAM) helps if you do a larger sim, but I
>>> dont
>>> > know performance wise, the onboard GPU, the more RAM the marrier...so
>>> yes,
>>> > normal work stations you can get 4 GPU's for a 300 US$ board, but then
>>> the
>>> > price goes way up (3-4000 US$ for an 8-10 gpu board). RAM ordered
>>> abroad is
>>> > also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests
>>> > software, not Gromacs, so would be nice to see performance...for a
>>> small 100
>>> > atom molecule and 500 solvent, using just the CPU I get it to run 5-10
>>> > minutes real for 1 ns sim, but tried simple large 800 amino, 25,000
>>> solvent
>>> > eq (NVT or NPT) runs and they clock at around 1 hour real for say 50 ps
>>> > eq's....
>>> >
>>> > Stephan
>>> >
>>> > Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
>>> > Von: "James Starlight" <jmsstarlight at gmail.com>
>>> > An: "Discussion list for GROMACS users" <gmx-users at gromacs.org>
>>> > Betreff: Re: [gmx-users] GPU-based workstation
>>> > Dear Dr. Watkins!
>>> >
>>> > Thank you for the suggestions!
>>> >
>>> > In the local shops I've found only Core i7 with 6 cores (like Core
>>> > i7-39xx) and 4 cores. Should I obtain much better performance with 6
>>> cores
>>> > than with 4 cores in case of i7 cpu (assuming that I run simulation in
>>> > cpu+gpu mode )?
>>> >
>>> > Also you've mentioned about 4 PCeI MD. Does it means that modern
>>> > work-station could have 4 GPU's in one home-like desktop ? According to
>>> my
>>> > current task I suppose that 2 GPU's would be suitable for my simulations
>>> > (assuming that I use typical ASUS MB and 650 Watt power unit). Have
>>> > someone tried to use several GPU's on one workstation ? What attributes
>>> of
>>> > MB should be taken into account for best performance on such multi-gpu
>>> > station ?
>>> >
>>> > James
>>> >
>>> > 2013/5/25 lloyd riggs <lloyd.riggs at gmx.ch>
>>> >
>>> >> There's also these, but 1 chip runs 6K US, they can get performance up
>>> to
>>> >> 2.3 teraflops per chip though double percission...but have no clue
>>> about
>>> >> integration with GPU's...Intell also sells their chips on PCIe
>>> cards...but
>>> >> get only about 350 Gflops, and run 1K US$.
>>> >>
>>> >> http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor
>>> >> http://www.xilinx.com/
>>> >>
>>> >> They can design them though to fit a PCIe slot and run about the same,
>>> but
>>> >> still need the board, ram etc...
>>> >>
>>> >> Mostly just to dream about, they say you can order them with radiation
>>> >> shielding as well...so...
>>> >>
>>> >> Stephan Watkins
>>> >>
>>> >> *Gesendet:* Freitag, 24. Mai 2013 um 13:17 Uhr
>>> >> *Von:* "James Starlight" <jmsstarlight at gmail.com>
>>> >> *An:* "Discussion list for GROMACS users" <gmx-users at gromacs.org>
>>> >> *Betreff:* [gmx-users] GPU-based workstation
>>> >> Dear Gromacs Users!
>>> >>
>>> >>
>>> >> I'd like to build new workstation for performing simulation on GPU with
>>> >> Gromacs 4.6 native cuda support.
>>> >> Recently I've used such setup with Core i5 cpu and nvidia 670 GTX video
>>> >> and obtain good performance ( ~ 20 ns\day for typical 60.000 atom
>>> system
>>> >> with SD integrator)
>>> >>
>>> >>
>>> >> Now I'd like to build multi-gpu wokstation.
>>> >>
>>> >> My question - How much GPU would give me best performance on the
>>> typical
>>> >> home-like workstation. What algorithm of Ncidia GPU integration should
>>> I
>>> >> use (e.g SLI etc) ?
>>> >>
>>> >>
>>> >> Thanks for help,
>>> >>
>>> >>
>>> >> James
>>> >> --
>>> >> gmx-users mailing list gmx-users at gromacs.org
>>> >> http://lists.gromacs.org/mailman/listinfo/gmx-users
>>> >> * Please search the archive at
>>> >> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>> >> * Please don't post (un)subscribe requests to the list. Use the
>>> >> www interface or send it to gmx-users-request at gromacs.org.
>>> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>> >>
>>> >> --
>>> >> gmx-users mailing list gmx-users at gromacs.org
>>> >> http://lists.gromacs.org/mailman/listinfo/gmx-users
>>> >> * Please search the archive at
>>> >> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>> >> * Please don't post (un)subscribe requests to the list. Use the
>>> >> www interface or send it to gmx-users-request at gromacs.org.
>>> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>> >>
>>> > --
>>> > gmx-users mailing list gmx-users at gromacs.org
>>> > http://lists.gromacs.org/mailman/listinfo/gmx-users
>>> > * Please search the archive at
>>> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>> > * Please don't post (un)subscribe requests to the list. Use the
>>> > www interface or send it to gmx-users-request at gromacs.org.
>>> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>> >
>>> > --
>>> > gmx-users mailing list gmx-users at gromacs.org
>>> > http://lists.gromacs.org/mailman/listinfo/gmx-users
>>> > * Please search the archive at
>>> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>> > * Please don't post (un)subscribe requests to the list. Use the
>>> > www interface or send it to gmx-users-request at gromacs.org.
>>> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>> --
>>> gmx-users mailing list gmx-users at gromacs.org
>>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>> * Please don't post (un)subscribe requests to the list. Use the
>>> www interface or send it to gmx-users-request at gromacs.org.
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>> --
>>> gmx-users mailing list    gmx-users at gromacs.org
>>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>> * Please don't post (un)subscribe requests to the list. Use the
>>> www interface or send it to gmx-users-request at gromacs.org.
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>
>>
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists