Aw: Re: [gmx-users] GPU-based workstation

[Szilárd Páll]

On Sat, May 25, 2013 at 2:16 PM, Broadbent, Richard
<richard.broadbent09 at imperial.ac.uk> wrote:
> I've been running on my Universities GPU nodes these are one E5-xeon (6-cores 12 threads)  and have 4 Nvidia 690gtx's. My system is 93 000 atoms of DMF under NVE.  The performance has been a little disappointing

That sounds like a very imbalanced system for GROMACS, you have
essentially 8 GPUs with rather poor PCI-E performance (a board share a
single PCI-E bus) and only 12 CPU cores to "drive" the simulation.

~10ns/day. On my home system using a core i5-2500 and a nvidia 560ti I
get 5.4ns/day for the same system. On our HPC system using 32 nodes
each with 2 quad-core xeon processors I get 30-40ns/day.

That sounds somewhat low if these are all moderately fast CPUs and GPUs.

> I think that to achieve reasonable performance the system has to be balanced between CPU's and GPU's probably getting 2 high end GPU's and a top end xeon E5 or core i7 would be a good choice.

Indeed. Even two GPUs may be too much - unless the CPU in question is
a very high end i7 or E5.

Cheers,
--
Szilárd

>
>
> Richard
>
> From: lloyd riggs <lloyd.riggs at gmx.ch<mailto:lloyd.riggs at gmx.ch>>
> Reply-To: Discussion users <gmx-users at gromacs.org<mailto:gmx-users at gromacs.org>>
> Date: Saturday, 25 May 2013 12:02
> To: Discussion users <gmx-users at gromacs.org<mailto:gmx-users at gromacs.org>>
> Subject: Aw: Re: [gmx-users] GPU-based workstation
>
> More RAM the better, and the best I have seen is 4 GPU work station.  I can use/have used 4.  The GPU takes 2 slots though, so a 7-8 PCIe board is really 3-4 GPU, except the tyan mentioned (there designed as blades so an 8 or 10 slot board really holds 8 or 10 GPU's).  There's cooling problems though with GPU's, as on a board there packed, so extra cooling things may help not blow a GPU, but I would look for good ones (ask around), as its a video game market and they go for looks even though its in casing?  The external RAM (not onboard GPU RAM) helps if you do a larger sim, but I dont know performance wise, the onboard GPU, the more RAM the marrier...so yes, normal work stations you can get 4 GPU's for a 300 US$ board, but then the price goes way up (3-4000 US$ for an 8-10 gpu board).  RAM ordered abroad is also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests software, not Gromacs, so would be nice to see performance...for a small 100 atom molecule and 500 solvent, using just the CPU I get it to run 5-10 minutes real  for 1 ns sim, but tried simple large 800 amino, 25,000 solvent eq (NVT or NPT) runs and they clock at around 1 hour real for say 50 ps eq's....
>
> Stephan
>
> Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
> Von: "James Starlight" <jmsstarlight at gmail.com<mailto:jmsstarlight at gmail.com>>
> An: "Discussion list for GROMACS users" <gmx-users at gromacs.org<mailto:gmx-users at gromacs.org>>
> Betreff: Re: [gmx-users] GPU-based workstation
> Dear Dr. Watkins!
>
> Thank you for the suggestions!
>
> In the local shops I've found only Core i7 with 6 cores (like Core
> i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores
> than with 4 cores in case of i7 cpu (assuming that I run simulation in
> cpu+gpu mode )?
>
> Also you've mentioned about 4 PCeI MD. Does it means that modern
> work-station could have 4 GPU's in one home-like desktop ? According to my
> current task I suppose that 2 GPU's would be suitable for my simulations
> (assuming that I use typical ASUS MB and 650 Watt power unit). Have
> someone tried to use several GPU's on one workstation ? What attributes of
> MB should be taken into account for best performance on such multi-gpu
> station ?
>
> James
>
> 2013/5/25 lloyd riggs <lloyd.riggs at gmx.ch<mailto:lloyd.riggs at gmx.ch>>
>
>> There's also these, but 1 chip runs 6K US, they can get performance up to
>> 2.3 teraflops per chip though double percission...but have no clue about
>> integration with GPU's...Intell also sells their chips on PCIe cards...but
>> get only about 350 Gflops, and run 1K US$.
>>
>> http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor
>> http://www.xilinx.com/
>>
>> They can design them though to fit a PCIe slot and run about the same, but
>> still need the board, ram etc...
>>
>> Mostly just to dream about, they say you can order them with radiation
>> shielding as well...so...
>>
>> Stephan Watkins
>>
>> *Gesendet:* Freitag, 24. Mai 2013 um 13:17 Uhr
>> *Von:* "James Starlight" <jmsstarlight at gmail.com<mailto:jmsstarlight at gmail.com>>
>> *An:* "Discussion list for GROMACS users" <gmx-users at gromacs.org<mailto:gmx-users at gromacs.org>>
>> *Betreff:* [gmx-users] GPU-based workstation
>> Dear Gromacs Users!
>>
>>
>> I'd like to build new workstation for performing simulation on GPU with
>> Gromacs 4.6 native cuda support.
>> Recently I've used such setup with Core i5 cpu and nvidia 670 GTX video
>> and obtain good performance ( ~ 20 ns\day for typical 60.000 atom system
>> with SD integrator)
>>
>>
>> Now I'd like to build multi-gpu wokstation.
>>
>> My question - How much GPU would give me best performance on the typical
>> home-like workstation. What algorithm of Ncidia GPU integration should I
>> use (e.g SLI etc) ?
>>
>>
>> Thanks for help,
>>
>>
>> James
>> --
>> gmx-users mailing list gmx-users at gromacs.org<mailto:gmx-users at gromacs.org>
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> * Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-request at gromacs.org<mailto:gmx-users-request at gromacs.org>.
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> --
>> gmx-users mailing list gmx-users at gromacs.org<mailto:gmx-users at gromacs.org>
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> * Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-request at gromacs.org<mailto:gmx-users-request at gromacs.org>.
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
> --
> gmx-users mailing list gmx-users at gromacs.org<mailto:gmx-users at gromacs.org>
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org<mailto:gmx-users-request at gromacs.org>.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists