[gmx-users] Optimal hardware for running Gromacs

[Szilárd Páll]

Hi David,

If seem to be interested in details so please do check the paper Carsten
linked (further papers can be found here
http://www.gromacs.org/Gromacs_papers). I also recommend you to look up
discussions on the mailing list form the past. Many of your questions have
been thoroughly answered before, hence the selective answers below.

On Sat, Mar 12, 2016 at 2:52 AM, David Berquist <berquist at buffalo.edu>
wrote:

> I'm looking into building a desktop computer for running Gromacs
> simulations.  I'd like to know how I should configure it to get the best
> performance possible for the price.
>
> One example of something I might run on this system would be: simulate an
> ionic polymer of approximately 100k atoms for 1 millisecond.  Coarse
> grained atoms, using Martini force field.
>

Will you typically not use PME? In that case your runs will be more
GPU-bound than usual which is relevant because many of the benchmarks out
there may not give you an accurate idea about the ideal CPU/GPU ratio.


>
> I would be planning to spend something under $1500 on this system.
>

When? FYI: both NVIDIA and AMD are expected to launch new GPUs that will be
substantially faster.


>
>
> In regards to the graphics card, what factors are important?  I assume that
> the single precision Flop/s are a big factor, but what about memory
> bandwidth(on the card itself)?
>

This is a recurring question, I won't get into details, but in short:
(main) memory speed has little effect.


> I've noticed that Gromacs pages on GPUs tend to recommend nVidia cards,
> specifically those with CUDA Compute Capability 5.2.  But, AMD Radeon cards
> appear to offer the same processing power for far lower prices(or
> alternatively, significantly more processing power for the same price).


5.1 is the first release with OpenCL support and the performance with AMD
GPUs is good, but there was room for improvement. We have been working on
optimizations and in the next release AMD Fiji cards are going to be _very_
good in terms of both raw performance and even better in perf/$ metric.


> For example, the nVidia GTX 970 offers 3494 GFlop/s of single precision,
> and 109 GFlop/s of double precision.  It currently costs $290 at Newegg.
> On the other hand, the AMD Radeon R9 380 provides 3476.5 GFlop/s single
> precision, and 217.3 GFlop/s of double precision.  It currently costs $150
> on Newegg.
>
> That's practically the same single precision, and nearly double the
> double-precision for around half the price.
>
> Or, for about the same price as the nVidia GTX 970 I could get an AMD
> Radeon R9 390.  It provides 5120 GFlop/s of single precision and 640
> GFlop/s of double precision.  That costs $310 on Newegg.
>
> That is, on paper, 46.5% more single precision performance, and 487% more
> double precision performance(nearly 5 times as much) for only a 6.9% higher
> price.
>

MD is not LINPACK, so while the SP throughput is (for now) a good indicator
of the GROMACS GPU (kernel) performance, you can only use the relative SP
flop rate to estimate performance only within an architectural family; e.g.
given the GTX 960 performance you can guesstimate what you'd get on the 980.


>
> So, does Gromacs run a lot faster on nVidia GPUs?  Would the nVidia GTX 970
> provide similar or better performance than the AMD Radeon R9 390?
>

Based on quick extrapolation from the 980 and R290X numbers I had around,
those should perform similar, but results will greatly depend on the actual
stable clocks the cards run at.

However, does you 1.5k budget really not allow getting the next category of
cards? I'd recommend considering Fiji, especially the R9 Nano, or the GTX
980 / 980 Ti are good options.



> One more GPU Question; does Gromacs scale linearly across multiple graphics
> cards?   So, if I have two identical cards which offer 2000 GFlop/s
> performance each, would they both together perform about the same as a
> more-expensive single card which provides 4000GFlop/s(provided you have a
> fast enough CPU, of course)?
>

Define "scaling". Assuming you mean strong scaling, no you won't get linear
scaling, especially not from one to two GPUs.


>
> Now, how about the CPU performance?


Typically it matters quite a bit. For your Martini run not so much.

Read the paper and do some test runs if you can.


> Would it be better to spend a lot of
> my budget on a high-end GPU, and then go for a low cost CPU?  Something
> like an  AMD Athlon X4 860K Kaveri Quad-Core 3.7 GHz CPU?  Or would
> something like that bottleneck the performance of an nVidia GTX 970 or GTX
> 980?  Maybe a AMD FX-6300 Vishera 6-Core 3.5 GHz?  AMD FX-8300 Vishera
> 8-Core 3.3GHz?
> Does Intel vs AMD make any significant difference?  When running Gromacs on
> a system with GPU compute turned on, what sort of computing load does
> Gromacs put on the CPU?  Is there any commonly used CPU benchmark that can
> be used to approximate relative Gromacs performance between CPUs?
>

AMD CPUs are is sadly far far slower than Intel. Get a 4-core Haswell or
Skylake or the 6-core Hawell-E if you can afford.


>
>
> How important is the system RAM speed?  Should I spend more of my budget
> building a DDR4 system(more expensive CPU, motherboard, and RAM modules),
> or would that not make a significant difference over DDR3?
>

It is not. Bio-mol MD runs mostly in cache. If it does not it will run so
slow that you won't get much in terms of results.


>
> Would you happen to know what quantity of memory should be sufficient for
> common Gromacs simulation runs, or runs such as the example I included
> above?
>

Typically? It depends on the system size. For 100k ~0.5 Gb should be
enough, though.


>
> Finally, does the speed of the hard drive impact Gromacs performance?  The
> options are a cheap spinning-platter hard drive(100-200MB/s sequential
> write), a low-cost SSD(400-500MB/s sequential write), or a high-speed PCI-E
> SSD(1000-1500MB/s sequential write).  Would writing out the data(.xtc,
> .trr, .edr) to a slower hard drive(100-200MB/s sequential write speed)
> cause the Gromacs simulation to slow down?
>

No, but depending on the type of analysis an SSD may matter a lot (and as
opposed to the MD run itself the you may need non-trivial amount of memory
too).


> I don't see any reason that a cheap spinning platter hard drive would slow
> Gromacs simulations down, but I want to make sure I'm not missing
> anything.
>
> Thank you for your time.  I'd be very grateful for any information you can
> give me, or direct me to.
>
> David Berquist
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>