[gmx-users] NVIDIA GTX cards in Rackable servers, how do you do it ?

[Szilárd Páll]

On Fri, Feb 20, 2015 at 2:17 PM, David McGiven <davidmcgivenn at gmail.com> wrote:
> Dear Gromacs users and developers,
>
> We are thinking about buying a new cluster of ten or twelve 1U/2U machines
> with 2 Intel Xeon CPU's 8-12 cores each. Some of the 2600v2 or v3 series.
> Not yet clear the details, we'll see.

If you can afford them get the 14/16 or 18 core v3 Haswells, those are
*really* fast, but a pair can cost as much as a decent car.

Get IVB (v2) if it saves you a decent amount of money compared to v3.
The AVX2 with FMA of the Haswell chips is great, but if you run
GROMACS with GPUs on them my guess is that a higher frequency v2 will
be more advantageous than the v3's AVX2 support. Won't swear on this
as I have not tested thoroughly.

> I've been told in this list that NVIDIA GTX offer the best
> performance/price ratio for gromacs 5.0.

Yes, that is the case.

> However, I am wondering ... How do you guys use the GTX cards in rackable
> servers ?
>
> GTX cards are consummer grade, for personal workstations, gaming, and so on
> and it's nearly impossible to find any servers manufacturer like HP, Dell,
> SuperMicro, etc. to certify that those cards will function properly on
> their servers.

Certification can be an issue - unless you buy many and you can cut a
deal with a company. There are some companies that do certify servers,
but AFAIK most/all are US-based. I won't do public a long
advertisement here, but you can find many names if you browse NVIDIA's
GPU computing site (and as a matter of fact the AMBER GPU site is
quite helpful in this respect too).

You can consider getting vanilla server nodes and plug the GTX cards
in yourself. In general, I can recommend Supermicro, they have pretty
good value servers from 1 to 4U. The easiest is to use the latter
because GTX cards will just fit vertically, but it will be a serious
waste of rack-space. With a bit of tinkering you may be able to get
GTX cards into 3U, but you'll either need cards with connectors on the
back or 90 deg angled 4-pin PCIE power cables. Otherwise you can only
fit the cards with PCIE raisers and I have no experience with that
setup, but I know some build denser machines with GTX cards.

Cheer,

--
Szilárd

> What are your views about this ?
>
> Thanks.
>
> Best Regards
> --
> Gromacs Users mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.