[gmx-users] GPU support on macOS 10.14

Thu Jul 11 11:36:14 CEST 2019

Hi Mark and Mike,

thank you very much for the answer. I finally managed to compile and use Gromacs (2019.3) with openCL support on my MacBook Pro (2016). However, I still had some errors with the PME calculations after using mdrun. This seems to be a known issue and PME calculations on GPUs are deactivated in the next release of Gromacs according to this Bug report: http://redmine.gromacs.org/issues/2719 . However I would not recommend to do this (hopefully only temporary) fix based on my experience. I report my approach to 1) hopefully help other people to install Gromacs with GPU support on MacOS and 2) tell why it could be better to not deactivate PME calculations on MacOS:

First I installed Gromacs without GPU support. The performance for my test system (cubic box, polymer melt of ca. 8000 atoms with some water molecules inside) is much better when I compile it with gcc (ca. 60 ns/day) than with clang (ca. 30 ns/day). I used clang from LLVM and not the preinstalled version on the MacBook. In both cases, I compiled Gromacs with AVX2_256 and external fftw3.

The problem is that I could not compile Gromacs with OpenCl support and the gcc compiler (probably because Apples OpenCL implementation is compiled with the default clang compiler). I then followed the suggested installation from Michael and could install Gromacs, but got PME errors mentioned in the bug report with preinstalled clFFT. I tried running mdrun -pme cpu. This approach did not produce any error, but after about 10000-30000 steps (depending on the initial state) of NpT equilibration the system crashes with some LINCS warnings from nonbonded interactions. Running the same system with mdrun -nb cpu -pme cpu works perfectly for a much longer time, but this means that you do not use the GPU (including OpenCL with clFFT) because bonded intaeractions were automatically assigned to the CPUs by Gromacs.

15-inch MacBook Pros contain two GPUs: An AMD Radeon Pro (in my case 460) and an Intel HD Graphics (in my case 530). If you compile Gromacs in the suggested way, Gromacs will only detect the AMD GPU. If you add -DGMX_OPENCL_NB_CLUSTER_SIZE=4, which is required for Intel GPUs, Gromacs will detect both GPUs. In this case I get still problems with PME calculations from mdrun (some missing interactions). But if I force Gromacs to only use the Intel GPU (via -gmx mdrun -gpu_id 1) I can use the Intel GPU without any problem. So it seems that the error in the described bug is only relevant for Apples OpenCL implementation in combination with clFFT and AMD GPUs. Regarding the fact, that 13-inch MacBook Pros run only with an Intel Iris Graphics GPU I assume that deactivating PME calculations von MacOS in the next release of Gromacs is not such a good idea, but I did not test it with an Iris Graphics GPU.

It would be nice if some of the people involved in this bug report (Erik Lindahl, Szillard Pall, Mark Abraham) could check and comment on this issue.

Regarding the performance: With OpenCL support (on the Intel HD Graphics) I get about 45 ns/day for the same test system, about 50% more than without GPU support, but still less than with gcc and CPU only. I assume this relation will change if I go to much bigger systems when more NB/PME calculations run on the GPU.

Falk

Gesendet: Mittwoch, 03. Juli 2019 um 17:05 Uhr
Von: "Michael Williams" <michael.r.c.williams at gmail.com>
An: gmx-users at gromacs.org
Betreff: Re: [gmx-users] GPU support on macOS 10.14
Hi Falk,

I actually compiled Gromacs with openCL GPU support on my MacBook Pro (2017) a couple months ago (with some help from others on this mailing list). I’ve pasted below the cmake settings that worked. You might be able to find more details searching the list for that thread... it took me a little while to get it working. Definitely worthwhile though. Have a good one,

Mike

cmake .. -DCMAKE_INSTALL_PREFIX=/Users/michael/.local/apps/gromacs-2018.5-apple-clang-omp-ocl -DCMAKE_LIBRARY_PATH=/Users/michael/.local/lib -DCMAKE_INCLUDE_PATH=/Users/michael/.local/include -DCMAKE_C_COMPILER=/usr/bin/clang -DCMAKE_CXX_COMPILER=/usr/bin/clang++ -DCMAKE_C_FLAGS="-Xpreprocessor -fopenmp -lomp -L/Users/michael/.local/lib -I/Users/michael/.local/include" -DCMAKE_CXX_FLAGS="-Xpreprocessor -fopenmp -lomp -L/Users/michael/.local/lib -I/Users/michael/.local/include" -DGMX_FFT_LIBRARY=fftw3 -DGMX_GPU=ON -DGMX_USE_OPENCL=ON

My $HOME/.local directory is the prefix I used to install appropriate versions of hwloc (1.11.12), libomp (7.0.1), and fftw3 (3.3.8) that I compiled with the system’s default clang (in OSX 10.14.3, Mojave).

> On Jul 3, 2019, at 8:41 AM, Mark Abraham <mark.j.abraham at gmail.com> wrote:
>
> Hi,
>
>> On Wed., 3 Jul. 2019, 15:47 Falk Hoffmann, <falk_hoffmann at gmx.de> wrote:
>>
>> Hi!
>>
>> I have a question regarding the GPU support of GROMACS for MacOS. The
>> newest MacBooks (after 2015) are equipped with AMD GPUs which means that
>> GROMACS cannot be compiled with CUDA. GROMACS could be compiled with OpenCL
>> until MacOS 10.13. But this is not possible for MacBooks which have been
>> updated to Mojave (10.14) because Apple deprecated OpenCL (and OpenGL) in
>> this OS in favor of Metal2.
>>
>
> Indeed Apple deprecated such support, but it is not yet removed and AFAIK
> is still viable for running codes that require them. Apple's own website
> declares many models that support OpenCL, but does not note whether there
> are OS version constraints... I assume that 10.14 and up don't install such
> components by default any more, but I haven't heard that it is impossible
> to do so... Can anyone confirm or deny?
>
>> Regarding
>
>> the improved performance of GPUs especially for nonbonded interactions of
>> biomolecules in bigger systems it would be very useful to use the power of
>> the AMD GPUs in the future. Is there any plan to provide support for Metal2
>> in new releases of GROMACS in the near future?
>>
>
> It would be wildly unlikely for GROMACS to support a second proprietary
> language, and particularly one that is not used in the HPC sector. I hope a
> way forward will emerge, but all we know now is that it won't be Metal2 :-)
>
> Mark
>
> Or is there another way to compile GROMACS with GPU support under
>> MacOS>=10.14? I tried it with OpenCL, but of course it did not work. I
>> could perfectly compile it without GPU support and run it on CPUs only, but
>> this is not my intention.
>>
>> Kind regards,
>> Falk
>>
>> BTW: If this email is better placed in the GROMACS developers list, please
>> move it there.
>> --
>> Gromacs Users mailing list
>>
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List[http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List] before
>> posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists[http://www.gromacs.org/Support/Mailing_Lists]
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users[https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users] or
>> send a mail to gmx-users-request at gromacs.org.
> --
> Gromacs Users mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List[http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List] before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists[http://www.gromacs.org/Support/Mailing_Lists]
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users[https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users] or send a mail to gmx-users-request at gromacs.org.
--
Gromacs Users mailing list

* Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List[http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List] before posting!

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists[http://www.gromacs.org/Support/Mailing_Lists]

* For (un)subscribe requests visit
https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users[https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users] or send a mail to gmx-users-request at gromacs.org.