[gmx-users] Using GROMACS with (AMD) OpenCL

Mon Dec 5 15:08:25 CET 2016

Hi Gregory,

On Sat, Dec 3, 2016 at 7:57 PM, Gregory Man Kai Poon <gpoon at gsu.edu> wrote:

> Hi Szilárd,
>
>
> Thanks for your tips.  It turned out that between installing AMDGPU-PRO,
> then AMD APP SDK, the link to libopencl.so became broken.  Following your
> suggestions (if I interpreted them correctly), I linked against the copy
> installed by AMDGPU-PRO.

Not exactly, what I suggest is to use the libOpenCL shipped by distros. You
can install it together with the dev packages e.g. for
Ubuntu opencl-headers and ocl-icd-libopencl1. This way you only need to
have vendor driver/runtime components (installed typically with the driver)
when you actually lunch the application. This is an advantage in many
cases, especially for local installs.
(Unless one does network installs used across multiple hosts where the
system libOpenCL may not be installed everywhere or not in the same
location.)

That resolved the error and I was able to complete the installation.

 I also took the opportunity (as per your suggestion) to migrate to the
> 2016 release.  I had wanted to stay with 5.1.x because we had many
> simulations generated with 5.1.2 and 5.1.4 which we want to extend, and I
> was wary of potential issues with being 100% forward compatible.  I am
> testing one of these simulations now.  So far the program has not
> complained yet (expected I think) and is using the GPU.  If you know of any
> potential issues with extending simulations created by previous versions of
> GROMACS, I would be most interested in knowing them.
>

I don't know of issues, but do consult the release notes
(http://manual.gromacs.org/documentation/2016/ReleaseNotes/index.html);
there *may* be subtle changes, so do test continuation carefully and if in
doubt consider the trading performance for consistency.

Cheers,
--
Szilárd

>
> Thanks again,
>
> Gregory
>
> ________________________________
> From: gromacs.org_gmx-users-bounces at maillist.sys.kth.se <
> gromacs.org_gmx-users-bounces at maillist.sys.kth.se> on behalf of Szilárd
> Páll <pall.szilard at gmail.com>
> Sent: Saturday, December 3, 2016 10:34:46 AM
> To: Discussion list for GROMACS users
> Subject: Re: [gmx-users] Using GROMACS with (AMD) OpenCL
>
> Hi Gregory,
>
> First of all, I'd strongly recommend you to start with the 2016
> release, 2016.1 has been out for some time. This will be more robust
> and has a lot of performance improvement too especially on AMD (see
> https://na01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fmanual.gromacs.org%2Fdocumentation%2F2016%
> 2FReleaseNotes%2F&data=01%7C01%7Cgpoon%40gsu.edu%
> 7C1c6c949cc26b455fee8f08d41b91eece%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=4w9grG36HpkQGK%2F44RJhN3f9EQrhhcNBGxnXVxjL8WY%3D&reserved=0
> ).
>
> Regarding the link error, to be honest I can't tell what's wrong.
> However, note that you do not necessarily need to link against the
> SDK, I have clarified this in the last version of the docs:
> https://na01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fmanual.gromacs.org%2Fdocumentation%2F2016%
> 2Finstall-guide%2Findex.html%23opencl-gpu-acceleration&
> data=01%7C01%7Cgpoon%40gsu.edu%7C1c6c949cc26b455fee8f08d41b91eece%
> 7C515ad73d8d5e4169895c9789dc742a70%7C0&sdata=
> VKwlCmqzHORZ8PvEEjcYzR0pXx9WtH45S7LpiVUNBY4%3D&reserved=0
>
> While linking against the APPSDK libOpenCL _should_ work,  I'd
> recommend that you try to link against the stock libOpenCL.so. If that
> does not work feel free to share the cmake invocation and cache file
> and we may be able to figure out something.
>
> Cheers,
> --
> Szilárd
>
>
> On Sat, Dec 3, 2016 at 4:02 PM, Gregory Man Kai Poon <gpoon at gsu.edu>
> wrote:
> > Hi Milan and Szilárd,
> >
> >
> > Many thanks for your comments.  So I installed Ubuntu 16.04, installed
> the AMDGPU-PRO driver, installed the AMD APP SDK.  Then I attempted to
> install GROMACS 5.1.4 as follows:
> >
> >
> > cmake .. -DGMX_BUILD_OWN_FFTW=ON -DREGRESSIONTEST_DOWNLOAD=ON
> -DGMX_GPU=ON -DGMX_USE_OPENCL=ON
> >
> >
> > Then when I invoked make, things went fine for a bit, then halted with:
> >
> >
> > [  1%] Built target fftwBuild
> > make[2]: *** No rule to make target '/home/popeye/AMDAPPSDK-3.0/lib/x86_64/libOpenCL.so',
> needed by 'lib/libgromacs.so.1.4.0'.  Stop.
> > CMakeFiles/Makefile2:2026: recipe for target 'src/gromacs/CMakeFiles/libgromacs.dir/all'
> failed
> > make[1]: *** [src/gromacs/CMakeFiles/libgromacs.dir/all] Error 2
> > Makefile:160: recipe for target 'all' failed
> >
> > I am not nearly good enough to make out what is going on, except to
> guess that something may not be correct with the AMDAPPSDK-3.0
> installation.  The folder is there, and I checked the install log -- it
> didn't report any error there.
> >
> >
> > Thanks again for your continued help.
> >
> >
> > Gregory
> >
> > ________________________________
> > From: gromacs.org_gmx-users-bounces at maillist.sys.kth.se <
> gromacs.org_gmx-users-bounces at maillist.sys.kth.se> on behalf of Szilárd
> Páll <pall.szilard at gmail.com>
> > Sent: Friday, December 2, 2016 11:22:29 AM
> > To: Discussion list for GROMACS users
> > Subject: Re: [gmx-users] Using GROMACS with (AMD) OpenCL
> >
> > Hi,
> >
> > While for the pre-Polaris hardware Milan is right, for Polaris you
> > *need* a AMDGPU-PRO (which only runs on newer distros/kernels).
> >
> > The current AMDGPU-PRO from the OpenCL point of view is, AFAIK more or
> > less the same compiler/runtime as the last catalyst release (15.12).
> >
> > One notable caveat is that with AMDGPU-PRO power management seems to
> > be overly aggressive (messed up?) -- at least for the load GROMACS
> > generates. Uou can loose quite some performance unless you force the
> > performance level to "high" (hint use radcard to set
> > power_dpm_force_performance_level=high); this has been obsered on both
> > Polaris and Fiji.
> >
> > See here:
> > https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fwiki.archlinux.org%2Findex.php%2FATI%
> 23Dynamic_power_management&data=01%7C01%7Cgpoon%40gsu.edu%
> 7Cee289917aa5a4b74dc5808d41acf7095%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=H0FRzdUGcKtgWvVRkE7fawUX7PZtmr
> z%2FcwaJq7AaC%2Bo%3D&reserved=0
> >
> > Cheers,
> > --
> > Szilárd
> >
> >
> > On Fri, Dec 2, 2016 at 4:13 PM,  <melichercik at leaf.nh.cas.cz> wrote:
> >> Hi Gregory,
> >> it is possible to use with both - Catalyst or AMDGPU, so you can
> choose. As there won't be (I think) any more updates for Catalyst, I'd
> prefer to use newer AMDGPU (PRO) choice. I don't have Polaris based card,
> only some 79xx/R9 280 (X) (uding Catalyst) and one Fury Nano (used Catalyst
> also, but now AMDGPU PRO). I made the Gromacs' compilation only with the
> Catalyst drivers, but with AMDGPU shouldn't be more complicated. Just
> install the system with AMDGPU (I don't know it "PRO part" is needed,
> probably yes) and try to compile (don't forget about the OpenCL switch). If
> there would be some missing libraries, than you have to install some -dev
> versions of packages. Probably Ubuntu has it's own way how to find the
> right package, but you can also use site at packages.debian.org (as
> Ubuntu is based on Debian).
> >> Anyway I'm trying to have most of things directly from the
> distribution, cause it would automatically upgrades and fixes bugs. Outside
> the distro I have only Gromacs (performancd issues - compilation to exact
> machine - and several version installed) and AMDGPU PRO (on that node with
> Fury Nano, cause Debian doesn't have it in its repositories.
> >> If any problem, plaase ask more with some details.
> >>
> >> Best,
> >>
> >> Milan
> >>
> >> On Fri, Dec 02, 2016 at 01:56:03PM +0000, Gregory Man Kai Poon wrote:
> >>> Hello all,
> >>>
> >>>
> >>> I would like to get some practical advice on setting up GROMACS with
> OpenCL using an AMD GPU (an RX 480 is what I happen to have).  The AMD APP
> SDK indicates that it supports the Catalyst Omega 15.7 driver -- is that
> compatible to the proprietary Catalyst driver for Linux?  Since this
> proprietary driver is only compatible with Ubuntu up to version 15.10, does
> that mean that best software configuration (say for GROMACS 5.1.4) is:
> >>>
> >>>
> >>> Ubuntu 15.10 + propriety Catalyst driver + AMD APP SDK
> >>>
> >>>
> >>> There is also a newer AMDGPU driver that works with Ubuntu 16.04 -- is
> that compatible with the AMD APP SDK and would allow GROMACS to be set up
> on a newer OS?  If I really should be going about some other way (given the
> GPU), please suggest also.
> >>>
> >>>
> >>> I have googled the subject a bit but was unable to find enough info to
> proceed.  Thanks in advance for your insight.
> >>>
> >>>
> >>> Best regards,
> >>>
> >>> Gregory
> >>> --
> >>> Gromacs Users mailing list
> >>>
> >>> * Please search the archive at https://na01.safelinks.
> protection.outlook.com/?url=http%3A%2F%2Fwww.gromacs.org%
> 2FSupport%2FMailing_Lists%2FGMX-Users_List&data=01%7C01%7Cgpoon%40gsu.edu%
> 7Cee289917aa5a4b74dc5808d41acf7095%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=Fy3ZQv%2F6Bcv0OvManZDOOLONJETh5aVB4AhtPgrRjCY%3D&reserved=0
> before posting!
> >>>
> >>> * Can't post? Read https://na01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fwww.gromacs.org%2FSupport%2FMailing_Lists&
> data=01%7C01%7Cgpoon%40gsu.edu%7Cee289917aa5a4b74dc5808d41acf7095%
> 7C515ad73d8d5e4169895c9789dc742a70%7C0&sdata=%2FaHdxnstcsfRO7e%
> 2FeV3Qy2GczTKZygS0Kk2SJ48jI80%3D&reserved=0
> >>>
> >>> * For (un)subscribe requests visit
> >>> https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fmaillist.sys.kth.se%2Fmailman%2Flistinfo%
> 2Fgromacs.org_gmx-users&data=01%7C01%7Cgpoon%40gsu.edu%
> 7Cee289917aa5a4b74dc5808d41acf7095%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=WK39xPDUllxQd0nklwQsNpYSRJCkbr0GJyOMiVP1bPM%3D&reserved=0
> or send a mail to gmx-users-request at gromacs.org.
> >> --
> >> Gromacs Users mailing list
> >>
> >> * Please search the archive at https://na01.safelinks.
> protection.outlook.com/?url=http%3A%2F%2Fwww.gromacs.org%
> 2FSupport%2FMailing_Lists%2FGMX-Users_List&data=01%7C01%7Cgpoon%40gsu.edu%
> 7Cee289917aa5a4b74dc5808d41acf7095%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=Fy3ZQv%2F6Bcv0OvManZDOOLONJETh5aVB4AhtPgrRjCY%3D&reserved=0
> before posting!
> >>
> >> * Can't post? Read https://na01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fwww.gromacs.org%2FSupport%2FMailing_Lists&
> data=01%7C01%7Cgpoon%40gsu.edu%7Cee289917aa5a4b74dc5808d41acf7095%
> 7C515ad73d8d5e4169895c9789dc742a70%7C0&sdata=%2FaHdxnstcsfRO7e%
> 2FeV3Qy2GczTKZygS0Kk2SJ48jI80%3D&reserved=0
> >>
> >> * For (un)subscribe requests visit
> >> https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fmaillist.sys.kth.se%2Fmailman%2Flistinfo%
> 2Fgromacs.org_gmx-users&data=01%7C01%7Cgpoon%40gsu.edu%
> 7Cee289917aa5a4b74dc5808d41acf7095%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=WK39xPDUllxQd0nklwQsNpYSRJCkbr0GJyOMiVP1bPM%3D&reserved=0
> or send a mail to gmx-users-request at gromacs.org.
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at https://na01.safelinks.
> protection.outlook.com/?url=http%3A%2F%2Fwww.gromacs.org%
> 2FSupport%2FMailing_Lists%2FGMX-Users_List&data=01%7C01%7Cgpoon%40gsu.edu%
> 7Cee289917aa5a4b74dc5808d41acf7095%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=Fy3ZQv%2F6Bcv0OvManZDOOLONJETh5aVB4AhtPgrRjCY%3D&reserved=0
> before posting!
> >
> > * Can't post? Read https://na01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fwww.gromacs.org%2FSupport%2FMailing_Lists&
> data=01%7C01%7Cgpoon%40gsu.edu%7Cee289917aa5a4b74dc5808d41acf7095%
> 7C515ad73d8d5e4169895c9789dc742a70%7C0&sdata=%2FaHdxnstcsfRO7e%
> 2FeV3Qy2GczTKZygS0Kk2SJ48jI80%3D&reserved=0
> >
> > * For (un)subscribe requests visit
> > https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fmaillist.sys.kth.se%2Fmailman%2Flistinfo%
> 2Fgromacs.org_gmx-users&data=01%7C01%7Cgpoon%40gsu.edu%
> 7Cee289917aa5a4b74dc5808d41acf7095%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=WK39xPDUllxQd0nklwQsNpYSRJCkbr0GJyOMiVP1bPM%3D&reserved=0
> or send a mail to gmx-users-request at gromacs.org.
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at https://na01.safelinks.
> protection.outlook.com/?url=http%3A%2F%2Fwww.gromacs.org%
> 2FSupport%2FMailing_Lists%2FGMX-Users_List&data=01%7C01%7Cgpoon%40gsu.edu%
> 7C1c6c949cc26b455fee8f08d41b91eece%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=8DAji1k4rr%2BP%2F3CTGuvYva%2F4W0IkP0kqbhsBxt1%2BoyU%3D&
> reserved=0 before posting!
> >
> > * Can't post? Read https://na01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fwww.gromacs.org%2FSupport%2FMailing_Lists&
> data=01%7C01%7Cgpoon%40gsu.edu%7C1c6c949cc26b455fee8f08d41b91eece%
> 7C515ad73d8d5e4169895c9789dc742a70%7C0&sdata=DE5AfpXVQ%2Bg0CP%2BZCUpU8iTp%
> 2F451ZM6EPuBIY%2FP2%2Bsg%3D&reserved=0
> >
> > * For (un)subscribe requests visit
> > https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fmaillist.sys.kth.se%2Fmailman%2Flistinfo%
> 2Fgromacs.org_gmx-users&data=01%7C01%7Cgpoon%40gsu.edu%
> 7C1c6c949cc26b455fee8f08d41b91eece%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=xCN9Fc0G5JMDGNRH26JnSYXhFFtPHAMoeU%2B7Iy0quoA%3D&reserved=0
> or send a mail to gmx-users-request at gromacs.org.
> --
> Gromacs Users mailing list
>
> * Please search the archive at https://na01.safelinks.
> protection.outlook.com/?url=http%3A%2F%2Fwww.gromacs.org%
> 2FSupport%2FMailing_Lists%2FGMX-Users_List&data=01%7C01%7Cgpoon%40gsu.edu%
> 7C1c6c949cc26b455fee8f08d41b91eece%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=8DAji1k4rr%2BP%2F3CTGuvYva%2F4W0IkP0kqbhsBxt1%2BoyU%3D&
> reserved=0 before posting!
>
> * Can't post? Read https://na01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fwww.gromacs.org%2FSupport%2FMailing_Lists&
> data=01%7C01%7Cgpoon%40gsu.edu%7C1c6c949cc26b455fee8f08d41b91eece%
> 7C515ad73d8d5e4169895c9789dc742a70%7C0&sdata=DE5AfpXVQ%2Bg0CP%2BZCUpU8iTp%
> 2F451ZM6EPuBIY%2FP2%2Bsg%3D&reserved=0
>
> * For (un)subscribe requests visit
> https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fmaillist.sys.kth.se%2Fmailman%2Flistinfo%
> 2Fgromacs.org_gmx-users&data=01%7C01%7Cgpoon%40gsu.edu%
> 7C1c6c949cc26b455fee8f08d41b91eece%7C515ad73d8d5e4169895c9789dc74
> 2a70%7C0&sdata=xCN9Fc0G5JMDGNRH26JnSYXhFFtPHAMoeU%2B7Iy0quoA%3D&reserved=0
> or send a mail to gmx-users-request at gromacs.org.
> --
> Gromacs Users mailing list
>
> * Please search the archive at http://www.gromacs.org/
> Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>