[gmx-users] [gmx-developers] GTX 680 non detected on OS X

Uliano Guerrini uliano.guerrini at unimi.it
Sun Jun 12 12:05:20 CEST 2016 

Hi Szilárd,

thanks for the shell trick (it worked smoothly). 

I’m happy to report I tamed it (partly, this is an "ad hoc” fix needs to have /usr/local/cuda symlink to the actual installation) with this patch (applied on release-2016 branch):

uliano at MacPro:~/local/src/gromacs$ git log -p -1
commit f20accef9216d9e7acb4989ca83897d33344b357
Author: uliano <uliano.guerrini at gmail.com>
Date:   Sun Jun 12 11:39:40 2016 +0200

    hardcoded fix to have cuda runtime loaded on OSX without changing DYLD_LIBRARY_PATH which is unavailble since OSX 10.11

diff --git a/cmake/gmxManageGPU.cmake b/cmake/gmxManageGPU.cmake
index 208b47f..c4e6d3c 100644
--- a/cmake/gmxManageGPU.cmake
+++ b/cmake/gmxManageGPU.cmake
@@ -99,6 +99,7 @@ if(GMX_GPU OR GMX_GPU_AUTO AND CAN_RUN_CUDA_FIND_PACKAGE)
             endif()
         endforeach(elem)
     endif()
+    set(CUDA_LIBRARIES /usr/local/cuda/lib/libcudart.dylib -Wl,-rpath,/usr/local/cuda/lib)
 endif()
 
 # Depending on the current vale of GMX_GPU and GMX_GPU_AUTO:
uliano at MacPro:~/local/src/gromacs$ 

cmake command line used was the following:

uliano at MacPro:~/local/src/gromacs_build$ cmake ../gromacs -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DGMX_GPU=on -DCMAKE_PREFIX_PATH="/opt/local;/usr/local/cuda"  -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda -DCMAKE_INSTALL_PREFIX=/opt/gromacs  -DCUDA_HOST_COMPILER=/usr/bin/clang++ -DGMX_SIMD=SSE4.1

and now the shared library is there:

uliano at MacPro:~/Desktop/DimeroGromacs$ otool -L /opt/gromacs/bin/gmx
/opt/gromacs/bin/gmx:
	@rpath/libgromacs.2.dylib (compatibility version 2.0.0, current version 2.0.0)
	@rpath/libcudart.7.5.dylib (compatibility version 0.0.0, current version 7.5.27)
	/opt/local/lib/libhwloc.5.dylib (compatibility version 12.0.0, current version 12.8.0)
	/opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	/opt/local/lib/libfftw3f.3.dylib (compatibility version 8.0.0, current version 8.4.0)
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
	/opt/local/lib/libomp/libomp.dylib (compatibility version 5.0.0, current version 5.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)


I can comment the relevant cmake code with some observations:

    if(APPLE)
        set(CUDA_USE_STATIC_CUDA_RUNTIME OFF CACHE STRING "Use the static version of the CUDA runtime library if available")
    endif()

    find_package(CUDA ${REQUIRED_CUDA_VERSION} ${FIND_CUDA_QUIETLY})

    # Cmake 2.8.12 (and CMake 3.0) introduced a new bug where the cuda
    # library dir is added twice as an rpath on APPLE, which in turn causes
    # the install_name_tool to wreck the binaries when it tries to remove this
    # path. Since this is set inside the cuda module, we remove the extra rpath
    # added in the library string - an rpath is not a library anyway, and at
    # least for Gromacs this works on all CMake versions. This should be
    # reasonably future-proof, since newer versions of CMake appear to handle
    # the rpath automatically based on the provided library path, meaning
    # the explicit rpath specification is no longer needed.
    if(APPLE AND (CMAKE_VERSION VERSION_GREATER 2.8.11))
        foreach(elem ${CUDA_LIBRARIES})
            if(elem MATCHES "-Wl,.*")
                list(REMOVE_ITEM CUDA_LIBRARIES ${elem})
            endif()
        endforeach(elem)
    endif()
    set(CUDA_LIBRARIES /usr/local/cuda/lib/libcudart.dylib -Wl,-rpath,/usr/local/cuda/lib)

What actually happens (at least on my machine) is that set(CUDA_USE_STATIC_CUDA_RUNTIME OFF doesn’t work as after 
find_package(CUDA ${REQUIRED_CUDA_VERSION} ${FIND_CUDA_QUIETLY})

CUDA_LIBRARIES contains /usr/local/cuda/lib/libcudart_static.a;-Wl,-rpath,/usr/local/cuda/lib

the following foreach loop just removes -Wl,-rpath,/usr/local/cuda/lib from it (and it is required!)

I believe that something between gmxManageGPU.cmake and findCUDA should still be fixed. 

My cmake knowledge is very limited (close to nothing), however my (presumably rare) OSX 10.11 + CUDA 7.5 machine is available for testing if someone has hints.

Uliano


> Il giorno 11 giu 2016, alle ore 17:02, Szilárd Páll <pall.szilard at gmail.com> ha scritto:
> 
> Uliano,
> 
> Thanks for the update! Such info is always useful to share.
> 
> Weird indeed and as I have not much OS X experience I can't comment on
> most of this stuff -- except one bash-ism that may help rendering your
> shell useless after setting the dynamic library path.
> 
> [env] MY_ENV_VAR=value gmx mdrun ...
> 
> will set the MY_ENV_VAR variable only for the process you prefix it with!
> 
> Cheers,
> --
> Szilárd
> 
> 
> On Sat, Jun 11, 2016 at 8:09 AM, Uliano Guerrini
> <uliano.guerrini at unimi.it> wrote:
>> HI all, I’m resuming this thread with some updates
>> 
>> I “believed" to have set them, however for (silly?) security reason since OSX 10.11 with the introduction of System Integrity Protection (SIP) Apple disallows the user to see the content od environment variables starting with DYLD* and to propagate their modification from one process to the child, so setting them in .bash_profile is useless
>> 
>> however I was able to run with GPU by
>> 
>> cmake ../gromacs-5.1.2 -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DGMX_GPU=on -DCMAKE_PREFIX_PATH="/opt/local;/Developer/NVIDIA/CUDA-7.5/lib"  -DCUDA_TOOLKIT_ROOT_DIR=/Developer/NVIDIA/CUDA-7.5 -DCMAKE_INSTALL_PREFIX=/opt/gromacs  -DCUDA_HOST_COMPILER=/usr/bin/clang++ -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF
>> 
>> and then
>> 
>> export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-7.5/lib:$DYLD_LIBRARY_PATH
>> 
>> in the shell just before issuing
>> 
>> gmx mdrun
>> 
>> however that shell become completely useless for most part of other commands as it has lost any previous $DYLD_LIBRARY_PATH (being not accessible it is copied an empty string)
>> 
>> What I find puzzling is that while the dynamic linking works (helped by the $DYLD_LIBRARY_PATH) there appears to be no trace of libcudart.dylib in the executable
>> uliano at MacPro:/opt/gromacs/bin$ otool -L gmx
>> gmx:
>>        @rpath/libgromacs.1.dylib (compatibility version 1.0.0, current version 1.2.0)
>>        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
>>        /opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
>>        /opt/local/lib/libfftw3f.3.dylib (compatibility version 8.0.0, current version 8.4.0)
>>        /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
>>        /opt/local/lib/libomp/libomp.dylib (compatibility version 5.0.0, current version 5.0.0)
>>        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)
>> uliano at MacPro:/opt/gromacs/bin$
>> 
>> I have tried different options in the cmake command line hoping to have it included in the executable (as all other shared library actually are!) but without success
>> 
>> Uliano
>> 
>>> Il giorno 01 giu 2016, alle ore 17:35, Uliano Guerrini <uliano.guerrini at unimi.it> ha scritto:
>>> 
>>> 
>>>> Il giorno 01 giu 2016, alle ore 11:18, Mark Abraham <mark.j.abraham at gmail.com> ha scritto:
>>>> 
>>>> Hi,
>>>> 
>>>> Googling around, I see that some places recommend things such as
>>>> 
>>>> export PATH=/Developer/NVIDIA/CUDA-6.0/bin:$PATH
>>>> export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-6.0/lib:$DYLD_LIBRARY_PATH
>>>> 
>>>> 
>>>> Does that seem adaptable to your setup as a possible fix for getting
>>>> dynamic linking to work?
>>>> 
>>>> Mark
>>> 
>>> I had already set them
>>> 
>>> Uliano
>>> 
>>> --
>>> Gromacs Users mailing list
>>> 
>>> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>>> 
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>> 
>>> * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.
>> 
>> 
>> 
>> --
>> Gromacs Users mailing list
>> 
>> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>> 
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> 
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.
> -- 
> Gromacs Users mailing list
> 
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
> 
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> 
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.

More information about the gromacs.org_gmx-users mailing list