[gmx-users] Can't run v. 4.6.6 on worker nodes if they are compiled with SIMD

[Seyyed Mohtadin Hashemi]

On Fri, Jul 25, 2014 at 3:17 PM, Seyyed Mohtadin Hashemi <haadah at gmail.com>
wrote:

> On Fri, Jul 25, 2014 at 3:00 PM, Seyyed Mohtadin Hashemi <haadah at gmail.com
> > wrote:
>
>> Hi everyone,
>>
>> I'm having a very weird problem with GROMACS 4.6.6:
>>
>> I am currently testing out GPU capabilities and was trying to compile
>> GROMACS with CUDA (v6.0). I can not make this work if I compile GROMACS
>> with SIMD, no matter what kernel I choose - I have tried everything from
>> SSE2 to AVX_256.
>>
>> The log-in node, where I compile, has AMD Interlagos CPUs (worker nodes
>> use Xeon E5-2630 and are equipped with Tesla K20), but I do not think this
>> is the problem - I have compiled GROMACS, using the log-in node, without
>> CUDA but with AVX_256 SIMD and everything works. As soon as CUDA is added
>> to the mix, I get "Illegal Instruction" every time I try to run on the
>> worker nodes.
>>
>> Compiling on worker nodes gives the same result. However, as soon as I
>> set SIMD=None everything works and I am able to run simulation using GPUs,
>> this is regardless of if I use log-in node or worker node to compile.
>>
>>
>> The cmake string used to configure is:
>> ccmake .. -DCMAKE_INSTALL_PREFIX=/work/gromacs4gpu -DGMX_DOUBLE=OFF
>> -DGMX_DEFAULT_SUFFIX=OFF -DGMX_BINARY_SUFFIX=_4gpu -DGMX_LIBS_SUFFIX=_4gpu
>> -DGMX_GPU=ON -DBUILD_SHARED_LIBS=OFF -DGMX_PREFER_STATIC_LIBS=ON
>> -GMX_MPI=OFF -DGMX_CPU_ACCELERATION=AVX_256
>>
>> CUDA v6.0 and FFTW v3.3.4 (single precision) libs are set globally and
>> correctly identified by GROMACS. To remove OpenMPI as a problem I am
>> compiling without it (compiling with OpenMPI produced the same behavior as
>> without), once I have found the error I will compile with OpenMPI v1.6.5.
>>
>> I get these warnings during the configuration, nothing important:
>>
>>  A BLAS library was not found by CMake in the paths available to it.
>> Falling back on the GROMACS internal version of the BLAS library instead.
>> This is fine for normal usage.
>>
>>  A LAPACK library was not found by CMake in the paths available to it.
>> Falling back on the GROMACS internal version of the LAPACK library instead.
>> This is fine for normal usage.
>>
>> I am currently trying to compile and test GROMACS 5.0 to see if it also
>> exhibits the same behavior.
>>
>> I hope that someone can point me in the direction of a possible solution,
>> if not then I will file a bug report.
>>
>> Regards,
>> Mohtadin
>>
>
> Forgot to attached the md.log. Please find attached md.log for both v4.6.6
> and v5.0
>
> Unfortunately, v5.0 seems to exhibit the same behavior as v4.6.6.
>

The logs:
4.6.6: http://pastebin.com/vxagEEZC
5.0: http://pastebin.com/eMemFq1J