[gmx-users] Can't run v. 4.6.6 on worker nodes if they are compiled with SIMD

[Justin Lemkul]

On 7/25/14, 4:17 PM, Seyyed Mohtadin Hashemi wrote:
> On Fri, Jul 25, 2014 at 3:00 PM, Seyyed Mohtadin Hashemi <haadah at gmail.com>
> wrote:
>
>> Hi everyone,
>>
>> I'm having a very weird problem with GROMACS 4.6.6:
>>
>> I am currently testing out GPU capabilities and was trying to compile
>> GROMACS with CUDA (v6.0). I can not make this work if I compile GROMACS
>> with SIMD, no matter what kernel I choose - I have tried everything from
>> SSE2 to AVX_256.
>>
>> The log-in node, where I compile, has AMD Interlagos CPUs (worker nodes
>> use Xeon E5-2630 and are equipped with Tesla K20), but I do not think this
>> is the problem - I have compiled GROMACS, using the log-in node, without
>> CUDA but with AVX_256 SIMD and everything works. As soon as CUDA is added
>> to the mix, I get "Illegal Instruction" every time I try to run on the
>> worker nodes.
>>
>> Compiling on worker nodes gives the same result. However, as soon as I set
>> SIMD=None everything works and I am able to run simulation using GPUs, this
>> is regardless of if I use log-in node or worker node to compile.
>>
>>
>> The cmake string used to configure is:
>> ccmake .. -DCMAKE_INSTALL_PREFIX=/work/gromacs4gpu -DGMX_DOUBLE=OFF
>> -DGMX_DEFAULT_SUFFIX=OFF -DGMX_BINARY_SUFFIX=_4gpu -DGMX_LIBS_SUFFIX=_4gpu
>> -DGMX_GPU=ON -DBUILD_SHARED_LIBS=OFF -DGMX_PREFER_STATIC_LIBS=ON
>> -GMX_MPI=OFF -DGMX_CPU_ACCELERATION=AVX_256
>>
>> CUDA v6.0 and FFTW v3.3.4 (single precision) libs are set globally and
>> correctly identified by GROMACS. To remove OpenMPI as a problem I am
>> compiling without it (compiling with OpenMPI produced the same behavior as
>> without), once I have found the error I will compile with OpenMPI v1.6.5.
>>
>> I get these warnings during the configuration, nothing important:
>>
>>   A BLAS library was not found by CMake in the paths available to it.
>> Falling back on the GROMACS internal version of the BLAS library instead.
>> This is fine for normal usage.
>>
>>   A LAPACK library was not found by CMake in the paths available to it.
>> Falling back on the GROMACS internal version of the LAPACK library instead.
>> This is fine for normal usage.
>>
>> I am currently trying to compile and test GROMACS 5.0 to see if it also
>> exhibits the same behavior.
>>
>> I hope that someone can point me in the direction of a possible solution,
>> if not then I will file a bug report.
>>
>> Regards,
>> Mohtadin
>>
>
> Forgot to attached the md.log. Please find attached md.log for both v4.6.6
> and v5.0
>

The list does not accept attachments.  Please post the files somewhere for download.

-Justin

-- 
==================================================

Justin A. Lemkul, Ph.D.
Ruth L. Kirschstein NRSA Postdoctoral Fellow

Department of Pharmaceutical Sciences
School of Pharmacy
Health Sciences Facility II, Room 601
University of Maryland, Baltimore
20 Penn St.
Baltimore, MD 21201

jalemkul at outerbanks.umaryland.edu | (410) 706-7441
http://mackerell.umaryland.edu/~jalemkul

==================================================