[gmx-users] nvcc fatal: 'avx512f': expected a number

Wed Oct 24 14:38:06 CEST 2018

Hi,

On Wed, Oct 24, 2018 at 11:50 AM Ehsan Moravveji <e.moravveji at gmail.com>
wrote:

> Dear Mark,
>
> Sincere thanks for your swift reply, and apologies for the duplicate
> message; I realized that my original message had no subject, so, I thought
> it is more proper to send the message with a meaningful topic;
>

OK

> Regarding your comment, I am building GROMACS from source, and without
> intervention from EasyBuild. In order to be transparent, I also put my full
> cmake command arguments in the original message as well. Since some other
> dependencies are installed by EasyBuild as modules, I am setting some of
> the paths in a compatible way on our system, hence e.g. EBROOTCUDA, etc.
>

Then you will have a easier time if you start with a simple invocation,
rather than manually configuring a whole host of things, including CUDA
flags. If e.g. those flags are wrongly escaped for the different wrappers,
then you will get mysterious issues like this :-) So I suggest you start
with

cmake -DGMX_SIMD=AVX_512 -DCUDA_TOOLKIT_ROOT_DIR=whatever

and gradually add the complexity that you think is needed (e.g. MKL, MPI,
NVML...)

To double check your feedback, let me ask the following:
> I am explicitly setting the flag -DGMX_SIMD='AVX_512'. Is it equivalent to
>

That's a GROMACS cmake flag.

> the flag you proposed (-mavx512f), or I have to add the latter to the
>

That's a flag for the compiler. They're not the same. But when the GROMACS
build tries to target AVX512, it checks what flags works and organizes to
use them. I assume that the way you're adding things manually to the CUDA
flags is breaking e.g. the tricks that FindCUDA.cmake has to do in order to
invoke the CUDA compiler.

Mark

> argument list to cmake, too?
>
> Sincere thanks,
> Ehsan
>
>
> On Wed, Oct 24, 2018 at 10:42 AM Mark Abraham <mark.j.abraham at gmail.com>
> wrote:
>
> > Hi,
> >
> > You sent this email yesterday. Please be a little more patient.
> >
> > Unfortunately, I have no idea how EasyBuild works, and suspect that the
> > problem lies in the way it is wrapping arguments that then get passed to
> > nvcc. If you can reproduce this issue with a simple call to cmake with
> > minimal extra stuff, then we can explore whether there's an issue with
> the
> > GROMACS build system. But since -mavx512f is a valid compiler flag that
> > GROMACS checked was valid before using it, I suspect the problem isn't
> > ours.
> >
> > Mark
> >
> > On Wed, Oct 24, 2018 at 9:55 AM Ehsan Moravveji <e.moravveji at gmail.com>
> > wrote:
> >
> > > Dear GMX users,
> > >
> > > I would like to share a difficulty installing GROMACS/2018.3 with you,
> > > and kindly ask for your wisdom.
> > >
> > > I use GCC/6.4.0 on a Linux cluster with CentOS 7.5 kernel and GPU
> > > nodes equipped with 4 x P100 Nvidia devices.
> > >
> > >
> > > Below is the cmake configuration that I use, followed by the error (in
> > > boldface font) that I
> > > receive. I am not sure if there is still an extra flag to set to
> resolve
> > > this issue, or some other steps must be taken?
> > >
> > > I appreciate your input.
> > >
> > > --
> > > Kind regards,
> > > Ehsan Moravveji
> > >
> > > ###########################################################
> > > cmake "${cwd}/${dir_src}" \
> > >   -DGMX_SIMD='AVX_512' \
> > >   -DBLAS_mkl_LIBRARY="${DIR_MKL_LIB}/libmkl_rt.so" \
> > >   -DLAPACK_mkl_lapack_LIBRARY="DIR_MKL_LIB/libmkl_sequential.so" \
> > >   -DBLAS_guide_LIBRARY="${DIR_MKL_LIB}/libmkl_rt.so" \
> > >   -DCMAKE_C_COMPILER="${GMX_MPICC}"
> -DCMAKE_CXX_COMPILER="${GMX_MPICXX}"
> > \
> > >   -DMPI_C_COMPILER="${GMX_MPICC}" -DMPI_CXX_COMPILER="${GMX_MPICXX}" \
> > >   -DCMAKE_MPI_C_COMPILER="${GMX_MPICC}"
> > > -DCMAKE_MPI_CXX_COMPILER="${GMX_MPICXX}" \
> > >   -DCMAKE_INSTALL_PREFIX="${dir_install}" \
> > >   -DCMAKE_PREFIX_PATH="${DIR_MKL_LIB}" \
> > >   -DCMAKE_VERBOSE_MAKEFILE=ON \
> > >   -DGMX_MPI=ON \
> > >   -DGMX_GPU=ON \
> > >   -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF -DEXTRA_CXX_FLAGS="-std=c++11" \
> > >   -DGMX_USE_NVML=ON -DGMX_BUILD_OWN_FFTW=ON \
> > >   -DHWLOC_INCLUDE_DIRS="${EBROOTHWLOC}/include"
> > > -DHWLOC_LIBRARIES="${EBROOTHWLOC}/lib/libhwloc.so" \
> > >   -DCUDA_HOST_COMPILER="${PATH_NVCC}"
> > > -DNVML_INCLUDE_DIR="${EBROOTCUDA}/include" \
> > >   -DNVML_LIBRARY="${EBROOTCUDA}/lib64/stubs/libnvidia-ml.so" \
> > >   -DCUDA_TOOLKIT_ROOT_DIR="${EBROOTCUDA}"
> > > -DCUDA_SDK_ROOT_DIR="${EBROOTCUDA}" \
> > >   -DCUDA_PROPAGATE_HOST_FLAGS=ON -DCUDA_VERBOSE_BUILD=ON \
> > >   -DGMX_CUDA_TARGET_SM="${SM_VER}"
> -DGMX_CUDA_TARGET_COMPUTE="${SM_VER}"
> > \
> > >   -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF \
> > >   -DCUDA_CUDART_LIBRARY="${EBROOTCUDA}/lib64/libcudart.so"
> > > -DCUDA_CUDA_LIBRARY="${EBROOTCUDA}/lib64/libcuda.so" \
> > >
> > >
> > >
> >
> -DCUDA_NVCC_FLAGS="-gencode;arch=compute_${SM_VER},code=compute_${SM_VER};-use_fast_math;-D_FORCE_INLINES"
> > > \
> > >   -DGMX_PREFER_STATIC_LIBS=OFF -DGMX_X11=OFF
> > >
> > > ###########################################################
> > >
> > > -- Generating dependency file:
> > >
> > >
> >
> /work/GROMACS/gromacs-2018.3-build/src/gromacs/CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_cuda.cu.o.NVCC-depend
> > > /software/CUDA/9.2.148/bin/nvcc -M -D__CUDACC__
> > > /work/GROMACS/gromacs-2018.3/src/gromacs/mdlib/nbnxn_cuda/
> nbnxn_cuda.cu
> > -o
> > >
> > >
> >
> /work/GROMACS/gromacs-2018.3-build/src/gromacs/CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/libgromacs_generated_nbnxn_cuda.cu.o.NVCC-depend
> > > -ccbin /software/CUDA/9.2.148/bin/nvcc -m64 --std
> > > c++11 -Dlibgromacs_EXPORTS -DGMX_DOUBLE=0 -DHAVE_CONFIG_H
> > > -DUSE_STD_INTTYPES_H -Xcompiler
> > >
> > >
> >
> ,\"-mavx512f\",\"-mfma\",\"-fPIC\",\"-O3\",\"-DNDEBUG\",\"-funroll-all-loops\",\"-fexcess-precision=fast\"
> > > -gencode arch=compute_60,code=sm_60 -gencode
> > > arch=compute_60,code=compute_60 -use_fast_math -gencode
> > > arch=compute_60,code=compute_60 -use_fast_math -D_FORCE_INLINES -DNVCC
> > > -I/software/CUDA/9.2.148/include
> > > -I/work/GROMACS/gromacs-2018.3/src/external/lmfit
> > >
> > >
> >
> -I/work/GROMACS/gromacs-2018.3-build/src/contrib/fftw/fftwBuild-prefix/include
> > > -I/work/GROMACS/gromacs-2018.3-build/src
> > > -I/work/GROMACS/gromacs-2018.3/src/external/thread_mpi/include
> > > -I/work/GROMACS/gromacs-2018.3/src
> > > -I/software/hwloc/1.11.11-GCCcore-6.4.0/include
> > > -I/work/GROMACS/gromacs-2018.3/src/external/tng_io/include
> > > -I/work/GROMACS/gromacs-2018.3-build/tng/include*nvcc fatal   :
> > > 'avx512f': expected a number
> > > CMake Error at libgromacs_generated_nbnxn_cuda.cu.o.Release.cmake:219
> > > (message):
> > >   Error generating
> > >
> > >
> > >
> >
> /work/GROMACS/gromacs-2018.3-build/src/gromacs/CMakeFiles/libgromacs.dir/mdlib/nbnxn_cuda/./libgromacs_generated_nbnxn_cuda.cu.o*
> > > --
> > > Gromacs Users mailing list
> > >
> > > * Please search the archive at
> > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > > posting!
> > >
> > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> > >
> > > * For (un)subscribe requests visit
> > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> > > send a mail to gmx-users-request at gromacs.org.
> > >
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> > send a mail to gmx-users-request at gromacs.org.
> >
>
>
> --
> Kind regards,
> Ehsan Moravveji
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>