[gmx-developers] Compiling with GPU/mkl on MSVC 2019

[Adriaan Riet]

I downloaded the proposed change, and everything compiled wonderfully.
Thank you so much for the change!

On Tue, Jul 23, 2019 at 4:39 PM Mark Abraham <mark.j.abraham at gmail.com>
wrote:

> Hi,
>
> On Tue, 23 Jul 2019 at 20:47, Adriaan Riet <aar93 at case.edu> wrote:
>
>>
>>
>> Adriaan Riet
>> 801-602-0172
>> adriaan.riet at case.edu
>> Hello,
>>
>> I have been working on building gromacs with GMX_GPU=ON in MSVC 2019 (and
>> cuda 10.1)  I've used the Ninja and Visual studio 2019 Win64 generators,
>> and I'm using the x64 cl.exe as the host compiler, cxx compiler, etc.
>>
>
> Great. We haven't prioritized trying to make such a setup work, as it's
> not commonly used for computation, and we don't have a machine with the MS
> compiler and a NVIDA GPU. But perhaps we can get things in order with your
> help :-)
>
>
>> With GMX_GPU=OFF, I only needed to make sure there were no spaces in some
>> of the cmake flags (I changed CUDA_TOOLKIT_ROOT_DIR, MKL_LIBRARIES,
>> MKL_INCLUDE_DIR), and everything builds and runs fine.
>>
>> With GMX_GPU=ON, I've seen a couple of issues. First, and I've gotten
>> around this by changing CUDA_HOST_COMPILER, the location ${VCInstallDir}bin
>> no longer points to a valid compiler. MS has opted to change the location
>> and provide separate x86 and x64 compilers and cross compilers. I've
>> pointed the CUDA_HOST_COMPILER variable to the same compiler as the other
>> CMAKE_C**_COMPILER variables,
>> C:\Program Files (x86)\Microsoft Visual
>> Studio\2019\Community\VC\Tools\MSVC\14.21.27702\bin\Hostx64\x64\cl.exe
>> I'm not sure that this is the compiler I should be using, as GPU should
>> be single precision, but I wanted to be consistent.
>>
>
> That looks fine.
>
> I'm running into an issue in the CUDA code. Specifically, there are a few
>> lines in nbnxm_cuda.cu that set up for launching the kernel (lines 834,
>> 879, etc):
>>
>> const auto kernelArgs          = prepareGpuKernelArguments(kernelFn,
>> config,
>>
>>  &numColumns,
>>                                                                &xqPtr,
>>
>>  &setFillerCoords,
>>                                                                &d_x,
>>
>>  &d_atomIndices,
>>                                                                &d_cxy_na,
>>                                                                &d_cxy_ind,
>>
>>  &cellOffset,
>>
>>  &numAtomsPerCell);
>>     launchGpuKernel(kernelFn, config, nullptr, "XbufferOps", kernelArgs);
>>
>> the relevant template generating prepareGpuKernelArguments is (I think,
>> from cudautils.cuh):
>>
>> template <typename ... Args>
>> std::array<void *, sizeof ... (Args)> prepareGpuKernelArguments(void
>>                 (*kernel)(Args...),
>>                                                                 const
>> KernelLaunchConfig & /*config */,
>>                                                                 const
>> Args *...          argsPtrs)
>> {
>>     std::array<void *, sizeof ... (Args)> kernelArgs;
>>     prepareGpuKernelArgument(kernel, &kernelArgs, 0, argsPtrs ...);
>>     return kernelArgs;
>> }
>>
>> the compiler throws a fit with this, stating that (I've changed the order
>> to what I understand to be cause/effect):
>>
>> 1. "Error C2782 'std::array<void*,sizeof...(Args)>
>> prepareGpuKernelArguments(void (__cdecl *)(Args...),const
>> KernelLaunchConfig &,const Args *...)': template parameter 'Args' is
>> ambiguous "
>> 2. " 'prepareGpuKernelArguments': no matching overloaded function found
>> 3. " 'kernelArgs': cannot be used before it is initialized
>> 4. " 'kernelArgs': an object of const-qualified type must be initialized
>> 5. " 'void launchGpuKernel<const float3*__restrict ,rvec(*),const
>> int*__restrict ,int,int>(void (__cdecl *)(const float3 *__restrict ,rvec
>> (*),const int *__restrict ,int,int),const KernelLaunchConfig &,CommandEvent
>> *,const char *,const std::array<void *,5> &)': cannot convert argument 5
>> from 'int' to 'const std::array<void *,5> &' nbnxm_cuda.cu 886
>>
>> Seems to me that VS isn't recognizing some feature here. Is this
>> something that might be easy to fix?
>>
>
> Googling around, it seems that MSVC does struggle more with variadic
> templates than the other major compilers. I experimented a bit with
> Compiler Explorer, and I think the problem arises because the three kinds
> of arguments to prepareGpuKernelArguments aren't clear enough to the
> compiler. If so, then I think the proposed change at
> https://gerrit.gromacs.org/c/gromacs/+/12183 will fix it. Other compilers
> still seem to be happy, and it can't affect the quality of the generated
> code (touch wood). If you're able to use the download link there, you can
> get a tarball of source code that you can build in the same way you have
> done with the official tarball so far, and let us know how you get on. (By
> the way, 2019 has had several patch-level releases so far, which you'd
> probably want to take advantage of...)
>
> Mark
>
>
>> Thanks,
>> Adriaan Riet
>> --
>> Gromacs Developers mailing list
>>
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>> posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> or send a mail to gmx-developers-request at gromacs.org.
>
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> or send a mail to gmx-developers-request at gromacs.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20190724/14c0b233/attachment.html>