[gmx-developers] OpenACC Development

[Millad Ghane]

Thanks Erik for your reply too.

Just two quick things:

1- I wasn't planning to propose a universal model for every architecture
(CPU, GPU, Xeon PHI) by OpenACC. I was looking into replacing CUDA kernels
with OpenACC ones. My goal was to replace mostly any CUDA codes with
OpenACC (or even make them co-exist with each other).

2- I have to report back to my supervisor. So, my question is to you (she
might ask me) is that: did you tried to port the code to OpenACC and then
found out that it is not a good fit? Or, you are saying it is not good
based on your experience?

BTW, I totally agree (and have experience) with the fact that writing
low-level code is so much profitable, productive, and close-to-optimal
compared to a writing a high-level code, but the code loses its
portability (the fact is that low-level codes are close to hardware than
high-level ones). That's why some of these compilers advertise that they
are produce close-to-architecture codes and they suggest that it is better
to rely on us and write codes using high-level languages!


Best Regards,
Millad





> Hi Millad,
>
> As Berk explained, the problem is that there is no optimal OpenACC
> programming model you can phrase everything in.
>
> Xeon Phi, Cuda, AMD, and x86 have completely different properties. At some
> point we will have to choose a layout for higher-level data structures,
> and there is no single model that will be optimal for all of them.
>
>
> OpenACC will never be able to compete with:
>
> 1) The low-level CUDA optimization we already have for NVIDIA
> 2) The low-level SIMD implementation we have for all CPU architectures
> 3) The low-level OpenCL implementation we have for AMD
> 4) The low-level AVX512 kernels we for KNL.
>
>
> We are still not entirely happy with the KNL performance, but that is not
> something that can be fixed in the kernels - instead we need to rewrite
> the highest-level code so we can have hundreds of threads executing
> efficiently. OpenACC in any kernels won’t fix that.
>
>
> Cheers,
>
> Erik
>
>
>
>
>> On 01 Jul 2016, at 18:17, Millad Ghane <mghane at cs.uh.edu> wrote:
>>
>> Hi Berk,
>>
>> Thanks for your reply.
>>
>> I know that OpenACC is a programming language. What I meant by OpenACC
>> architecture was "OpenACC software architecture". I know that
>> eventually,
>> the OpenACC code is executed by an underlying hardware architecture
>> (which
>> could be NVidia or AMD/ATI or even Xeon PHI from Intel).
>>
>> What I hope to achieve: introducing a set of kernels that are optimized
>> for OpenACC.
>> What I have done: trying to parallelize the 4x4 Plain-C code for CPU
>> using
>> OpenACC compiler directives.
>>
>> The problem is that the code and especially the data structures (in
>> Plain-C CPU kernels) get complex in kernel level to some extent.
>> Therefore, that was what I was looking for: introducing kernels for
>> OpenACC programming model.
>>
>>
>> Best Regards,
>> Millad
>>
>>
>>
>>
>>> Hi Millad,
>>>
>>> Welcome to the list.
>>>
>>> GROMACS aims for close to optimal performance on all relevant hardware.
>>> To achieve this, we write highly optimized non-bonded (and other)
>>> kernels for all relevant architectures. Currently we have plain-C CPU
>>> kernels (extremely slow), SIMD, CUDA and OpenCL non-bonded kernels. The
>>> optimization is not so much in the arithmetics, but rather in the data
>>> layout. OpenACC is not an architecture, but a programming standard. So
>>> you would need to choose a target architecture for your OpenACC
>>> "acceleration" and then choose one of our kernel types. But the only
>>> gain of this would be semi-automated offloading. The question is then
>>> if
>>> the offloading will be efficient enough.
>>>
>>> Cheers,
>>>
>>> Berk
>>>
>>> On 2016-07-01 02:05, Millad Ghane wrote:
>>>> Hello everyone,
>>>>
>>>> I am a PhD student in computer science at University of Houston.
>>>> Currently, in this Summer as an intern, I am working with the physics
>>>> department in our school to work on GROMACS in order to port it to
>>>> OpenACC. My adviser for this project is Prof. Cheung.
>>>>
>>>> My understanding is that GROMCS currently supports NVIDIA GPUs (and
>>>> also
>>>> SIMDs on CPUs), however my job is to investigate the ability of
>>>> transferring the code to OpenACC, which is more heterogeneous and
>>>> ubiquitous compared to CUDA.
>>>>
>>>> My question regarding the development of GROMACS is that whether you
>>>> are
>>>> supporting or planning to support OpenACC currently or in future. And,
>>>> if
>>>> you are not supporting OpenACC, my question is that how can I
>>>> introduce
>>>> new "kernel functions" for supporting OpenACC. How much work should be
>>>> done?
>>>>
>>>> Based on my understanding, you had different kernel function for
>>>> different
>>>> architectures (CPU, CPU with SIMD, GPU). I wanted to know how much
>>>> effort
>>>> is required to introduce new architecture like OpenACC?
>>>>
>>>> Before getting in touch with you, I dig into some of the code and
>>>> tried
>>>> to
>>>> parallelize the CPU version of kernel code using OpenACC constructs,
>>>> but
>>>> the code gets messy and the data is not transferred correctly on the
>>>> device. So, my hope is to introduce new kernels for OpenACC like the
>>>> way
>>>> you introduced different ones for CPUs and GPUs. This way, we have
>>>> more
>>>> controls over data transfers and kernel codes.
>>>>
>>>>
>>>> I hope I was clear enough and make you interested.
>>>>
>>>>
>>>> Best Regards,
>>>> Millad Ghane
>>>> Computer Science Department
>>>> University of Houston
>>>>
>>>>
>>>
>>> --
>>> Gromacs Developers mailing list
>>>
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>>> posting!
>>>
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>> * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>>> or
>>> send a mail to gmx-developers-request at gromacs.org.
>>>
>>
>>
>> --
>> Gromacs Developers mailing list
>>
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>> posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> or send a mail to gmx-developers-request at gromacs.org.
>
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or
> send a mail to gmx-developers-request at gromacs.org.