[gmx-developers] Double Precision on GPUs

Szilárd Páll pall.szilard at gmail.com
Wed Aug 14 14:10:22 CEST 2019 

Hi,

On Thu, Aug 1, 2019 at 3:51 PM David Lister <me at davidlister.ca> wrote:
>
> Hi,
>
> This is great! So it seems like this mixed/fixed precision version would likely solve the issues I am running into with single precision, while being faster than double precision. It also seems that there are other uses like run reproducibility. It seems like a useful feature, which is also not too hard to implement.

Indeed.

> I'm really not sure how open source software development works form this side of the fence, would this be a feature to add to the roadmap? If so, how does that process work?

Sure, but before we move the idea from the list of "would be nice to
have" to "planned features", we need to find the resources, i.e.
someone with time and know-how to drive the implementation. If we have
the commitment, we can plan the feature and help with the execution.

> Also, sorry if this is inadvertently trampling on norms, I just really really want this feature :)

No problem, best to ask if something is unclear.

Cheers,
--
Szilárd

>
> Cheers,
> David
>
> On Wed, Jul 31, 2019 at 2:55 PM Michael R Shirts <Michael.Shirts at colorado.edu> wrote:
>>
>> Fixed precision would be kind of cool for reproducibility.  I doubt I have the time (or the coding ability), but I'm interested to hear discussion in this and advise/brainstorm on the numerical stability and potential integrator issues.
>>
>> Best,
>> ~~~~~~~~~~~~~~~~
>> Michael Shirts
>> Associate Professor
>> michael.shirts at colorado.edu
>> http://www.colorado.edu/lab/shirtsgroup/
>> Phone: (303) 735-7860
>> Office: JSCBB C123
>> Department of Chemical and Biological Engineering
>> University of Colorado Boulder
>>
>>
>> On 7/31/19, 12:38 PM, "gromacs.org_gmx-developers-bounces at maillist.sys.kth.se on behalf of Szilárd Páll" <gromacs.org_gmx-developers-bounces at maillist.sys.kth.se on behalf of pall.szilard at gmail.com> wrote:
>>
>>     Hi,
>>
>>     My gut feeling is that, while it would be slightly more effort, a
>>     mixed precision with 64-bit fixed precision coordinates would be
>>     better for performance (in particular on consumer GPUs). We've also
>>     discussed in the past fixed precision (32 or 64-bit depending on the
>>     use-case) force/energy accumulation/reductions too which would also
>>     allow a mix-precision mode with reproducibility. Even if at first not
>>     tuned for production, it would at least be very useful to have such a
>>     set of GPU kernels for "mdrun -reprod" mode.
>>
>>     I had plans to work on such extensions, but other projects have been
>>     keeping me busy, so I've not had the time. If somebody is interested
>>     in contributing, I can help to at least coordinate, and when I'm a bit
>>     less busy also with the coding.
>>
>>     Cheers,
>>     --
>>     Szilárd
>>
>>     On Wed, Jul 31, 2019 at 7:33 PM David Lister <me at davidlister.ca> wrote:
>>     >
>>     > That sounds really interesting! It makes sense, that it would be the case. How would I go about implementing that to test?
>>     >
>>     > Would this also help with minimization? I find that minimized geometries are also fairly different between single and double precision. It seems as though single precision considers vdw much less than double precision, whereas double precision doesn't. This is most apparent in cases without solvents.
>>     >
>>     > Cheers,
>>     > David
>>     >
>>     > On Wed, Jul 31, 2019 at 10:45 AM Michael R Shirts <Michael.Shirts at colorado.edu> wrote:
>>     >>
>>     >>
>>     >>
>>     >> I think that in many cases the single precision force computation is not what is limiting the accuracy. It is rather the integration and in particular the constraints using single precision.
>>     >>
>>     >>
>>     >>
>>     >> I recall talking over old results with Erik (many years ago) that showed exactly this, and seem to have been found by other people; integration and constraints cannot really be done well in single precision; but if they are in double precision (or even mixed precision, with careful analysis of rounding steps), with the force in single precision, then numerical performance is much, much better.
>>     >>
>>     >>
>>     >>
>>     >> Best,
>>     >>
>>     >> ~~~~~~~~~~~~~~~~
>>     >>
>>     >> Michael Shirts
>>     >>
>>     >> Associate Professor
>>     >>
>>     >> michael.shirts at colorado.edu
>>     >>
>>     >> http://www.colorado.edu/lab/shirtsgroup/
>>     >>
>>     >> Phone: (303) 735-7860
>>     >>
>>     >> Office: JSCBB C123
>>     >>
>>     >> Department of Chemical and Biological Engineering
>>     >>
>>     >> University of Colorado Boulder
>>     >>
>>     >>
>>     >>
>>     >>
>>     >>
>>     >> From: <gromacs.org_gmx-developers-bounces at maillist.sys.kth.se> on behalf of Berk Hess <hess at kth.se>
>>     >> Reply-To: "gmx-developers at gromacs.org" <gmx-developers at gromacs.org>
>>     >> Date: Wednesday, July 31, 2019 at 8:42 AM
>>     >> To: "gmx-developers at gromacs.org" <gmx-developers at gromacs.org>
>>     >> Subject: Re: [gmx-developers] Double Precision on GPUs
>>     >>
>>     >>
>>     >>
>>     >> Hi,
>>     >>
>>     >> I think that in many cases the single precision force computation is not what is limiting the accuracy. It is rather the integration and in particular the constraints using single precision. Another solution is to have integration and constraints in double precision and (parts of) the force computation in single precision.
>>     >>
>>     >> Cheers,
>>     >>
>>     >> Berk
>>     >>
>>     >> On 7/31/19 4:35 PM, David Lister wrote:
>>     >>
>>     >> Hello,
>>     >>
>>     >>
>>     >>
>>     >> Sorry this isn't part of the email thread from earlier, I found the thread in the archives and decided to chime in. Not sure how to reply to old threads, but the link is here for consistency. https://mailman-1.sys.kth.se/pipermail/gromacs.org_gmx-developers/2019-July/010488.html
>>     >>
>>     >>
>>     >>
>>     >> I am also be very interested in having double precision supported on GPUs. I'm using gromacs for a project that stretches the target use case and have found the single precision just doesn't give good results. The same simulation run in both single and double gives different results, with the double precision simulation giving the results expected by theory. Unfortunately though, double precision is much slower.
>>     >>
>>     >>
>>     >>
>>     >> Having GPU support for double precision would be a really big help for me as well.
>>     >>
>>     >>
>>     >>
>>     >> Cheers,
>>     >>
>>     >> David
>>     >>
>>     >>
>>     >>
>>     >>
>>     >>
>>     >> --
>>     >> Gromacs Developers mailing list
>>     >>
>>     >> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before posting!
>>     >>
>>     >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>     >>
>>     >> * For (un)subscribe requests visit
>>     >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or send a mail to gmx-developers-request at gromacs.org.
>>     >
>>     > --
>>     > Gromacs Developers mailing list
>>     >
>>     > * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before posting!
>>     >
>>     > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>     >
>>     > * For (un)subscribe requests visit
>>     > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or send a mail to gmx-developers-request at gromacs.org.
>>     --
>>     Gromacs Developers mailing list
>>
>>     * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before posting!
>>
>>     * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>>     * For (un)subscribe requests visit
>>     https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or send a mail to gmx-developers-request at gromacs.org.
>>
>> --
>> Gromacs Developers mailing list
>>
>> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or send a mail to gmx-developers-request at gromacs.org.
>
> --
> Gromacs Developers mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or send a mail to gmx-developers-request at gromacs.org.

More information about the gromacs.org_gmx-developers mailing list