[gmx-developers] gromacs 5.1rc1 OpenCL problem with Parrinello-Rahman

Wouter Boomsma wb at bio.ku.dk
Mon Jul 20 21:09:02 CEST 2015


Hi,

I don't know if this is in any way related, but the OpenMM community has
had problems with the OpenCL implementation on OSX for several years
now. From the main download page:

*NOTE: Bug in recent versions of Mac OSX produces errors with the OpenMM
> OpenCL platform*. We are currently working with Apple to resolve the
> issue. In the meantime, if you use the OpenCL platform, we recommend that
> you do NOT upgrade your OS beyond 10.7.4. Those running the CUDA platform
> are unaffected. *Last updated: Nov. 21, 2012*


Cheers,
Wouter


On Mon, Jul 20, 2015 at 8:07 PM, Szilárd Páll <pall.szilard at gmail.com>
wrote:

> Hi,
>
> At this point my guess is that the source of the issue is the particular
> version of the Apple OpenCL runtime or compiler you have. Unfortunately,
> our version info does not contain any OpenCL runtime/compiler version
> information, but you should be able to query that and compare it against
> someone else's version:
> system_profiler | grep -A 10 OpenCL
>
> Cheers,
>
> --
> Szilárd
>
> On Mon, Jul 20, 2015 at 7:50 PM, Carlo Camilloni <
> carlo.camilloni at gmail.com> wrote:
>
>> Hi again,
>>
>> I don’t know if you had the opportunity to reproduce this bug, in the
>> meantime I have compiled it with gcc 4.8, openMP and sse4.1,
>> again cpu results are consistently right while OpenCL results are
>> consistently wrong, so it looks like it is NOT a problem of Clang 6
>> but it could be either a problem of the gpu driver or maybe OpenCL
>> itself, I don’t know..
>>
>> this is the new log
>>
>> GROMACS version:    VERSION 5.1-rc1
>> Precision:          single
>> Memory model:       64 bit
>> MPI library:        thread_mpi
>> OpenMP support:     enabled (GMX_OPENMP_MAX_THREADS = 32)
>> GPU support:        enabled
>> OpenCL support:     enabled
>> invsqrt routine:    gmx_software_invsqrt(x)
>> SIMD instructions:  SSE4.1
>> FFT library:        fftw-3.3.4-sse2-avx
>> RDTSCP usage:       enabled
>> C++11 compilation:  disabled
>> TNG support:        enabled
>> Tracing support:    disabled
>> Built on:           Mon 20 Jul 2015 18:21:43 BST
>> Built by:           carlo [CMAKE]
>> Build OS/arch:      Darwin 13.4.0 x86_64
>> Build CPU vendor:   GenuineIntel
>> Build CPU brand:    Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz
>> Build CPU family:   6   Model: 62   Stepping: 4
>> Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm mmx
>> msr nonstop_tsc pcid pclmuldq pdcm pdpe1gb popcnt pse rdrnd rdtscp sse2
>> sse3 sse4.1 sse4.2 ssse3 tdt x2apic
>> C compiler:         /opt/local/bin/gcc GNU 4.8.5
>> C compiler flags:    -msse4.1    -Wextra -Wno-missing-field-initializers
>> -Wno-sign-compare -Wpointer-arith -Wall -Wno-unused -Wunused-value
>> -Wunused-parameter  -O3 -DNDEBUG -funroll-all-loops
>> -fexcess-precision=fast  -Wno-array-bounds
>> C++ compiler:       /opt/local/bin/g++ GNU 4.8.5
>> C++ compiler flags:  -msse4.1    -Wextra -Wno-missing-field-initializers
>> -Wpointer-arith -Wall -Wno-unused-function  -O3 -DNDEBUG -funroll-all-loops
>> -fexcess-precision=fast  -Wno-array-bounds
>> Boost version:      1.58.0 (external)
>> OpenCL include dir: /System/Library/Frameworks/OpenCL.framework
>> OpenCL library:     /System/Library/Frameworks/OPENCL.framework
>> OpenCL version:     1.2
>>
>>
>> Best,
>> Carlo
>>
>>
>>
>> On 17 Jul 2015, at 13:54, Carlo Camilloni <carlo.camilloni at gmail.com>
>> wrote:
>>
>> Hi,
>>
>> Unfortunately on the Mac Pro I am running os x 10.9.5 and the newer clang
>> is not yet available so I cannot test it
>> with a different version. Erik if you have time to tell me what to change
>> to get gromacs compiled with gcc I can give it a try
>>
>> Carlo
>>
>>
>> On 16 Jul 2015, at 20:28, Mark Abraham <mark.j.abraham at gmail.com> wrote:
>>
>> Hi,
>>
>> Philip Fowler made a recent positive report on OpenCL+AMD+Mac and he was
>> using /usr/bin/cc Clang 6.1.0.6020053, so there may be an issue with Clang
>> 6.0 (whatever real clang version that is...)
>>
>> Mark
>>
>> On Thu, Jul 16, 2015 at 7:20 PM Erik Lindahl <erik.lindahl at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> The one obvious difference I can see is that you have clang 6.0.0, while
>>> at least my Macbook pro says 6.1.0.
>>>
>>> Could you check if there is any update available for the developer
>>> tools, and if so does that help?
>>>
>>> It is actually much better to use gcc since you will then also use all
>>> CPU cores. However, that requires patching a header file in /usr/include;
>>> I’ll dig up what I did there - we need to put it in the documentation
>>> anyway.
>>>
>>> Cheers,
>>>
>>> Erik
>>>
>>> From: Carlo Camilloni <carlo.camilloni at gmail.com>
>>> <carlo.camilloni at gmail.com>
>>> Reply: Carlo Camilloni <carlo.camilloni at gmail.com>>
>>> <carlo.camilloni at gmail.com>
>>> Date: 16 Jul 2015 at 19:07:13
>>> To: Erik Lindahl <erik.lindahl at gmail.com>> <erik.lindahl at gmail.com>
>>> Cc: gromacs.org_gmx-developers at maillist.sys.kth.se
>>> <gromacs.org_gmx-developers at maillist.sys.kth.se>>
>>> <gromacs.org_gmx-developers at maillist.sys.kth.se>,
>>> gmx-developers at gromacs.org <gmx-developers at gromacs.org>>
>>> <gmx-developers at gromacs.org>
>>>
>>> Subject:  Re: [gmx-developers] gromacs 5.1rc1 OpenCL problem with
>>> Parrinello-Rahman
>>>
>>>  Hi Erik,
>>>
>>> here it is:
>>>
>>>
>>> GROMACS version:    VERSION 5.1-rc1
>>> Precision:          single
>>> Memory model:       64 bit
>>> MPI library:        none
>>> OpenMP support:     disabled
>>> GPU support:        enabled
>>> OpenCL support:     enabled
>>> invsqrt routine:    gmx_software_invsqrt(x)
>>> SIMD instructions:  AVX_256
>>> FFT library:        fftw-3.3.4-sse2-avx
>>> RDTSCP usage:       enabled
>>> C++11 compilation:  disabled
>>> TNG support:        enabled
>>> Tracing support:    disabled
>>> Built on:           Thu 16 Jul 2015 09:34:14 BST
>>> Built by:           carlo@ [CMAKE]
>>> Build OS/arch:      Darwin 13.4.0 x86_64
>>> Build CPU vendor:   GenuineIntel
>>> Build CPU brand:    Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz
>>> Build CPU family:   6   Model: 62   Stepping: 4
>>> Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm
>>> mmx msr nonstop_tsc pcid pclmuldq pdcm pdpe1gb popcnt pse rdrnd rdtscp sse2
>>> sse3 sse4.1 sse4.2 ssse3 tdt x2apic
>>> C compiler:
>>> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
>>> Clang 6.0.0.6000057
>>> C compiler flags:    -mavx    -Wall -Wno-unused -Wunused-value
>>> -Wunused-parameter -Wno-unknown-pragmas  -O3 -DNDEBUG
>>> C++ compiler:
>>> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
>>> Clang 6.0.0.6000057
>>> C++ compiler flags:  -mavx    -Wextra -Wno-missing-field-initializers
>>> -Wpointer-arith -Wall -Wno-unused-function -Wno-unknown-pragmas  -O3
>>> -DNDEBUG
>>> Boost version:      1.58.0 (external)
>>> OpenCL include dir: /System/Library/Frameworks/OpenCL.framework
>>> OpenCL library:     /System/Library/Frameworks/OPENCL.framework
>>> OpenCL version:     1.2
>>>
>>>
>>> Running on 1 node with total 12 logical cores, 2 compatible GPUs
>>> Hardware detected:
>>>   CPU info:
>>>     Vendor: GenuineIntel
>>>     Brand:  Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz
>>>     Family:  6  model: 62  stepping:  4
>>>     CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm mmx
>>> msr nonstop_tsc pcid pclmuldq pdcm pdpe1gb popcnt pse rdrnd rdtscp sse2
>>> sse3 sse4.1 sse4.2 ssse3 tdt x2apic
>>>     SIMD instructions most likely to fit this hardware: AVX_256
>>>     SIMD instructions selected at GROMACS compile time: AVX_256
>>>   GPU info:
>>>     Number of GPUs detected: 2
>>>     #0: name: AMD Radeon HD - FirePro D500 Compute Engine, vendor: AMD,
>>> device version: OpenCL 1.2 , stat: compatible
>>>     #1: name: AMD Radeon HD - FirePro D500 Compute Engine, vendor: AMD,
>>> device version: OpenCL 1.2 , stat: compatible
>>>
>>> Carlo
>>>
>>>
>>>
>>> On 16 Jul 2015, at 18:59, Erik Lindahl <erik.lindahl at gmail.com> wrote:
>>>
>>> Hi Carlo,
>>>
>>> IIRC, I managed to get it working on a Mac Pro with AMD Firepro D300
>>> GPUs. Please post the contents of the top of your logfile (where it says
>>> everything about the compilers & config), and mention what hardware you
>>> tried it on - then we’ll see if we can reproduce it.
>>>
>>> Cheers,
>>>
>>> Erik
>>>
>>> From: Carlo Camilloni <carlo.camilloni at gmail.com>
>>> <carlo.camilloni at gmail.com>
>>> Reply: gmx-developers at gromacs.org <gmx-developers at gromacs.org>>
>>> <gmx-developers at gromacs.org>
>>> Date: 16 Jul 2015 at 18:52:21
>>> To: gromacs.org_gmx-developers at maillist.sys.kth.se
>>> <gromacs.org_gmx-developers at maillist.sys.kth.se>>
>>> <gromacs.org_gmx-developers at maillist.sys.kth.se>
>>> Subject:  Re: [gmx-developers] gromacs 5.1rc1 OpenCL problem with
>>> Parrinello-Rahman
>>>
>>>  Hi,
>>>
>>> I tested the OpenCL kernel on my macbook (nvidia gpu) and here it
>>> produces the correct forces,
>>> so it could be a problem related to amd+osx, or maybe to some specific
>>> compiler/os version
>>>
>>> Carlo
>>>
>>>
>>> > On 15 Jul 2015, at 17:42, Carlo Camilloni <carlo.camilloni at gmail.com>
>>> wrote:
>>> >
>>> > Hi,
>>> >
>>> > these are the tests that fail:
>>> >
>>> > FAILED. Check checkpot.out (12 errors), checkforce.out (3516 errors)
>>> file(s) in dd121 for dd121
>>> > FAILED. Check checkpot.out (10 errors), checkforce.out (4027 errors)
>>> file(s) in nbnxn-energy-groups for nbnxn-energy-groups
>>> > FAILED. Check checkpot.out (26 errors), checkforce.out (2998 errors)
>>> file(s) in nbnxn-free-energy for nbnxn-free-energy
>>> > FAILED. Check checkpot.out (26 errors), checkforce.out (2998 errors)
>>> file(s) in nbnxn-free-energy-vv for nbnxn-free-energy-vv
>>> > FAILED. Check checkpot.out (11 errors), checkforce.out (4039 errors)
>>> file(s) in nbnxn-ljpme-geometric for nbnxn-ljpme-geometric
>>> > FAILED. Check checkpot.out (14 errors), checkforce.out (52 errors)
>>> file(s) in nbnxn-ljpme-LB for nbnxn-ljpme-LB
>>> > FAILED. Check checkpot.out (14 errors), checkforce.out (52 errors)
>>> file(s) in nbnxn-ljpme-LB-geometric for nbnxn-ljpme-LB-geometric
>>> > FAILED. Check checkpot.out (10 errors), checkforce.out (4029 errors)
>>> file(s) in nbnxn-vdw-force-switch for nbnxn-vdw-force-switch
>>> > FAILED. Check checkpot.out (10 errors), checkforce.out (4032 errors)
>>> file(s) in nbnxn-vdw-potential-switch for nbnxn-vdw-potential-switch
>>> > FAILED. Check checkpot.out (4 errors), checkforce.out (250 errors)
>>> file(s) in nbnxn-vdw-potential-switch-argon for
>>> nbnxn-vdw-potential-switch-argon
>>> > FAILED. Check checkpot.out (10 errors), checkforce.out (4027 errors)
>>> file(s) in nbnxn_pme for nbnxn_pme
>>> > FAILED. Check checkpot.out (10 errors), checkforce.out (4027 errors)
>>> file(s) in nbnxn_pme_order5 for nbnxn_pme_order5
>>> > FAILED. Check checkpot.out (10 errors), checkforce.out (4027 errors)
>>> file(s) in nbnxn_pme_order6 for nbnxn_pme_order6
>>> > FAILED. Check checkpot.out (9 errors), checkforce.out (4028 errors)
>>> file(s) in nbnxn_rf for nbnxn_rf
>>> > FAILED. Check checkpot.out (2 errors), checkforce.out (4 errors)
>>> file(s) in nbnxn_rzero for nbnxn_rzero
>>> > FAILED. Check mdrun.out, md.log file(s) in nbnxn_vsite for nbnxn_vsite
>>> > FAILED. Check checkpot.out (13 errors), checkforce.out (15512 errors)
>>> file(s) in octahedron for octahedron
>>> > FAILED. Check mdrun.out, md.log file(s) in position-restraints for
>>> position-restraints
>>> > FAILED. Check mdrun.out, md.log file(s) in pull_constraint for
>>> pull_constraint
>>> > FAILED. Check checkpot.out (10 errors), checkforce.out (4021 errors)
>>> file(s) in pull_cylinder for pull_cylinder
>>> > FAILED. Check checkpot.out (11 errors), checkforce.out (39054 errors)
>>> file(s) in swap_x for swap_x
>>> > FAILED. Check checkpot.out (11 errors), checkforce.out (39053 errors)
>>> file(s) in swap_y for swap_y
>>> > FAILED. Check checkpot.out (12 errors), checkforce.out (39054 errors)
>>> file(s) in swap_z for swap_z
>>> > 23 out of 60 complex tests FAILED
>>> > FAILED. Check mdrun.out, md.log file(s) in expanded for expanded
>>> > FAILED. Check mdrun.out, md.log file(s) in transformAtoB for
>>> transformAtoB
>>> > 2 out of 10 freeenergy tests FAILED
>>> >
>>> >
>>> > Carlo
>>> >
>>> >
>>> >>
>>> >>
>>> >> Message: 4
>>> >> Date: Wed, 15 Jul 2015 15:35:13 +0000
>>> >> From: Mark Abraham <mark.j.abraham at gmail.com>
>>> >> To: gmx-developers at gromacs.org,
>>> >> gromacs.org_gmx-developers at maillist.sys.kth.se
>>> >> Subject: Re: [gmx-developers] gromacs 5.1rc1 OpenCL problem with
>>> >> Parrinello-Rahman
>>> >> Message-ID:
>>> >> <CAMNuMATveVRRyBBwn312xrY+w3M7deC2Hs3A7PZnaeugkw+VVA at mail.gmail.com>
>>> >> Content-Type: text/plain; charset="utf-8"
>>> >>
>>> >> Hi,
>>> >>
>>> >> Thanks. If a difference of that magnitude can be seen, then it should
>>> also
>>> >> show up when running the regressiontests (e.g. cmake
>>> >> -DREGRESSIONTEST_DOWNLOAD=on and then make check) as a failure
>>> >> of complex/nbnxn-ljpme-LB (which is the only P-R test that can run on
>>> the
>>> >> GPU). If other tests fail, then the problem is actually more
>>> widespread.
>>> >>
>>> >> It may be that there is some issue with some part of the
>>> Mac+clang+OpenCL
>>> >> stack - we didn't target it during development, and at the last
>>> minute when
>>> >> Erik was unexpectedly able to get it to compile. I don't know if he
>>> got
>>> >> tests to pass. Erik?
>>> >>
>>> >> Mark
>>> >>
>>> >> On Wed, Jul 15, 2015 at 5:22 PM Carlo Camilloni <
>>> carlo.camilloni at gmail.com>
>>> >> wrote:
>>> >>
>>> >>>
>>> >>> Dear Mark and Szilard,
>>> >>>
>>> >>> thanks for your answer. I filed a bug in redmine but in the meantime
>>> I was
>>> >>> running more tests and I am a bit scared by what I found:
>>> >>>
>>> >>> what I have done is the following I have performed a single step run
>>> with
>>> >>> gmx51-rc1 compiled with cuda, again clang and so on
>>> >>> and compared the forces on the first step with -nb cpu or not (I am
>>> using
>>> >>> -pforce 1), the forces are identical:
>>> >>>
>>> >>> ie.:
>>> >>>
>>> >>> cuda-gpu
>>> >>>
>>> >>> step 0 atom 1 x 3.940 5.612 2.226 force 1.90839e+03
>>> >>> step 0 atom 2 x 3.852 5.659 2.211 force 4.24845e+02
>>> >>> step 0 atom 3 x 3.979 5.665 2.303 force 6.89472e+02
>>> >>> step 0 atom 4 x 3.992 5.610 2.139 force 7.42053e+02
>>> >>>
>>> >>>
>>> >>> cpu:
>>> >>>
>>> >>> step 0 atom 1 x 3.940 5.612 2.226 force 1.90839e+03
>>> >>> step 0 atom 2 x 3.852 5.659 2.211 force 4.24845e+02
>>> >>> step 0 atom 3 x 3.979 5.665 2.303 force 6.89472e+02
>>> >>> step 0 atom 4 x 3.992 5.610 2.139 force 7.42053e+02
>>> >>>
>>> >>> if I do the same test on the version compiled with OpenCL
>>> >>>
>>> >>> cpu:
>>> >>>
>>> >>> (the former are done on my macbook pro avx2_256 this latter on a
>>> MacPro
>>> >>> avx_256, this should
>>> >>> explain the small differences in the forces)
>>> >>>
>>> >>> step 0 atom 1 x 3.940 5.612 2.226 force 1.90838e+03
>>> >>> step 0 atom 2 x 3.852 5.659 2.211 force 4.24848e+02
>>> >>> step 0 atom 3 x 3.979 5.665 2.303 force 6.89470e+02
>>> >>> step 0 atom 4 x 3.992 5.610 2.139 force 7.42043e+02
>>> >>>
>>> >>> opencl-gpu:
>>> >>> step 0 atom 1 x 3.940 5.612 2.226 force 1.48597e+03
>>> >>> step 0 atom 2 x 3.852 5.659 2.211 force 6.26942e+02
>>> >>> step 0 atom 3 x 3.979 5.665 2.303 force 8.44032e+02
>>> >>> step 0 atom 4 x 3.992 5.610 2.139 force 7.92786e+02
>>> >>>
>>> >>> I am afraid there is something wrong in OpenCL kernels.
>>> >>>
>>> >>> I am using the topol-nvt-nogen.tpr I have uploaded on redmine.
>>> >>>
>>> >>> Best,
>>> >>> Carlo
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Gromacs Developers mailing list
>>> >>>
>>> >>> * Please search the archive at
>>> >>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
>>> before
>>> >>> posting!
>>> >>>
>>> >>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>> >>>
>>> >>> * For (un)subscribe requests visit
>>> >>>
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>>> >>> or send a mail to gmx-developers-request at gromacs.org.
>>> >>>
>>> >> -------------- next part --------------
>>> >> An HTML attachment was scrubbed...
>>> >> URL: <
>>> http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150715/fb1d4126/attachment.html
>>> >
>>> >>
>>> >> ------------------------------
>>> >>
>>> >> --
>>> >> Gromacs Developers mailing list
>>> >>
>>> >> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>>> posting!
>>> >>
>>> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>> >>
>>> >> * For (un)subscribe requests visit
>>> >>
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>>> or send a mail to gmx-developers-request at gromacs.org.
>>> >>
>>> >> End of gromacs.org_gmx-developers Digest, Vol 135, Issue 17
>>> >> ***********************************************************
>>> >
>>>
>>> --
>>> Gromacs Developers mailing list
>>>
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>>> posting!
>>>
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>> * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>>> or send a mail to gmx-developers-request at gromacs.org.
>>>  --
>>> Erik Lindahl <erik.lindahl at gmail.com>
>>> Professor of Biophysics, Dept. Biochemistry & Biophysics, Stockholm
>>> University
>>> Professor of Theoretical biophysics, Dept. Theoretical Physics, Royal
>>> Inst. Technology
>>> Science for Life Laboratory, Box 1031, 17121 Solna, Sweden
>>>
>>>  --
>>> Erik Lindahl <erik.lindahl at gmail.com>
>>> Professor of Biophysics, Dept. Biochemistry & Biophysics, Stockholm
>>> University
>>> Professor of Theoretical biophysics, Dept. Theoretical Physics, Royal
>>> Inst. Technology
>>> Science for Life Laboratory, Box 1031, 17121 Solna, Sweden
>>> --
>>> Gromacs Developers mailing list
>>>
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>>> posting!
>>>
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>> * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>>> or send a mail to gmx-developers-request at gromacs.org.
>>
>>
>>
>>
>> --
>> Gromacs Developers mailing list
>>
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>> posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> or send a mail to gmx-developers-request at gromacs.org.
>>
>
>
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> or send a mail to gmx-developers-request at gromacs.org.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150720/9f2077e3/attachment-0003.html>


More information about the gromacs.org_gmx-developers mailing list