[gmx-users] Problems with TI on GPUs

[Mark Abraham]

Hi,

Ok, that's starting to sound like either a code bug or hardware setup bug.
Please try with 5.1.2 (but it looks like maybe you have already) and/or
open an issue at https://redmine.gromacs.org and add your tpr file. I'll
try it on some hardware setups we know we trust. ;-)

Mark

On Tue, 12 Jul 2016 19:18 Yannic Alber <yannic.alber at tu-dortmund.de> wrote:

> Hi,
>
> In short, this works for every combination. But with the big disadvantage
> of being very slow and not using our precious little gpu´s.
>
> Thanks for your fast reply Mark.
>
> > Hi,
> >
> > What happens in your failing cases (or even all of them) when adding -nb
> > cpu, to force the run off the GPU?
> >
> > Mark
> >
> > On Tue, 12 Jul 2016 19:02 Yannic Alber <yannic.alber at tu-dortmund.de>
> > wrote:
> >
> >> Dear all,
> >>
> >> we struggle to get a TI on our computer running. The specifications are
> >> listed below. As you can see, its a two socket, two graphics cards
> >> machine. Therefore, the plan is to run two simulations in parallel. But
> >> we
> >> can't get a single one to run.
> >>
> >> Running on 1 node with total 20 cores, 20 logical cores, 2 compatible
> >> GPUs
> >> Hardware detected:
> >>   CPU info:
> >>     Vendor: GenuineIntel
> >>     Brand:  Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz
> >>     SIMD instructions most likely to fit this hardware: AVX2_256
> >>     SIMD instructions selected at GROMACS compile time: AVX2_256
> >>   GPU info:
> >>     Number of GPUs detected: 2
> >>     #0: NVIDIA GeForce GTX 1080, compute cap.: 6.1, ECC:  no, stat:
> >> compatible
> >>     #1: NVIDIA GeForce GTX 1080, compute cap.: 6.1, ECC:  no, stat:
> >> compatible
> >>
> >> The simulation system in question is a protein-ligand-complex in
> >> TIP3P-water and amber ff99SB as force field.
> >>
> >> Now lets get into the messy details. We tried different mdrun
> >> commandline
> >> argument rotations, for example:
> >>
> >> gmx mdrun -s md.tpr -pin on -ntomp 2 -ntmpi 5 -gpu_id 00000 -deffnm md
> >> (does not work)
> >> gmx mdrun -s md.tpr -pin on -ntomp 5 -ntmpi 2 -gpu_id 00 -deffnm md
> >> (does not work)
> >> gmx mdrun -s md.tpr -pin on -ntomp 10 -ntmpi 1 -gpu_id 0 -deffnm md
> >> (does not work)
> >> gmx mdrun -s md.tpr -deffnm md
> >> (does work, uses the complete compute node including the both gpu´s)
> >>
> >> The error which gromacs gives us, is rather irritating (explanation
> >> follows further down). Here a little excerpt:
> >>
> >> Step 191, time 0.382 (ps)  LINCS WARNING
> >> relative constraint deviation after LINCS:
> >> rms 0.000002, max 0.000010 (between atoms 3421 and 3424)
> >> bonds that rotated more than 30 degrees:
> >>  atom 1 atom 2  angle  previous, current, constraint length
> >>    3702   3703   31.0    0.1090   0.1090      0.1090
> >> Wrote pdb files with previous and current coordinates
> >>
> >> These errors vary, but refer all to a misplacement or unusual rotations.
> >> Gromacs states, that this is because of our "unstable" system. However,
> >> this explanation can be excluded, because the starting configuration of
> >> the
> >> simulations in question already ran 20 ns in gromacs on a CPU-Cluster.
> >>
> >> We also tested different commands for cmake. A example is shown here:
> >>
> >> cmake .. -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DGMX_GPU=on
> >> -DGMX_FFT_LIBRARY=fftw3 -DGMX_BUILD_OWN_FFTW=ON
> >> -DREGRESSIONTEST_DOWNLOAD=ON
> >>
> >> Compilerwise we tried gcc (v.4.8.5) and intel (v.15.0.1).
> >>
> >> I would really appreciate your help and thank you very much in advance.
> >>
> >>
> >> Sincerely,
> >> Yannic
> >>
> >>
> >>
> >>
> >>
> >> --
> >> Gromacs Users mailing list
> >>
> >> * Please search the archive at
> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> >> posting!
> >>
> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>
> >> * For (un)subscribe requests visit
> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> >> send a mail to gmx-users-request at gromacs.org.
> >>
> >
>
>
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>