[gmx-users] Gromacs 4.6.7 with MPI and OpenMP

[Malcolm Tobias]

Mark,

On Friday 08 May 2015 13:48:30 Mark Abraham wrote:

> What kind of simulation are you testing with? A reaction-field water box
> will have almost nothing to do on the CPU, so no real change with #threads.
> Check with your users, but a PME test case is often more appropriate.

I have no idea, I have very little background with molecular dynamics.  All I can say is I can run the very same input files on our old cluster and observe the expected behavior, but with this same system on our new cluster I'm not seeing the OpenMP threads launch like I'd expect.

> > > > Number of CPUs detected (16) does not match the number reported by
> > OpenMP
> > > > (1).
...
> 
> OK. Well that 1 reported by mdrun is literally the return value from
> calling omp_get_num_procs(), so the solution is to look for what part of
> the ecosystem is setting that to 1 and give that a slap ;-) IIRC the use of
> -ntomp 4 means mdrun will go and use 4 threads anyway, but it'd be good to
> fix the wider context.

I think this is likely the problem, for whatever reason my build of GROMACs thinks there's only 1 core and is not launching the OpenMP threads.  

I've never been mistaken for a C programmer, but when I run my little Hello World OpenMP code it seems to behave as I expect.  If I request 8 of the CPU cores on a 16 core system:

qsub -I -l nodes=1:ppn=8:gpus=2,walltime=24:00:00

omp_get_num_procs reports 8 cores:

[mtobias at gpu22 C]$ ./a.out 
hello world
hello world
 8
hello world
 8

FWIW, I ran the same GROMACs run outside of the queuing system to verify that the CPUSETs were not causing the issue.

> Sure, you need root access. You don't need it for running cmake when that
> runs a pile of unsecure code ;-)

Fair enough point.  I guess I've learned to trust scientific programmers too much over the years ;-)
  
> YMMV but hyperthreads were generally not useful with GROMACS 4.6. That is
> changing for newer hardware and GROMACS, however.

We've got hyperthreading disabled on all of our systems.

Malcolm

> 
> Mark
> 
> 
> > Cheers,
> > Malcolm
> >
> >
> > --
> > Malcolm Tobias
> > 314.362.1594
> >
> >
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> > send a mail to gmx-users-request at gromacs.org.
> >
> 
-- 
Malcolm Tobias
314.362.1594