[gmx-users] problems with intel I7 (2.67 GHz)

[Martti Louhivuori]

[Alexey Shvetsov, 18.12.2009]:
>On Пятница 18 декабря 2009 23:47:34 David van der Spoel wrote:
>>Alexey Shvetsov wrote:
>>>On Пятница 18 декабря 2009 19:29:15 XAvier Periole wrote:
>>>>Dears,
>>>>
>>>>we have been trying to run gmx-4.0.X on new machines with the
>>>>Intel I7 CPUs. It is a quad core intel on which multi-threading is
>>>>a "default" feature. This makes the system to behave as a 8 cores
>>>>machine.
>>>>
>>>>We observed some strange behavior of gromacs. Note that is
>>>>certainly not due to gromacs itself.
>>>>
>>>>Running jobs using 4 cores we have observed the following behavior:
>>>>- the info on the output files (log, xtc, edr ...) is not always printed
>>>>- gromacs switches cores, eg: cores 1,2,4,5, and then 2,3,5,6 ... and 
>>>>this with no apparent logic.
>>>>
>>>>Jobs using 8 cores become extremely slow, more than expected,
>>>>and if running at all.
>>>>
>>>>Running 2 jobs with 4 cores each, one was running with 4 cores, the
>>>>second was using only 2 cores although mpirun was asked to use 4.
>>>>
>>>>This is more disturbing since it was also possible to run two jobs
>>>>using 4 cores each successfully and have effectively increased
>>>>efficiency: each job running at 5/6 of the speed of one run on 4 cores
>>>>and running alone.
>>>>
>>>>This was observed with:
>>>>- Ubuntu-9.10 (karmic) / kernel linux 2.6.31-14-generic (GNOME 2.28.1)
>>>>- openmpi-1.3.2/1.4.X
>>>>- gromacs 4.0.5/4.0.7
>>>>
>>>>Anyone has any idea what could be going on on those I7?
>>>>
>>>>Thanks,
>>>>We: Djurre (& XAvier)
>>>
>>>Well its usual behavior of systems with HT (Hyper threading) enabled. so
>>>its better to disable HT on such cpus so any cpu eating application like
>>>gromacs will run faster on it.
>>
>>No, this is not what I have heard from Berk Hess, he said that running 8
>>processes on 4 cores gives a speed up of 6-7 times a single core.
>>
>>It sounds more like you need to tune the kernel such that jobs stay on
>>the core they are on. As far as I know Ubuntu is a Desktop OS, maybe you
>>want a Server OS?
>>
>In this case seems cpuaffinity was turned off for mpi apps. check that 
>kernel has cpusets support
>alexxy at xeon /misc/_Work/RecA/pca $ zgrep CPUSET /proc/config.gz
>CONFIG_CPUSETS=y
>CONFIG_PROC_PID_CPUSET=y
>
>And check that your openmpi compiled with support of cpusets

The problem of Gromacs stalling on i7 when using multiple CPUs is a MPI
problem. It is most likely caused by a shared memory bug in Open MPI
that was fixed in the latest release (1.4.1).

Switching to openmpi-1.4.1 solves the problem.

Best regards,
-martti-
--
Post-doctoral research fellow
Moleculaire Dynamica | University of Groningen
Nijenborgh 4, 9747AG Groningen, the Netherlands
tel. +(31) 50 363 4339 | fax. +(31) 50 363 4398