[gmx-developers] thread problem on AMD 12 core chips?

[Sander Pronk]

The only reason the system I could imagine the call could fail would be a permissions problem. 
I've submitted a patch to release-4-5-patches that doesn't check for errors in setting thread affinity.

Sander 

On 13 Jan 2012, at 14:20 , David van der Spoel wrote:

> On 2012-01-13 11:23, Sander Pronk wrote:
>> This error is generated when the thread library can't create threads for some reason (out of memory, or some ulimit; I've never seen it before). It is probably due to the OS.
>> 
>> There might be a chance that this is due to thread affinity API incompatibility: if the number of threads is equal to the number of hardware threads (cores, etc), thread_mpi will enforce thread affinity.
>> 
>> Could you try with:
>> 
>> mdrun -nt 30
>> 
>> (or whichever number other than 32 that is compatible with domain decomposition)
>> 
>> and report whether that works?
> 
> It does indeed work with 30 threads, but not with 32.
> Strange because I thought I tested it with 16, but apparently not, because that works as well.
> 
> Note that the machine have 4 cpus with 8 cores (not 12 as I stated previously).
> 
>> 
>> Sander
>> 
>> 
>> On 12 Jan 2012, at 14:39 , David van der Spoel wrote:
>> 
>>> On 2012-01-12 13:31, Ake Sandgren wrote:
>>>> On Thu, 2012-01-12 at 13:18 +0100, David van der Spoel wrote:
>>>>> On 2012-01-12 11:36, Berk Hess wrote:
>>>>>> On 01/12/2012 11:24 AM, David van der Spoel wrote:
>>>>>>> On 2012-01-12 11:17, Berk Hess wrote:
>>>>>>>> Which compiler is this?
>>>>>>>> We get lots of warnings with gcc4.6, but we run regularly on 32 and 64
>>>>>>>> core nodes.
>>>>>>> 
>>>>>>> Thread model: posix
>>>>>>> gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
>>>>>>> 
>>>>>>> Is intel more reliable?
>>>>>> This is not a matter of platform I would think.
>>>>>> We have only used AMD platform with 32 or more MPI threads.
>>>>>> I would guess this is a thread-mpi bug or a compiler issue.
>>>>> Compiling with Intel C 12.0.3.174 gives the same error, but the same
>>>>> pthread library is linked in.
>>>>> 
>>>>> Other tips for debugging this?
>>>>> 
>>>>>> 
>>>>>> Berk
>>>>>>>> 
>>>>>>>> Berk
>>>>>>>> 
>>>>>>>> On 01/12/2012 11:04 AM, David van der Spoel wrote:
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> I'm trying to compile and install gromacs release-4-5-patches on a new
>>>>>>>>> cluster with four 12-core AMD chips (abisko in Umea, Sweden). However
>>>>>>>>> the threaded code bails out with the following message:
>>>>>>>>> 
>>>>>>>>> Reading file topol.tpr, VERSION 4.5.5-dev-20120111-9181e (double
>>>>>>>>> precision)
>>>>>>>>> Starting 32 threads
>>>>>>>>> tMPI error: tMPI Initialization error (in valid comm)
>>>>>>>>> 
>>>>>>>>> First, I'm a bit confused why the code detects only 32 cores, second
>>>>>>>>> it shows above error and quits.
>>>>>>>>> 
>>>>>>>>> Any clues?
>>>> 
>>>> 
>>>> Abisko's current nodes are 4-socket 8-core (the 12-cores are still under
>>>> test)
>>>> 
>>>> If you are using openmpi it does not have support for MPI threads
>>>> compiled in (the openib part of openmpi doesn't support this yet) that
>>>> probably explains your problem.
>>>> 
>>> I see, that explains the 32. But gromacs uses it's own mpi-over-threads implementation that does not use any MPI whatsoever.
>>> 
>>> 
>>> --
>>> David van der Spoel, Ph.D., Professor of Biology
>>> Dept. of Cell&  Molec. Biol., Uppsala University.
>>> Box 596, 75124 Uppsala, Sweden. Phone:	+46184714205.
>>> spoel at xray.bmc.uu.se    http://folding.bmc.uu.se
>>> --
>>> gmx-developers mailing list
>>> gmx-developers at gromacs.org
>>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>>> Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-developers-request at gromacs.org.
>> 
> 
> 
> -- 
> David van der Spoel, Ph.D., Professor of Biology
> Dept. of Cell & Molec. Biol., Uppsala University.
> Box 596, 75124 Uppsala, Sweden. Phone:	+46184714205.
> spoel at xray.bmc.uu.se    http://folding.bmc.uu.se
> -- 
> gmx-developers mailing list
> gmx-developers at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-developers-request at gromacs.org.