[gmx-users] compile Gromacs using Cray compilers

Roland Schulz roland at utk.edu
Tue May 21 01:48:01 CEST 2013


Hi,

I agree with Mark that it is probably not worth it. Why are you
interested in it? Are you working on the Cray compiler or planning to
use some specific features?

You need to apply the patch here: https://gerrit.gromacs.org/#/c/2343/
to get it to compile. But of course then it will still be much slower
than GCC (and untested - so make sure to run tests). It would need
support for at least atomics and intrinsics to get comparable speed.

Roland

On Mon, May 20, 2013 at 7:01 PM, Mark Abraham <mark.j.abraham at gmail.com> wrote:
> Hi,
>
> Fixing or working around Cray's compilers (whether atomic support or vector
> intrinsics) is really not worth any of our time if gcc+mpi is available.
>
> Mark
>
>
> On Tue, May 21, 2013 at 12:40 AM, Humayun Arafat <hasagar at gmail.com> wrote:
>
>> Hi Roland,
>>
>> I can actually try to solve some issues with Cray compiler  if you can give
>> me some idea what is needed.
>> Currently I get error for the atomic.h inside thread_mpi. I think it needs
>> intrinsics.
>> Maybe there are some others as well.
>>
>> Please let me know the issues.
>>
>> thanks
>>
>>
>>
>> On Mon, May 20, 2013 at 5:09 PM, Roland Schulz <roland at utk.edu> wrote:
>>
>> > On Mon, May 20, 2013 at 1:23 PM, Humayun Arafat <hasagar at gmail.com>
>> wrote:
>> > > I am only interested in GPU code.
>> > > Is there anyway I can just run this using cray compilers?
>> > Why do you want to use the cray compilers? As Mark said those aren't
>> > supported. We reported the problems with them to Cray but AFAIK they
>> > haven't fixed them.
>> > FYI Cray's also have other compilers usually installed. You usually
>> > use them by something like: "module swap PrgEnv-cray PrgEnv-gnu"
>> >
>> > Roland
>> >
>> > >
>> > > thanks
>> > >
>> > >
>> > > On Mon, May 20, 2013 at 12:06 PM, Szilárd Páll <szilard.pall at cbr.su.se
>> > >wrote:
>> > >
>> > >> The thread-MPI library provides the thread affinity setting
>> > >> functionality to mdrun, hence certain parts of it will always be
>> > >> compiled in, even with GMX_MPI=ON. Apparently, the Cray compiler does
>> > >> not like some of the thread-MPI headers. Feel free to file a bug
>> > >> report on redmine.gromacs.org, but *don't* expect it to get high
>> > >> priority (reasons below).
>> > >>
>> > >> FYI, the Cray compiler does not support SIMD intrinsics and therefore
>> > >> you can't use any of the SIMD accelerated  code. hence, the overall
>> > >> performance will be at least 3x lower than with a compiler that has
>> > >> decent SIMD instrinsics support.
>> > >>
>> > >> --
>> > >> Szilárd
>> > >>
>> > >>
>> > >> On Mon, May 20, 2013 at 6:16 PM, Humayun Arafat <hasagar at gmail.com>
>> > wrote:
>> > >> > Hi,
>> > >> >
>> > >> > I enabled mpich2 and used that to compile gromacs using this command
>> > >> > cmake   -DGMX_MPI=ON ..
>> > >> >
>> > >> > But I am still getting the error.
>> > >> >
>> > >> > [ 10%] Building C object
>> > >> > src/gmxlib/CMakeFiles/gmx.dir/gmx_thread_affinity.c.o
>> > >> > CC-20 craycc: ERROR File =
>> > >> > /home/users/mmm/gromacs/include/thread_mpi/atomic.h, Line = 202
>> > >> >   The identifier "tMPI_Thread_mutex_t" is undefined.
>> > >> >
>> > >> >   static tMPI_Thread_mutex_t tMPI_Atomic_mutex =
>> > >> > TMPI_THREAD_MUTEX_INITIALIZER;
>> > >> >
>> > >> >
>> > >> > It seems like even though it is disabling threadmpi, it is still
>> > making
>> > >> > some tests
>> > >> > I added some part of the cmake output.
>> > >> >
>> > >> > -- MPI is not compatible with thread-MPI. Disabling thread-MPI.
>> > >> > -- Checking for MPI_IN_PLACE
>> > >> > -- Performing Test MPI_IN_PLACE_COMPILE_OK
>> > >> > -- Performing Test MPI_IN_PLACE_COMPILE_OK - Success
>> > >> > -- Checking for MPI_IN_PLACE - yes
>> > >> > -- Checking for CRAY XT Catamount compile
>> > >> > -- Checking for CRAY XT Catamount target - no
>> > >> > CMake Warning at cmake/ThreadMPI.cmake:52 (message):
>> > >> >   Atomic operations not found for this CPU+compiler combination.
>> >  Thread
>> > >> >   support will be unbearably slow: disable threads.  Atomic
>> operations
>> > >> > should
>> > >> >   work on all but the most obscure CPU+compiler combinations; if
>> your
>> > >> system
>> > >> >   is not obscure -- like, for example, x86 with gcc -- please
>> contact
>> > the
>> > >> >   developers.
>> > >> > Call Stack (most recent call first):
>> > >> >   cmake/ThreadMPI.cmake:100 (test_tmpi_atomics)
>> > >> >   CMakeLists.txt:558 (include)
>> > >> >
>> > >> >
>> > >> > thanks a lot for your help.
>> > >> >
>> > >> >
>> > >> >
>> > >> > On Mon, May 20, 2013 at 10:34 AM, Mark Abraham <
>> > mark.j.abraham at gmail.com
>> > >> >wrote:
>> > >> >
>> > >> >> Specify an MPI compiler and use cmake -DGMX_MPI=on by itself. That
>> is
>> > >> >> mutually exclusive with thread mpi.
>> > >> >>
>> > >> >> Mark
>> > >> >>
>> > >> >>
>> > >> >> On Mon, May 20, 2013 at 5:01 PM, Humayun Arafat <hasagar at gmail.com
>> >
>> > >> wrote:
>> > >> >>
>> > >> >> > Hi,
>> > >> >> >
>> > >> >> > I tried different options.
>> > >> >> >
>> > >> >> > cmake -DGMX_MPI=ON/OFF ..
>> > >> >> > cmake -DGMX_THREAD_MPI=OFF/ON ..
>> > >> >> >
>> > >> >> > I tried both of these two flags together. But none of them
>> worked.
>> > >> >> > The compilation fail in either gmx_omp.c or gmx_thread_affinity.c
>> > >> >> >
>> > >> >> > Can you please suggest me another way to turn off the thread mpi?
>> > >> >> >
>> > >> >> > Thanks
>> > >> >> > Humayun
>> > >> >> >
>> > >> >> > On Fri, May 17, 2013 at 6:54 PM, Mark Abraham <
>> > >> mark.j.abraham at gmail.com
>> > >> >> > >wrote:
>> > >> >> >
>> > >> >> > > Cray's compiler is largely/wholly untested. I'd suggest you use
>> > the
>> > >> >> > version
>> > >> >> > > of gcc that you know works.
>> > >> >> > >
>> > >> >> > > For use on a big cluster, you probably don't want Thread MPI
>> > anyway.
>> > >> >> Does
>> > >> >> > > cmake -DGMX_MPI work?
>> > >> >> > >
>> > >> >> > > Mark
>> > >> >> > >
>> > >> >> > >
>> > >> >> > > On Sat, May 18, 2013 at 12:01 AM, Humayun Arafat <
>> > hasagar at gmail.com
>> > >> >
>> > >> >> > > wrote:
>> > >> >> > >
>> > >> >> > > > Hi,
>> > >> >> > > >
>> > >> >> > > >
>> > >> >> > > > I need some help for the compilation of gromacs using Cray
>> > >> >> > > compilers(CCE).
>> > >> >> > > >
>> > >> >> > > > I can compile gromacs using GNU compilers but not using CCE.
>> > >> >> > > >
>> > >> >> > > > I am using gromacs 4.6 and cmake 2.8.4 on Cray XE6
>> > >> >> > > >
>> > >> >> > > >
>> > >> >> > > >
>> > >> >> > > > After doing cmake, when I try to do make, I am getting this
>> > error.
>> > >> >> > > >
>> > >> >> > > >
>> > >> >> > > >
>> > >> >> > > > CC-20 craycc: ERROR File =
>> > >> >> > > > /home/users/me/gromacs/include/thread_mpi/atomic.h, Line =
>> 202
>> > >> >> > > >
>> > >> >> > > > The identifier "tMPI_Thread_mutex_t" is undefined.
>> > >> >> > > >
>> > >> >> > > > static tMPI_Thread_mutex_t tMPI_Atomic_mutex =
>> > >> >> > > > TMPI_THREAD_MUTEX_INITIALIZER;
>> > >> >> > > >
>> > >> >> > > > ^
>> > >> >> > > >
>> > >> >> > > >
>> > >> >> > > >
>> > >> >> > > > Then I checked that the cmake  configuration had errors for
>> > this
>> > >> >> > atomic.h
>> > >> >> > > > header file.
>> > >> >> > > >
>> > >> >> > > > It does not find any atomic operations for cray compilers.
>> > >> >> > > >
>> > >> >> > > > Inside this header file, I see that there is nothing for Cray
>> > >> >> > compilers.
>> > >> >> > > >
>> > >> >> > > > It has GNU,Intel, xlc….
>> > >> >> > > >
>> > >> >> > > > My guess is that probably it is not adding any atomic header
>> > file
>> > >> for
>> > >> >> > > CCE.
>> > >> >> > > >
>> > >> >> > > >
>> > >> >> > > >
>> > >> >> > > >
>> > >> >> > > > Please suggest me some ways to fix this.
>> > >> >> > > >
>> > >> >> > > > Thank you for your help.
>> > >> >> > > >
>> > >> >> > > >
>> > >> >> > > > Thanks
>> > >> >> > > >
>> > >> >> > > > Humayun
>> > >> >> > > > --
>> > >> >> > > > gmx-users mailing list    gmx-users at gromacs.org
>> > >> >> > > > http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > >> >> > > > * Please search the archive at
>> > >> >> > > > http://www.gromacs.org/Support/Mailing_Lists/Search before
>> > >> posting!
>> > >> >> > > > * Please don't post (un)subscribe requests to the list. Use
>> the
>> > >> >> > > > www interface or send it to gmx-users-request at gromacs.org.
>> > >> >> > > > * Can't post? Read
>> > http://www.gromacs.org/Support/Mailing_Lists
>> > >> >> > > >
>> > >> >> > > --
>> > >> >> > > gmx-users mailing list    gmx-users at gromacs.org
>> > >> >> > > http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > >> >> > > * Please search the archive at
>> > >> >> > > http://www.gromacs.org/Support/Mailing_Lists/Search before
>> > posting!
>> > >> >> > > * Please don't post (un)subscribe requests to the list. Use the
>> > >> >> > > www interface or send it to gmx-users-request at gromacs.org.
>> > >> >> > > * Can't post? Read
>> http://www.gromacs.org/Support/Mailing_Lists
>> > >> >> > >
>> > >> >> > --
>> > >> >> > gmx-users mailing list    gmx-users at gromacs.org
>> > >> >> > http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > >> >> > * Please search the archive at
>> > >> >> > http://www.gromacs.org/Support/Mailing_Lists/Search before
>> > posting!
>> > >> >> > * Please don't post (un)subscribe requests to the list. Use the
>> > >> >> > www interface or send it to gmx-users-request at gromacs.org.
>> > >> >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> > >> >> >
>> > >> >> --
>> > >> >> gmx-users mailing list    gmx-users at gromacs.org
>> > >> >> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > >> >> * Please search the archive at
>> > >> >> http://www.gromacs.org/Support/Mailing_Lists/Search before
>> posting!
>> > >> >> * Please don't post (un)subscribe requests to the list. Use the
>> > >> >> www interface or send it to gmx-users-request at gromacs.org.
>> > >> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> > >> >>
>> > >> > --
>> > >> > gmx-users mailing list    gmx-users at gromacs.org
>> > >> > http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > >> > * Please search the archive at
>> > >> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> > >> > * Please don't post (un)subscribe requests to the list. Use the
>> > >> > www interface or send it to gmx-users-request at gromacs.org.
>> > >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> > >> --
>> > >> gmx-users mailing list    gmx-users at gromacs.org
>> > >> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > >> * Please search the archive at
>> > >> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> > >> * Please don't post (un)subscribe requests to the list. Use the
>> > >> www interface or send it to gmx-users-request at gromacs.org.
>> > >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> > >>
>> > > --
>> > > gmx-users mailing list    gmx-users at gromacs.org
>> > > http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > > * Please search the archive at
>> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> > > * Please don't post (un)subscribe requests to the list. Use the
>> > > www interface or send it to gmx-users-request at gromacs.org.
>> > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> > >
>> > >
>> > >
>> > >
>> >
>> >
>> >
>> > --
>> > ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
>> > 865-241-1537, ORNL PO BOX 2008 MS6309
>> > --
>> > gmx-users mailing list    gmx-users at gromacs.org
>> > http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > * Please search the archive at
>> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> > * Please don't post (un)subscribe requests to the list. Use the
>> > www interface or send it to gmx-users-request at gromacs.org.
>> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >
>> --
>> gmx-users mailing list    gmx-users at gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> * Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-request at gromacs.org.
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
>
>
>



-- 
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309



More information about the gromacs.org_gmx-users mailing list