[gmx-users] BGQ compilation with verlet kernels: #include file "kernel_impl.h" not found.

[Mark Abraham]

On Tue, Sep 17, 2013 at 5:31 PM, Christopher Neale
<chris.neale at mail.utoronto.ca> wrote:
> Dear Users:
>
> I am attempting to use the new BGQ kernels with the version of gromacs at https://gerrit.gromacs.org/#/c/2572/ , which I obtained by:
>
> git init
> git fetch https://gerrit.gromacs.org/gromacs refs/changes/72/2572/1 && git checkout FETCH_HEAD
>
> I then attempted to compile like this:
>
> module purge
> module load vacpp/12.1 xlf/14.1 mpich2/xl
> module load cmake/2.8.8
> module load fftw/3.3.2
> export FFTW_LOCATION=/scinet/bgq/Libraries/fftw-3.3.2
> cmake ../source/ \
>       -DCMAKE_PREFIX_PATH=$FFTW_LOCATION \
>       -DCMAKE_INSTALL_PREFIX=$(pwd) \
>       -DGMX_X11=OFF \
>       -DGMX_MPI=OFF \

It's theoretically possible to make use of non-MPI mdrun on BG/Q, but
I've never tried it and wouldn't encourage it. What's your objective?
I'd be closer to prohibiting a thread-MPI or serial build on
BlueGene/Q than to doing work to make it possible.

>       -DGMX_PREFER_STATIC_LIBS=ON
> make -j 4
> make install
>
> #######################
>
> That gave me the following error during make:
>
> Scanning dependencies of target gromacs_include_links
> [  0%] Generating gromacs
> [  0%] Built target gromacs_include_links
> Scanning dependencies of target gmx
> [  0%] [  0%] [  0%] [  0%] [  0%] [  0%] [  0%] [  0%] [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmxcpp.c.o
> [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/md_logging.c.o
> Building C object src/gmxlib/CMakeFiles/gmx.dir/vmddlopen.c.o
> [  1%] [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/chargegroup.c.o
> Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_cpuid.c.o
> [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmxfio_rw.c.o
> Building C object src/gmxlib/CMakeFiles/gmx.dir/gmxfio.c.o
> [  1%] [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/trxio.c.o
> [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_system_xdr.c.o
> Building C object src/gmxlib/CMakeFiles/gmx.dir/wgms.c.o
> Building C object src/gmxlib/CMakeFiles/gmx.dir/wman.c.o
> Building C object src/gmxlib/CMakeFiles/gmx.dir/smalloc.c.o
> Building C object src/gmxlib/CMakeFiles/gmx.dir/rmpbc.c.o
> Building C object src/gmxlib/CMakeFiles/gmx.dir/nrjac.c.o
> Building C object src/gmxlib/CMakeFiles/gmx.dir/index.c.o
> Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_thread_affinity.c.o
> [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/mshift.c.o
> [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/macros.c.o
> [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/oenv.c.o
> [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/maths.c.o
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_omp.c.o
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/random.c.o
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_sort.c.o
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_random.c.o
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/md5.c.o
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/network.c.o
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/copyrite.c.o
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/cinvsqrtdata.c.o
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/writeps.c.o
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/mtop_util.c.o
> "/home/p/pomes/cneale/exec/gromacs-4.6.3_bgq/source/src/gmxlib/network.c", line 264.10: 1506-296 (S) #include file <spi/include/kernel/location.h> not found.

Hmm. That #include is taken from the BG/Q Application Development
manual, to set up the query you can see in
https://gerrit.gromacs.org/#/c/2572/1/src/gmxlib/network.c. I expected
it to Just Work, but it seems that code assumes MPI.

> make[2]: *** [src/gmxlib/CMakeFiles/gmx.dir/network.c.o] Error 1
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [src/gmxlib/CMakeFiles/gmx.dir/all] Error 2
> make: *** [all] Error 2
>
> #######################
>
> After some looking around, I added the following line before cmake:
>
> export CFLAGS=-I/bgsys/drivers/ppcfloor
>
> That solved the initial problem, but now I have the following error:
>
> ...
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/writeps.c.o
> [  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/mtop_util.c.o
> [  3%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_fatal.c.o
> "/bgsys/drivers/ppcfloor/spi/include/kernel/location.h", line 34.10: 1506-296 (S) #include file "kernel_impl.h" not found.
> "/bgsys/drivers/ppcfloor/spi/include/kernel/process.h", line 36.10: 1506-296 (S) #include file "kernel_impl.h" not found.
> "/bgsys/drivers/ppcfloor/spi/include/kernel/process.h", line 194.10: 1506-296 (S) #include file "process_impl.h" not found.
> "/bgsys/drivers/ppcfloor/spi/include/kernel/location.h", line 173.10: 1506-296 (S) #include file "location_impl.h" not found.
> make[2]: *** [src/gmxlib/CMakeFiles/gmx.dir/network.c.o] Error 1
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [src/gmxlib/CMakeFiles/gmx.dir/all] Error 2
> make: *** [all] Error 2
>
> #######################
>
> And it's harder to know what to do now since there are multiple instances of kernel_impl.h , etc.:
>
> bgqdev-fen1-$ find /bgsys/drivers/ppcfloor/spi/include/kernel/|grep kernel_impl.h
> /bgsys/drivers/ppcfloor/spi/include/kernel/cnk/kernel_impl.h
> /bgsys/drivers/ppcfloor/spi/include/kernel/firmware/kernel_impl.h
> /bgsys/drivers/ppcfloor/spi/include/kernel/klinux/kernel_impl.h
>
>
> Does this indicate that the computer that I am using really has such a non-standard BG/Q setup? or is it possible that compiling on the BG/Q with the new verlet kernel requires an entirely different approach?

That patch builds out of the box on JUQUEEN. What might be possible to
do about it depends what you're trying to do with MPI.

> I understand that the Jenkins Buildbot has successfully compiled this code. Is it possible to get a copy of that compilation script (I've never used Jenkins, perhaps it is not that simple)?

No, Jenkins failed (for reasons that shed no light). There's no
automated build for this hardware. All I was doing during development
were things like

cmake .. -DCMAKE_TOOLCHAIN_FILE=Platform/BlueGeneQ-static-XL-C
-DCMAKE_PREFIX_PATH=$HOME/progs/bgsys
-DCMAKE_INSTALL_PREFIX=$HOME/progs/bgsys

per the existing installation guide.

Cheers,

Mark


> Thank you,
> Chris.
>
>
>
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists