[gmx-users] BGQ compilation with verlet kernels: #include file "kernel_impl.h" not found.

[Christopher Neale]

Dear Users:

I am attempting to use the new BGQ kernels with the version of gromacs at https://gerrit.gromacs.org/#/c/2572/ , which I obtained by:

git init
git fetch https://gerrit.gromacs.org/gromacs refs/changes/72/2572/1 && git checkout FETCH_HEAD

I then attempted to compile like this:

module purge
module load vacpp/12.1 xlf/14.1 mpich2/xl
module load cmake/2.8.8
module load fftw/3.3.2
export FFTW_LOCATION=/scinet/bgq/Libraries/fftw-3.3.2
cmake ../source/ \
      -DCMAKE_PREFIX_PATH=$FFTW_LOCATION \
      -DCMAKE_INSTALL_PREFIX=$(pwd) \
      -DGMX_X11=OFF \
      -DGMX_MPI=OFF \
      -DGMX_PREFER_STATIC_LIBS=ON 
make -j 4
make install

#######################

That gave me the following error during make:

Scanning dependencies of target gromacs_include_links
[  0%] Generating gromacs
[  0%] Built target gromacs_include_links
Scanning dependencies of target gmx
[  0%] [  0%] [  0%] [  0%] [  0%] [  0%] [  0%] [  0%] [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmxcpp.c.o
[  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/md_logging.c.o
Building C object src/gmxlib/CMakeFiles/gmx.dir/vmddlopen.c.o
[  1%] [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/chargegroup.c.o
Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_cpuid.c.o
[  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmxfio_rw.c.o
Building C object src/gmxlib/CMakeFiles/gmx.dir/gmxfio.c.o
[  1%] [  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/trxio.c.o
[  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_system_xdr.c.o
Building C object src/gmxlib/CMakeFiles/gmx.dir/wgms.c.o
Building C object src/gmxlib/CMakeFiles/gmx.dir/wman.c.o
Building C object src/gmxlib/CMakeFiles/gmx.dir/smalloc.c.o
Building C object src/gmxlib/CMakeFiles/gmx.dir/rmpbc.c.o
Building C object src/gmxlib/CMakeFiles/gmx.dir/nrjac.c.o
Building C object src/gmxlib/CMakeFiles/gmx.dir/index.c.o
Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_thread_affinity.c.o
[  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/mshift.c.o
[  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/macros.c.o
[  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/oenv.c.o
[  1%] Building C object src/gmxlib/CMakeFiles/gmx.dir/maths.c.o
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_omp.c.o
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/random.c.o
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_sort.c.o
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_random.c.o
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/md5.c.o
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/network.c.o
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/copyrite.c.o
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/cinvsqrtdata.c.o
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/writeps.c.o
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/mtop_util.c.o
"/home/p/pomes/cneale/exec/gromacs-4.6.3_bgq/source/src/gmxlib/network.c", line 264.10: 1506-296 (S) #include file <spi/include/kernel/location.h> not found.
make[2]: *** [src/gmxlib/CMakeFiles/gmx.dir/network.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/gmxlib/CMakeFiles/gmx.dir/all] Error 2
make: *** [all] Error 2

#######################

After some looking around, I added the following line before cmake:

export CFLAGS=-I/bgsys/drivers/ppcfloor

That solved the initial problem, but now I have the following error:

...
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/writeps.c.o
[  2%] Building C object src/gmxlib/CMakeFiles/gmx.dir/mtop_util.c.o
[  3%] Building C object src/gmxlib/CMakeFiles/gmx.dir/gmx_fatal.c.o
"/bgsys/drivers/ppcfloor/spi/include/kernel/location.h", line 34.10: 1506-296 (S) #include file "kernel_impl.h" not found.
"/bgsys/drivers/ppcfloor/spi/include/kernel/process.h", line 36.10: 1506-296 (S) #include file "kernel_impl.h" not found.
"/bgsys/drivers/ppcfloor/spi/include/kernel/process.h", line 194.10: 1506-296 (S) #include file "process_impl.h" not found.
"/bgsys/drivers/ppcfloor/spi/include/kernel/location.h", line 173.10: 1506-296 (S) #include file "location_impl.h" not found.
make[2]: *** [src/gmxlib/CMakeFiles/gmx.dir/network.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/gmxlib/CMakeFiles/gmx.dir/all] Error 2
make: *** [all] Error 2

#######################

And it's harder to know what to do now since there are multiple instances of kernel_impl.h , etc.:

bgqdev-fen1-$ find /bgsys/drivers/ppcfloor/spi/include/kernel/|grep kernel_impl.h
/bgsys/drivers/ppcfloor/spi/include/kernel/cnk/kernel_impl.h
/bgsys/drivers/ppcfloor/spi/include/kernel/firmware/kernel_impl.h
/bgsys/drivers/ppcfloor/spi/include/kernel/klinux/kernel_impl.h


Does this indicate that the computer that I am using really has such a non-standard BG/Q setup? or is it possible that compiling on the BG/Q with the new verlet kernel requires an entirely different approach?

I understand that the Jenkins Buildbot has successfully compiled this code. Is it possible to get a copy of that compilation script (I've never used Jenkins, perhaps it is not that simple)?

Thank you,
Chris.