[gmx-developers] warning Emulating FMA instructions - this is probably not what you want!
Roland Schulz
roland at utk.edu
Thu Oct 18 15:51:45 CEST 2012
On Thu, Oct 18, 2012 at 9:19 AM, Szilárd Páll <szilard.pall at cbr.su.se>wrote:
> On Thu, Oct 18, 2012 at 3:12 PM, Roland Schulz <roland at utk.edu> wrote:
>
>>
>>
>> On Thu, Oct 18, 2012 at 7:01 AM, Szilárd Páll <szilard.pall at cbr.su.se>wrote:
>>
>>> That's correct. The point I was trying to make is that we might end up
>>> hearing about this issue in the future if people use CFLAGS just by
>>> routine. I fact, I'm not even sure that cmake is not supposed to pick up
>>> CLAGS as well and it's some later cmake pass rewriting compiler options is
>>> that the current code is trying to overcome, but it ends up loosing some
>>> options when CFLAGS is set.
>>>
>>> Any comments? Should CFLAGS work?
>>>
>> Yes. And I think it mostly works like it should. Environment variables
>> should only be read the first time you run. And it should overwrite any of
>> the default settings. Thus I think we should not set any flags if CFLAGS is
>> set. Sander added that with 0ce2e62b72d2 to cmake/gmxCFlags.cmake. None of
>> the flags set in that file are added to CMAKE_C_FLAGS if CFLAGS is set..
>> But the new acceleration depending options (e.g. -mavx) are set even if
>> CFLAGS is set. I think this is wrong. If you agree I can fix that.
>>
>
> Wait a second, if you want to override all flags instead of appending,
> we might have to change the acceleration detection to drop all acceleration
> unless the flag that would be added is already in the CFLAGS passed. This
> sounds a bit awkward, but it's the only way to ensure that one does not end
> up e.g. with GMX_ACCELERATION=AVX_256, but no -mavx flag passed which will
> result in nasty compiler errors.
>
I think it is important to be consistent. I'm not sure there is a cmake
standard whether CFLAGS should overwrite or append flags. With autoconf we
used to overwrite and with gmxCFlags we overwrite. Given that, I don't
think it is a good idea to try to be too helpful. I think if the user
manually specifies something we shouldn't mess with it. Nothing is
more annoying than a program which does something else then explicitly
told. So if I call cmake with explicit set CFLAGS those exactly should be
used. If anything at all we should give a warning that those flags are
probably needed. If we want to have a different variable which means "add
those cflags to the default ones" that's fine. But I think based on the
history the meaning of CFLAGS is "use exactly those flags" and thus even
the acceleration flags shouldn't be added to it.
Roland
>
> --
> Szilárd
>
>>
>> Roland
>>
>>
>>>
>>> --
>>> Szilárd
>>>
>>>
>>>
>>> On Thu, Oct 18, 2012 at 9:43 AM, Jochen Hub <jhub at gwdg.de> wrote:
>>>
>>>> Following Szilard, I tried to use
>>>>
>>>> cmake -DCMAKE_C_FLAGS=-mfma4 ...
>>>>
>>>> instead of export CFLAGS=-mfma4 to add the -mfma4 flag. Now, the only
>>>> difference in CMakeCache.txt is
>>>>
>>>> < CMAKE_C_FLAGS:STRING='-mavx -Wall -Wno-unused -Wunused-value '
>>>> ---
>>>> > CMAKE_C_FLAGS:STRING=-mavx -Wall -Wno-unused -Wunused-value -mfma4
>>>>
>>>> which is just what I wanted, right? (and the warning on emulating FMA
>>>> instructions does not appear any more).
>>>>
>>>> Jochen
>>>>
>>>>
>>>> Am 10/17/12 11:28 PM, schrieb Szilárd Páll:
>>>>
>>>>>
>>>>> That way of calling CMake triggers an interesting bug in the option
>>>>> generator and CFLAGS will essentially override part of the
>>>>> auto-generated compiler options. Diff of relevant lines in
>>>>> CMakeCache.txt are below, the only difference is that the second file
>>>>> ("+" in the diff) was generated by running "CFLAGS=-march=native
>>>>> cmake...":
>>>>>
>>>>> //Flags used by the compiler during all build types
>>>>> -CMAKE_C_FLAGS:STRING='-msse2 -Wall -Wno-unused -Wunused-value '
>>>>> +CMAKE_C_FLAGS:STRING='-msse2 -march=native '
>>>>> //Flags used by the compiler during debug builds.
>>>>> -CMAKE_C_FLAGS_DEBUG:STRING=-**fno-inline -g
>>>>> +CMAKE_C_FLAGS_DEBUG:STRING=-g
>>>>> -//Flags used by the compiler during release builds.
>>>>> -CMAKE_C_FLAGS_RELEASE:STRING=**-fomit-frame-pointer
>>>>> -funroll-all-loops
>>>>> -fexcess-precision=fast -O3 -DNDEBUG
>>>>> +//Flags used by the compiler during release builds (/MD /Ob1 /Oi
>>>>> +// /Ot /Oy /Gs will produce slightly less optimized but smaller
>>>>> +// files).
>>>>> +CMAKE_C_FLAGS_RELEASE:STRING=**-O3 -DNDEBUG
>>>>>
>>>>> As this messes with "considered to be best basic set of options", we
>>>>> could state that CFLAGS should not be used, but it might be a better
>>>>> alternative to make the configuration prepend the CFLAG to the
>>>>> CMAKE_C_FLAGS and keep our generated set of flags as well.
>>>>>
>>>>> I'm not sure which options is best. Thoughts?
>>>>>
>>>>> --
>>>>> Szilárd
>>>>>
>>>>>
>>>>> On Wed, Oct 17, 2012 at 10:12 PM, Jochen Hub <jhub at gwdg.de
>>>>> <mailto:jhub at gwdg.de>> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> many thanks for the help. I now compiled with gcc 4.7 and with
>>>>> manually adding export CFLAGS=-mfma4 (that is what you meant,
>>>>> right?). Now the warning does not appear any more.
>>>>>
>>>>> I am not sure though how Gromacs performed on this hardware using
>>>>> these settings, as the AVX kernels are apparently not in
>>>>> release-4.6.
>>>>>
>>>>> Thanks,
>>>>> Jochen
>>>>>
>>>>>
>>>>> Am 10/17/12 5:12 PM, schrieb Szilárd Páll:
>>>>>
>>>>> On Wed, Oct 17, 2012 at 4:46 PM, Berk Hess <hess at kth.se
>>>>> <mailto:hess at kth.se>
>>>>> <mailto:hess at kth.se <mailto:hess at kth.se>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Apparently icc can compile the code, so than it's not
>>>>> strange that
>>>>> you end up in this situation. But we should avoid this
>>>>> from
>>>>> happening.
>>>>> SSE4.1 would be the next supported level, but you really
>>>>> want the
>>>>> vex instructions and not plain SSE, as the latter is much
>>>>> slower.
>>>>>
>>>>>
>>>>> AMD explicitly states that only up to -msse3 should be used on
>>>>> Bulldozer
>>>>> and I have never tried to generate SSE4.1 instructions with icc
>>>>> and run
>>>>> it on AMD. I would not be surprised if it didn't work.
>>>>>
>>>>> --
>>>>> Szilárd
>>>>>
>>>>> The best thing would be to use gcc, preferably 4.7.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Berk
>>>>>
>>>>> ----- Reply message -----
>>>>> From: "Szilárd Páll" <szilard.pall at cbr.su.se
>>>>> <mailto:szilard.pall at cbr.su.se**>
>>>>> <mailto:szilard.pall at cbr.su.se
>>>>> <mailto:szilard.pall at cbr.su.se**>__>>
>>>>> To: "Discussion list for GROMACS development"
>>>>> <gmx-developers at gromacs.org
>>>>> <mailto:gmx-developers@**gromacs.org<gmx-developers at gromacs.org>
>>>>> >
>>>>> <mailto:gmx-developers at __groma**cs.org <http://gromacs.org>
>>>>> <mailto:gmx-developers@**gromacs.org<gmx-developers at gromacs.org>
>>>>> >>>
>>>>> Subject: [gmx-developers] warning Emulating FMA
>>>>> instructions - this
>>>>> is probably not what you want!
>>>>> Date: Wed, Oct 17, 2012 15:25
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> First of all, as far as I know, the new icc 13 can not
>>>>> generate
>>>>> AMD-compatible FMA4/XOP (v12 surely cant:
>>>>> http://developer.amd.com/__**Assets/CompilerOptQuickRef-__**
>>>>> 62004200.pdf<http://developer.amd.com/__Assets/CompilerOptQuickRef-__62004200.pdf>
>>>>> <http://developer.amd.com/**Assets/CompilerOptQuickRef-**
>>>>> 62004200.pdf<http://developer.amd.com/Assets/CompilerOptQuickRef-62004200.pdf>>),
>>>>>
>>>>>
>>>>> so I find it strange that you've ended up with
>>>>> GMX_ACCELERATION=AVX_128_FMA using an Intel compiler --
>>>>> unless you
>>>>> set the acceleration manually. To get FMA4 support you
>>>>> need
>>>>> to use a
>>>>> recent gcc version, the newer the better. The Verlet
>>>>> kernels don't
>>>>> benefit much from FMA4, so if you want to, you can use
>>>>> Intel
>>>>> Compiler, but then you need to set the acceleration to
>>>>> SSE2
>>>>> (max
>>>>> SSE3 works, but we don't use these instructions).
>>>>>
>>>>> On Wed, Oct 17, 2012 at 11:14 AM, Jochen Hub <
>>>>> jhub at gwdg.de
>>>>> <mailto:jhub at gwdg.de>
>>>>> <mailto:jhub at gwdg.de <mailto:jhub at gwdg.de>>> wrote:
>>>>>
>>>>> Hi developers,
>>>>>
>>>>> does anyone know how to interpret this icc warning:
>>>>>
>>>>> [ 8%]
>>>>>
>>>>> /home/jhub/src/gromacs/____**include/gmx_x86_avx_128_fma.h(**____88):
>>>>>
>>>>>
>>>>>
>>>>> warning #1224: #warning directive: Emulating FMA
>>>>> instructions -
>>>>> this is probably not what you want!
>>>>>
>>>>> #warning Emulating FMA instructions - this is
>>>>> probably not
>>>>> what you want!
>>>>>
>>>>>
>>>>> This warning is related to a bug in the build system,
>>>>> you'll need to
>>>>> add the -mfma4 flag to the compiler flags manually.
>>>>>
>>>>> --
>>>>> Szilard
>>>>>
>>>>>
>>>>> I am compiling 46-release on a Interlagos 6378 with
>>>>> OpenMPI and
>>>>> icc 13.0.0 20120731. My cmake line is:
>>>>>
>>>>> cmake $gmxsrc \
>>>>> -DFFTW_LIBRARY=$FFTW_LOCATION/**____lib/libfftw3f.a
>>>>> \
>>>>> -DFFTW3F_INCLUDE_DIR=$FFTW____**_LOCATION/include
>>>>> \
>>>>>
>>>>> -DFFTW3F_LIBRARIES=$FFTW_____**LOCATION/lib/libfftw3f.a \
>>>>>
>>>>>
>>>>> -DCMAKE_INSTALL_PREFIX=$(pwd) \
>>>>> -DGMX_X11=OFF \
>>>>> -DCMAKE_CXX_COMPILER=$MPICXX \
>>>>> -DCMAKE_C_COMPILER=$MPICC \
>>>>> -DGMX_MPI=ON \
>>>>> -DGMX_PREFER_STATIC_LIBS=ON \
>>>>> -DGMX_GPU=OFF
>>>>>
>>>>> and cmake reported:
>>>>>
>>>>> -- Performing Test GNU_AVX_CFLAG
>>>>> -- Performing Test GNU_AVX_CFLAG - Success
>>>>> -- Enabling 128-bit AVX Gromacs acceleration (with
>>>>> fused-multiply add), and it will help compiler
>>>>> optimization.
>>>>>
>>>>> Thanks a lot,
>>>>> Jochen
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> ------------------------------**____---------------------
>>>>>
>>>>>
>>>>>
>>>>> Dr. Jochen Hub
>>>>> Computational Molecular Biophysics Group
>>>>> Institute for Microbiology and Genetics
>>>>> Georg-August-University of Göttingen
>>>>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
>>>>> Phone: +49-551-39-14189 <tel:%2B49-551-39-14189>
>>>>> <tel:%2B49-551-39-14189>
>>>>> http://cmb.bio.uni-goettingen.**____de/
>>>>> <http://cmb.bio.uni-__**goettingen.de/<http://cmb.bio.uni-__goettingen.de/>
>>>>> <http://cmb.bio.uni-**goettingen.de/<http://cmb.bio.uni-goettingen.de/>
>>>>> >>
>>>>> ------------------------------**____---------------------
>>>>>
>>>>>
>>>>> --
>>>>> gmx-developers mailing list
>>>>> gmx-developers at gromacs.org <mailto:gmx-developers@**
>>>>> gromacs.org <gmx-developers at gromacs.org>>
>>>>> <mailto:gmx-developers at __groma**cs.org <http://gromacs.org>
>>>>> <mailto:gmx-developers@**gromacs.org<gmx-developers at gromacs.org>
>>>>> >>
>>>>> http://lists.gromacs.org/____**mailman/listinfo/gmx-____**
>>>>> developers<http://lists.gromacs.org/____mailman/listinfo/gmx-____developers>
>>>>> <http://lists.gromacs.org/__**mailman/listinfo/gmx-__**
>>>>> developers<http://lists.gromacs.org/__mailman/listinfo/gmx-__developers>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> <http://lists.gromacs.org/__**mailman/listinfo/gmx-__**
>>>>> developers<http://lists.gromacs.org/__mailman/listinfo/gmx-__developers>
>>>>> <http://lists.gromacs.org/**mailman/listinfo/gmx-**developers<http://lists.gromacs.org/mailman/listinfo/gmx-developers>
>>>>> >>
>>>>> Please don't post (un)subscribe requests to the list.
>>>>> Use the
>>>>> www interface or send it to
>>>>> gmx-developers-request at __groma**__cs.org<http://groma__cs.org><
>>>>> http://gromacs.org>
>>>>> <mailto:gmx-developers-__**request at gromacs.org<gmx-developers-__request at gromacs.org>
>>>>>
>>>>> <mailto:gmx-developers-**request at gromacs.org<gmx-developers-request at gromacs.org>
>>>>> >>.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> gmx-developers mailing list
>>>>> gmx-developers at gromacs.org <mailto:gmx-developers@**
>>>>> gromacs.org <gmx-developers at gromacs.org>>
>>>>> <mailto:gmx-developers at __groma**cs.org <http://gromacs.org>
>>>>>
>>>>> <mailto:gmx-developers@**gromacs.org<gmx-developers at gromacs.org>
>>>>> >>
>>>>>
>>>>> http://lists.gromacs.org/__**mailman/listinfo/gmx-__**
>>>>> developers<http://lists.gromacs.org/__mailman/listinfo/gmx-__developers>
>>>>> <http://lists.gromacs.org/**mailman/listinfo/gmx-**developers<http://lists.gromacs.org/mailman/listinfo/gmx-developers>
>>>>> >
>>>>> Please don't post (un)subscribe requests to the list. Use
>>>>> the
>>>>> www interface or send it to
>>>>> gmx-developers-request at __groma**cs.org <http://gromacs.org>
>>>>> <mailto:gmx-developers-**request at gromacs.org<gmx-developers-request at gromacs.org>
>>>>> >
>>>>> <mailto:gmx-developers-__**request at gromacs.org<gmx-developers-__request at gromacs.org>
>>>>> <mailto:gmx-developers-**request at gromacs.org<gmx-developers-request at gromacs.org>>>.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> ------------------------------**__---------------------
>>>>> Dr. Jochen Hub
>>>>> Computational Molecular Biophysics Group
>>>>> Institute for Microbiology and Genetics
>>>>> Georg-August-University of Göttingen
>>>>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
>>>>> Phone: +49-551-39-14189 <tel:%2B49-551-39-14189>
>>>>> http://cmb.bio.uni-goettingen.**__de/ <http://cmb.bio.uni-**
>>>>> goettingen.de/ <http://cmb.bio.uni-goettingen.de/>>
>>>>> ------------------------------**__---------------------
>>>>>
>>>>>
>>>>>
>>>> --
>>>> ------------------------------**---------------------
>>>> Dr. Jochen Hub
>>>> Computational Molecular Biophysics Group
>>>> Institute for Microbiology and Genetics
>>>> Georg-August-University of Göttingen
>>>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
>>>> Phone: +49-551-39-14189
>>>> http://cmb.bio.uni-goettingen.**de/ <http://cmb.bio.uni-goettingen.de/>
>>>> ------------------------------**---------------------
>>>> --
>>>> gmx-developers mailing list
>>>> gmx-developers at gromacs.org
>>>> http://lists.gromacs.org/**mailman/listinfo/gmx-**developers<http://lists.gromacs.org/mailman/listinfo/gmx-developers>
>>>> Please don't post (un)subscribe requests to the list. Use the www
>>>> interface or send it to gmx-developers-request@**gromacs.org<gmx-developers-request at gromacs.org>
>>>> .
>>>>
>>>
>>>
>>
>>
>> --
>> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
>> 865-241-1537, ORNL PO BOX 2008 MS6309
>>
>> --
>> gmx-developers mailing list
>> gmx-developers at gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>> Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-developers-request at gromacs.org.
>>
>
>
--
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20121018/e365b05a/attachment.html>
More information about the gromacs.org_gmx-developers
mailing list