[gmx-developers] warning Emulating FMA instructions - this is probably not what you want!
Szilárd Páll
szilard.pall at cbr.su.se
Thu Oct 18 17:32:25 CEST 2012
On Thu, Oct 18, 2012 at 3:51 PM, Roland Schulz <roland at utk.edu> wrote:
>
>
> On Thu, Oct 18, 2012 at 9:19 AM, Szilárd Páll <szilard.pall at cbr.su.se>wrote:
>
>> On Thu, Oct 18, 2012 at 3:12 PM, Roland Schulz <roland at utk.edu> wrote:
>>
>>>
>>>
>>> On Thu, Oct 18, 2012 at 7:01 AM, Szilárd Páll <szilard.pall at cbr.su.se>wrote:
>>>
>>>> That's correct. The point I was trying to make is that we might end up
>>>> hearing about this issue in the future if people use CFLAGS just by
>>>> routine. I fact, I'm not even sure that cmake is not supposed to pick up
>>>> CLAGS as well and it's some later cmake pass rewriting compiler options is
>>>> that the current code is trying to overcome, but it ends up loosing some
>>>> options when CFLAGS is set.
>>>>
>>>> Any comments? Should CFLAGS work?
>>>>
>>> Yes. And I think it mostly works like it should. Environment variables
>>> should only be read the first time you run. And it should overwrite any of
>>> the default settings. Thus I think we should not set any flags if CFLAGS is
>>> set. Sander added that with 0ce2e62b72d2 to cmake/gmxCFlags.cmake. None of
>>> the flags set in that file are added to CMAKE_C_FLAGS if CFLAGS is set..
>>> But the new acceleration depending options (e.g. -mavx) are set even if
>>> CFLAGS is set. I think this is wrong. If you agree I can fix that.
>>>
>>
>> Wait a second, if you want to override all flags instead of appending,
>> we might have to change the acceleration detection to drop all acceleration
>> unless the flag that would be added is already in the CFLAGS passed. This
>> sounds a bit awkward, but it's the only way to ensure that one does not end
>> up e.g. with GMX_ACCELERATION=AVX_256, but no -mavx flag passed which will
>> result in nasty compiler errors.
>>
>
> I think it is important to be consistent. I'm not sure there is a cmake
> standard whether CFLAGS should overwrite or append flags. With autoconf we
> used to overwrite and with gmxCFlags we overwrite. Given that, I don't
> think it is a good idea to try to be too helpful. I think if the user
> manually specifies something we shouldn't mess with it. Nothing is
> more annoying than a program which does something else then explicitly
> told. So if I call cmake with explicit set CFLAGS those exactly should be
> used. If anything at all we should give a warning that those flags are
> probably needed. If we want to have a different variable which means "add
> those cflags to the default ones" that's fine. But I think based on the
> history the meaning of CFLAGS is "use exactly those flags" and thus even
> the acceleration flags shouldn't be added to it.
>
Well, consistency in this case can be defined in many ways, but in
principle I agree. However, by "consistent" you are referring to adherence
to common *nix practices. At the same time, I was referring to prioritizing
internal consistency, i.e. trying hard to not end up with broken
configuration.Consequently, if we really want to stick hard "autoconf
emulation" (assuming that the CMake behavior expected with CFLAGS is not
documented to be the same as with autoconf), we need to *drop* all
acceleration and probably disable other things (like the gcc 4.4 -O3
bug workaround).
E.g. given a cmake invocation:
$ CC=gcc CFLAGS="-foo -bar" cmake
on a Sandy Bridge CPU, we should drop to GMX_ACCELERATION=SSE (because
that's on by default with gcc).
Of course we can and should issue a warning, but especially if we won't
have the configuration summary implemented for 4.6 (which based on the
amount of volunteers is probable), a lot of warnings can go unnoticed in
the 100-s of lines long CMake configuration output.
Cheers,
--
Szilárd
>
> Roland
>
>
>>
>> --
>> Szilárd
>>
>>>
>>> Roland
>>>
>>>
>>>>
>>>> --
>>>> Szilárd
>>>>
>>>>
>>>>
>>>> On Thu, Oct 18, 2012 at 9:43 AM, Jochen Hub <jhub at gwdg.de> wrote:
>>>>
>>>>> Following Szilard, I tried to use
>>>>>
>>>>> cmake -DCMAKE_C_FLAGS=-mfma4 ...
>>>>>
>>>>> instead of export CFLAGS=-mfma4 to add the -mfma4 flag. Now, the only
>>>>> difference in CMakeCache.txt is
>>>>>
>>>>> < CMAKE_C_FLAGS:STRING='-mavx -Wall -Wno-unused -Wunused-value '
>>>>> ---
>>>>> > CMAKE_C_FLAGS:STRING=-mavx -Wall -Wno-unused -Wunused-value -mfma4
>>>>>
>>>>> which is just what I wanted, right? (and the warning on emulating FMA
>>>>> instructions does not appear any more).
>>>>>
>>>>> Jochen
>>>>>
>>>>>
>>>>> Am 10/17/12 11:28 PM, schrieb Szilárd Páll:
>>>>>
>>>>>>
>>>>>> That way of calling CMake triggers an interesting bug in the option
>>>>>> generator and CFLAGS will essentially override part of the
>>>>>> auto-generated compiler options. Diff of relevant lines in
>>>>>> CMakeCache.txt are below, the only difference is that the second file
>>>>>> ("+" in the diff) was generated by running "CFLAGS=-march=native
>>>>>> cmake...":
>>>>>>
>>>>>> //Flags used by the compiler during all build types
>>>>>> -CMAKE_C_FLAGS:STRING='-msse2 -Wall -Wno-unused -Wunused-value '
>>>>>> +CMAKE_C_FLAGS:STRING='-msse2 -march=native '
>>>>>> //Flags used by the compiler during debug builds.
>>>>>> -CMAKE_C_FLAGS_DEBUG:STRING=-**fno-inline -g
>>>>>> +CMAKE_C_FLAGS_DEBUG:STRING=-g
>>>>>> -//Flags used by the compiler during release builds.
>>>>>> -CMAKE_C_FLAGS_RELEASE:STRING=**-fomit-frame-pointer
>>>>>> -funroll-all-loops
>>>>>> -fexcess-precision=fast -O3 -DNDEBUG
>>>>>> +//Flags used by the compiler during release builds (/MD /Ob1 /Oi
>>>>>> +// /Ot /Oy /Gs will produce slightly less optimized but smaller
>>>>>> +// files).
>>>>>> +CMAKE_C_FLAGS_RELEASE:STRING=**-O3 -DNDEBUG
>>>>>>
>>>>>> As this messes with "considered to be best basic set of options", we
>>>>>> could state that CFLAGS should not be used, but it might be a better
>>>>>> alternative to make the configuration prepend the CFLAG to the
>>>>>> CMAKE_C_FLAGS and keep our generated set of flags as well.
>>>>>>
>>>>>> I'm not sure which options is best. Thoughts?
>>>>>>
>>>>>> --
>>>>>> Szilárd
>>>>>>
>>>>>>
>>>>>> On Wed, Oct 17, 2012 at 10:12 PM, Jochen Hub <jhub at gwdg.de
>>>>>> <mailto:jhub at gwdg.de>> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> many thanks for the help. I now compiled with gcc 4.7 and with
>>>>>> manually adding export CFLAGS=-mfma4 (that is what you meant,
>>>>>> right?). Now the warning does not appear any more.
>>>>>>
>>>>>> I am not sure though how Gromacs performed on this hardware using
>>>>>> these settings, as the AVX kernels are apparently not in
>>>>>> release-4.6.
>>>>>>
>>>>>> Thanks,
>>>>>> Jochen
>>>>>>
>>>>>>
>>>>>> Am 10/17/12 5:12 PM, schrieb Szilárd Páll:
>>>>>>
>>>>>> On Wed, Oct 17, 2012 at 4:46 PM, Berk Hess <hess at kth.se
>>>>>> <mailto:hess at kth.se>
>>>>>> <mailto:hess at kth.se <mailto:hess at kth.se>>> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Apparently icc can compile the code, so than it's not
>>>>>> strange that
>>>>>> you end up in this situation. But we should avoid this
>>>>>> from
>>>>>> happening.
>>>>>> SSE4.1 would be the next supported level, but you really
>>>>>> want the
>>>>>> vex instructions and not plain SSE, as the latter is much
>>>>>> slower.
>>>>>>
>>>>>>
>>>>>> AMD explicitly states that only up to -msse3 should be used on
>>>>>> Bulldozer
>>>>>> and I have never tried to generate SSE4.1 instructions with
>>>>>> icc
>>>>>> and run
>>>>>> it on AMD. I would not be surprised if it didn't work.
>>>>>>
>>>>>> --
>>>>>> Szilárd
>>>>>>
>>>>>> The best thing would be to use gcc, preferably 4.7.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Berk
>>>>>>
>>>>>> ----- Reply message -----
>>>>>> From: "Szilárd Páll" <szilard.pall at cbr.su.se
>>>>>> <mailto:szilard.pall at cbr.su.se**>
>>>>>> <mailto:szilard.pall at cbr.su.se
>>>>>> <mailto:szilard.pall at cbr.su.se**>__>>
>>>>>> To: "Discussion list for GROMACS development"
>>>>>> <gmx-developers at gromacs.org
>>>>>> <mailto:gmx-developers@**gromacs.org<gmx-developers at gromacs.org>
>>>>>> >
>>>>>> <mailto:gmx-developers at __groma**cs.org <http://gromacs.org>
>>>>>> <mailto:gmx-developers@**gromacs.org<gmx-developers at gromacs.org>
>>>>>> >>>
>>>>>> Subject: [gmx-developers] warning Emulating FMA
>>>>>> instructions - this
>>>>>> is probably not what you want!
>>>>>> Date: Wed, Oct 17, 2012 15:25
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> First of all, as far as I know, the new icc 13 can not
>>>>>> generate
>>>>>> AMD-compatible FMA4/XOP (v12 surely cant:
>>>>>> http://developer.amd.com/__**Assets/CompilerOptQuickRef-__**
>>>>>> 62004200.pdf<http://developer.amd.com/__Assets/CompilerOptQuickRef-__62004200.pdf>
>>>>>> <http://developer.amd.com/**Assets/CompilerOptQuickRef-**
>>>>>> 62004200.pdf<http://developer.amd.com/Assets/CompilerOptQuickRef-62004200.pdf>>),
>>>>>>
>>>>>>
>>>>>> so I find it strange that you've ended up with
>>>>>> GMX_ACCELERATION=AVX_128_FMA using an Intel compiler --
>>>>>> unless you
>>>>>> set the acceleration manually. To get FMA4 support you
>>>>>> need
>>>>>> to use a
>>>>>> recent gcc version, the newer the better. The Verlet
>>>>>> kernels don't
>>>>>> benefit much from FMA4, so if you want to, you can use
>>>>>> Intel
>>>>>> Compiler, but then you need to set the acceleration to
>>>>>> SSE2
>>>>>> (max
>>>>>> SSE3 works, but we don't use these instructions).
>>>>>>
>>>>>> On Wed, Oct 17, 2012 at 11:14 AM, Jochen Hub <
>>>>>> jhub at gwdg.de
>>>>>> <mailto:jhub at gwdg.de>
>>>>>> <mailto:jhub at gwdg.de <mailto:jhub at gwdg.de>>> wrote:
>>>>>>
>>>>>> Hi developers,
>>>>>>
>>>>>> does anyone know how to interpret this icc warning:
>>>>>>
>>>>>> [ 8%]
>>>>>>
>>>>>> /home/jhub/src/gromacs/____**include/gmx_x86_avx_128_fma.h(*
>>>>>> *____88):
>>>>>>
>>>>>>
>>>>>> warning #1224: #warning directive: Emulating FMA
>>>>>> instructions -
>>>>>> this is probably not what you want!
>>>>>>
>>>>>> #warning Emulating FMA instructions - this is
>>>>>> probably not
>>>>>> what you want!
>>>>>>
>>>>>>
>>>>>> This warning is related to a bug in the build system,
>>>>>> you'll need to
>>>>>> add the -mfma4 flag to the compiler flags manually.
>>>>>>
>>>>>> --
>>>>>> Szilard
>>>>>>
>>>>>>
>>>>>> I am compiling 46-release on a Interlagos 6378 with
>>>>>> OpenMPI and
>>>>>> icc 13.0.0 20120731. My cmake line is:
>>>>>>
>>>>>> cmake $gmxsrc \
>>>>>> -DFFTW_LIBRARY=$FFTW_LOCATION/**____lib/libfftw3f.a
>>>>>> \
>>>>>> -DFFTW3F_INCLUDE_DIR=$FFTW____**_LOCATION/include
>>>>>> \
>>>>>>
>>>>>> -DFFTW3F_LIBRARIES=$FFTW_____**LOCATION/lib/libfftw3f.a \
>>>>>>
>>>>>>
>>>>>> -DCMAKE_INSTALL_PREFIX=$(pwd) \
>>>>>> -DGMX_X11=OFF \
>>>>>> -DCMAKE_CXX_COMPILER=$MPICXX \
>>>>>> -DCMAKE_C_COMPILER=$MPICC \
>>>>>> -DGMX_MPI=ON \
>>>>>> -DGMX_PREFER_STATIC_LIBS=ON \
>>>>>> -DGMX_GPU=OFF
>>>>>>
>>>>>> and cmake reported:
>>>>>>
>>>>>> -- Performing Test GNU_AVX_CFLAG
>>>>>> -- Performing Test GNU_AVX_CFLAG - Success
>>>>>> -- Enabling 128-bit AVX Gromacs acceleration (with
>>>>>> fused-multiply add), and it will help compiler
>>>>>> optimization.
>>>>>>
>>>>>> Thanks a lot,
>>>>>> Jochen
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> ------------------------------**____---------------------
>>>>>>
>>>>>>
>>>>>>
>>>>>> Dr. Jochen Hub
>>>>>> Computational Molecular Biophysics Group
>>>>>> Institute for Microbiology and Genetics
>>>>>> Georg-August-University of Göttingen
>>>>>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
>>>>>> Phone: +49-551-39-14189 <tel:%2B49-551-39-14189>
>>>>>> <tel:%2B49-551-39-14189>
>>>>>> http://cmb.bio.uni-goettingen.**____de/
>>>>>> <http://cmb.bio.uni-__**goettingen.de/<http://cmb.bio.uni-__goettingen.de/>
>>>>>> <http://cmb.bio.uni-**goettingen.de/<http://cmb.bio.uni-goettingen.de/>
>>>>>> >>
>>>>>> ------------------------------**____---------------------
>>>>>>
>>>>>>
>>>>>> --
>>>>>> gmx-developers mailing list
>>>>>> gmx-developers at gromacs.org <mailto:gmx-developers@**
>>>>>> gromacs.org <gmx-developers at gromacs.org>>
>>>>>> <mailto:gmx-developers at __groma**cs.org <http://gromacs.org>
>>>>>> <mailto:gmx-developers@**gromacs.org<gmx-developers at gromacs.org>
>>>>>> >>
>>>>>> http://lists.gromacs.org/____**mailman/listinfo/gmx-____**
>>>>>> developers<http://lists.gromacs.org/____mailman/listinfo/gmx-____developers>
>>>>>> <http://lists.gromacs.org/__**mailman/listinfo/gmx-__**
>>>>>> developers<http://lists.gromacs.org/__mailman/listinfo/gmx-__developers>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> <http://lists.gromacs.org/__**mailman/listinfo/gmx-__**
>>>>>> developers<http://lists.gromacs.org/__mailman/listinfo/gmx-__developers>
>>>>>> <http://lists.gromacs.org/**mailman/listinfo/gmx-**developers<http://lists.gromacs.org/mailman/listinfo/gmx-developers>
>>>>>> >>
>>>>>> Please don't post (un)subscribe requests to the list.
>>>>>> Use the
>>>>>> www interface or send it to
>>>>>> gmx-developers-request at __groma**__cs.org<http://groma__cs.org><
>>>>>> http://gromacs.org>
>>>>>> <mailto:gmx-developers-__**request at gromacs.org<gmx-developers-__request at gromacs.org>
>>>>>>
>>>>>> <mailto:gmx-developers-**request at gromacs.org<gmx-developers-request at gromacs.org>
>>>>>> >>.
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> gmx-developers mailing list
>>>>>> gmx-developers at gromacs.org <mailto:gmx-developers@**
>>>>>> gromacs.org <gmx-developers at gromacs.org>>
>>>>>> <mailto:gmx-developers at __groma**cs.org <http://gromacs.org>
>>>>>>
>>>>>> <mailto:gmx-developers@**gromacs.org<gmx-developers at gromacs.org>
>>>>>> >>
>>>>>>
>>>>>> http://lists.gromacs.org/__**mailman/listinfo/gmx-__**
>>>>>> developers<http://lists.gromacs.org/__mailman/listinfo/gmx-__developers>
>>>>>> <http://lists.gromacs.org/**mailman/listinfo/gmx-**developers<http://lists.gromacs.org/mailman/listinfo/gmx-developers>
>>>>>> >
>>>>>> Please don't post (un)subscribe requests to the list.
>>>>>> Use the
>>>>>> www interface or send it to
>>>>>> gmx-developers-request at __groma**cs.org <http://gromacs.org>
>>>>>> <mailto:gmx-developers-**request at gromacs.org<gmx-developers-request at gromacs.org>
>>>>>> >
>>>>>> <mailto:gmx-developers-__**request at gromacs.org<gmx-developers-__request at gromacs.org>
>>>>>> <mailto:gmx-developers-**request at gromacs.org<gmx-developers-request at gromacs.org>>>.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> ------------------------------**__---------------------
>>>>>> Dr. Jochen Hub
>>>>>> Computational Molecular Biophysics Group
>>>>>> Institute for Microbiology and Genetics
>>>>>> Georg-August-University of Göttingen
>>>>>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
>>>>>> Phone: +49-551-39-14189 <tel:%2B49-551-39-14189>
>>>>>> http://cmb.bio.uni-goettingen.**__de/ <http://cmb.bio.uni-**
>>>>>> goettingen.de/ <http://cmb.bio.uni-goettingen.de/>>
>>>>>> ------------------------------**__---------------------
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> ------------------------------**---------------------
>>>>> Dr. Jochen Hub
>>>>> Computational Molecular Biophysics Group
>>>>> Institute for Microbiology and Genetics
>>>>> Georg-August-University of Göttingen
>>>>> Justus-von-Liebig-Weg 11, 37077 Göttingen, Germany.
>>>>> Phone: +49-551-39-14189
>>>>> http://cmb.bio.uni-goettingen.**de/<http://cmb.bio.uni-goettingen.de/>
>>>>> ------------------------------**---------------------
>>>>> --
>>>>> gmx-developers mailing list
>>>>> gmx-developers at gromacs.org
>>>>> http://lists.gromacs.org/**mailman/listinfo/gmx-**developers<http://lists.gromacs.org/mailman/listinfo/gmx-developers>
>>>>> Please don't post (un)subscribe requests to the list. Use the www
>>>>> interface or send it to gmx-developers-request@**gromacs.org<gmx-developers-request at gromacs.org>
>>>>> .
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
>>> 865-241-1537, ORNL PO BOX 2008 MS6309
>>>
>>> --
>>> gmx-developers mailing list
>>> gmx-developers at gromacs.org
>>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>>> Please don't post (un)subscribe requests to the list. Use the
>>> www interface or send it to gmx-developers-request at gromacs.org.
>>>
>>
>>
>
>
> --
> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> 865-241-1537, ORNL PO BOX 2008 MS6309
>
> --
> gmx-developers mailing list
> gmx-developers at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-developers-request at gromacs.org.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20121018/c8d8c02d/attachment.html>
More information about the gromacs.org_gmx-developers
mailing list