[gmx-developers] manpage generation brakes build process

[Szilárd Páll]

PS: Does anyone know what is the easiest way to compile a/some file
with manually set compiler flags? Overriding compiler flags for a
single file is not possible (only appending).

--
Szilárd


On Thu, Aug 30, 2012 at 9:38 PM, Szilárd Páll <szilard.pall at cbr.su.se> wrote:
> On Thu, Aug 30, 2012 at 4:59 PM, Szilárd Páll <szilard.pall at cbr.su.se> wrote:
>> On Thu, Aug 30, 2012 at 4:38 PM, Roland Schulz <roland at utk.edu> wrote:
>>> Hi,
>>>
>>>
>>>
>>> On Thu, Aug 30, 2012 at 9:03 AM, Szilárd Páll <szilard.pall at cbr.su.se>
>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> Commit 26bd5f7 made the build process generate man pages and broke
>>>> compilation for the most common cross-compile builds: when the build
>>>> host doesn't support the instruction set of the target, e.g. a
>>>> cluster's head node with "older" CPUs than the cluster nodes. To me it
>>>> seems that the CMAKE_CROSSCOMPILING variable used to implement the
>>>> detection of cross-compilation is pretty much useless because
>>>> apparently it takes into account only the CMAKE_SYSTEM_NAME. This only
>>>> detects when one builds e.g Windows binaries on Linux, but nothing
>>>> else.
>>>>
>>>> The annoying part is that the build process fails a cryptic "error
>>>> 132" after linking the binary and the user won't have a clue what's
>>>> wrong.
>>>>
>>>> In my opinion by default automatically generating man pages is not
>>>> very important because:
>>>> - source releases can contain the man pages;
>>>
>>> This is already so. Cpack automatically adds the build man pages and the man
>>> pages are only build if they aren't in the source package. So this is a
>>> non-issue for users which download the source as a source package instead
>>> from git.
>>>
>>>>
>>>> - the development versions are either:
>>>>   - in pre-release stage when one wouldn't rely on the man pages being
>>>> very up-to-date anyway;
>>>>   - post-release stage when there should not be many major changes
>>>> compared to the latest patch release.
>>>>
>>>> I would suggest that we stick to installing static in-source man pages
>>>> by default and we can provide a switch to generate them during
>>>> building. It is fairly complicated to detect whether the binaries
>>>> built will run on the host,
>>>
>>> why? It is as simple as running a binary and see whether it worked, isn't
>>> it?
>>
>> A binary or the binary that would be required to execute successfully
>> to generate the man page?
>>
>> If it's only the architecture, instruction set, or operating system
>> incompatibility that can cause issues compiling and running a dummy
>> program should indeed be fine. However, I'm not sure that's always the
>> case. I'll think it through.
>>
>> Does anybody know of cross-compilation cases where a dummy program
>> would execute, but e.g. mdrun wouldn't?
>
> After giving this a second thought and discussing it with others, I
> think that it is not unfeasible to come up with a dummy program that
> tests weather the given CFLAGS will produce GROMACS binaries that can
> run on the host machine. However, this solution will certainly not be
> robust and will probably be a  maintenance headache.
>
> The issue is that illegal instructions are not only the asm or
> intrinsics used explicitly in the code but also what the compiler
> generates. As we allow the compiler to generate e.g. AVX instructions
> when setting GMX_ACCELERATION=AVX_256, the compiler will happily
> generate as much AVX as it can. As a result, a cross-compiled binary
> will in my case (compiled with AVX on a CPU with SSE2 support) fail
> already inside CopyRight(), apparently in the bromacs() call.
>
> Therefore, I think we either have to revert back to using in-source
> man pages (at least by default) or come up with a robust mechanism
> that at least will clearly state what the problem is instead of
> leaving the user with an "error 123"-type message (and some .err files
> deep in the build tree).
>
> Additionally, it would also be useful to have an early runtime check
> that warns the user if it looks like the binary will/could fail. This
> can happen relatively often e.g. when using multiple clusters with
> shared file system from which users can inevitably end up running a
> binary incompatible with the target cluster. A solution for this is to
> compile the CPU detection module with no optimization, run the
> detection as the very first thing in every tool's main() function and
> issue an error if it is sure that things will go wrong or at least a
> warning if the build host and target are different.
>
> --
> Szilárd
>
>>>
>>>>
>>>> so I think it is not worth writing such
>>>> code that is error-prone and hard to maintain.
>>>>
>>>> Does anybody have a better idea to fix this issue?
>>>
>>> I prefer if it is done automatically because that makes sure it is
>>> maintained. I think we should have as little as possible of things we do in
>>> the last second before a release (and thus delaying the release) to be able
>>> to release more frequently.
>>
>> Maintaining the documentation has not much to with whether it is built
>> at run-time or not and I can't imagine that there have been so many
>> bugs in the man page generation itself (which is the only thing that's
>> ensured to be working through the auto-builds).
>>
>>> I think it is OK as it is. Because disabling the build is as easy as
>>> GMX_BUILD_MANPAGES=no, and only effects developers and not users. But if you
>>> want me to add a check whether executing binaries is possible, I can do
>>> that.
>>
>> Actually, many advanced users use the git version of the releases --
>> partly because of the super-sluggish release cycle. I'm not saying
>> that the feature is not useful, but users shouldn't have to first
>> figure out how to make compilation not fail with a cryptic error that
>> doesn't even suggest a solution.
>>
>> Please do implement it If you (plural intended) think that the feature
>> is robust enough.
>>
>>> admin/.isreposource is not specific for github. It is for any download which
>>> is not using git to download source which didn't go through cpack.
>>
>> That is? Still sounds quite exotic, 2-3 official means of getting
>> Gromacs should be more than enough (git.gromacs.org, nightly source,
>> github as backup).
>>
>> Again, I don't mind a single hidden file, but IMO we shouldn't add
>> code, not even a single line just to support some exotic way to obtain
>> the source.
>>
>> --
>> Szilárd
>>
>>>  Roland
>>>
>>>>
>>>> --
>>>> Szilárd
>>>> --
>>>> gmx-developers mailing list
>>>> gmx-developers at gromacs.org
>>>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>>>> Please don't post (un)subscribe requests to the list. Use the
>>>> www interface or send it to gmx-developers-request at gromacs.org.
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
>>> 865-241-1537, ORNL PO BOX 2008 MS6309
>>>
>>> --
>>> gmx-developers mailing list
>>> gmx-developers at gromacs.org
>>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>>> Please don't post (un)subscribe requests to the list. Use the
>>> www interface or send it to gmx-developers-request at gromacs.org.