[gmx-developers] manpage generation brakes build process

Thu Aug 30 22:40:05 CEST 2012

Hi,

I think the best approach is not to use a dummy program but the programs
itself. That is guaranteed to be robust and should be simple.
Thus we would always try to generate the man pages but would suppress any
error message but store whether it succeeded. Then at the end if any
program execution failed we just give a warning that not all man-pages will
be available.

Roland

On Thu, Aug 30, 2012 at 3:38 PM, Szilárd Páll <szilard.pall at cbr.su.se>wrote:

> On Thu, Aug 30, 2012 at 4:59 PM, Szilárd Páll <szilard.pall at cbr.su.se>
> wrote:
> > On Thu, Aug 30, 2012 at 4:38 PM, Rolandp Schulz <roland at utk.edu> wrote:
> >> Hi,
> >>
> >>
> >>
> >> On Thu, Aug 30, 2012 at 9:03 AM, Szilárd Páll <szilard.pall at cbr.su.se>
> >> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Commit 26bd5f7 made the build process generate man pages and broke
> >>> compilation for the most common cross-compile builds: when the build
> >>> host doesn't support the instruction set of the target, e.g. a
> >>> cluster's head node with "older" CPUs than the cluster nodes. To me it
> >>> seems that the CMAKE_CROSSCOMPILING variable used to implement the
> >>> detection of cross-compilation is pretty much useless because
> >>> apparently it takes into account only the CMAKE_SYSTEM_NAME. This only
> >>> detects when one builds e.g Windows binaries on Linux, but nothing
> >>> else.
> >>>
> >>> The annoying part is that the build process fails a cryptic "error
> >>> 132" after linking the binary and the user won't have a clue what's
> >>> wrong.
> >>>
> >>> In my opinion by default automatically generating man pages is not
> >>> very important because:
> >>> - source releases can contain the man pages;
> >>
> >> This is already so. Cpack automatically adds the build man pages and
> the man
> >> pages are only build if they aren't in the source package. So this is a
> >> non-issue for users which download the source as a source package
> instead
> >> from git.
> >>
> >>>
> >>> - the development versions are either:
> >>>   - in pre-release stage when one wouldn't rely on the man pages being
> >>> very up-to-date anyway;
> >>>   - post-release stage when there should not be many major changes
> >>> compared to the latest patch release.
> >>>
> >>> I would suggest that we stick to installing static in-source man pages
> >>> by default and we can provide a switch to generate them during
> >>> building. It is fairly complicated to detect whether the binaries
> >>> built will run on the host,
> >>
> >> why? It is as simple as running a binary and see whether it worked,
> isn't
> >> it?
> >
> > A binary or the binary that would be required to execute successfully
> > to generate the man page?
> >
> > If it's only the architecture, instruction set, or operating system
> > incompatibility that can cause issues compiling and running a dummy
> > program should indeed be fine. However, I'm not sure that's always the
> > case. I'll think it through.
> >
> > Does anybody know of cross-compilation cases where a dummy program
> > would execute, but e.g. mdrun wouldn't?
>
> After giving this a second thought and discussing it with others, I
> think that it is not unfeasible to come up with a dummy program that
> tests weather the given CFLAGS will produce GROMACS binaries that can
> run on the host machine. However, this solution will certainly not be
> robust and will probably be a  maintenance headache.
>
> The issue is that illegal instructions are not only the asm or
> intrinsics used explicitly in the code but also what the compiler
> generates. As we allow the compiler to generate e.g. AVX instructions
> when setting GMX_ACCELERATION=AVX_256, the compiler will happily
> generate as much AVX as it can. As a result, a cross-compiled binary
> will in my case (compiled with AVX on a CPU with SSE2 support) fail
> already inside CopyRight(), apparently in the bromacs() call.
>
> Therefore, I think we either have to revert back to using in-source
> man pages (at least by default) or come up with a robust mechanism
> that at least will clearly state what the problem is instead of
> leaving the user with an "error 123"-type message (and some .err files
> deep in the build tree).
>
> Additionally, it would also be useful to have an early runtime check
> that warns the user if it looks like the binary will/could fail. This
> can happen relatively often e.g. when using multiple clusters with
> shared file system from which users can inevitably end up running a
> binary incompatible with the target cluster. A solution for this is to
> compile the CPU detection module with no optimization, run the
> detection as the very first thing in every tool's main() function and
> issue an error if it is sure that things will go wrong or at least a
> warning if the build host and target are different.
>
> --
> Szilárd
>
> >>
> >>>
> >>> so I think it is not worth writing such
> >>> code that is error-prone and hard to maintain.
> >>>
> >>> Does anybody have a better idea to fix this issue?
> >>
> >> I prefer if it is done automatically because that makes sure it is
> >> maintained. I think we should have as little as possible of things we
> do in
> >> the last second before a release (and thus delaying the release) to be
> able
> >> to release more frequently.
> >
> > Maintaining the documentation has not much to with whether it is built
> > at run-time or not and I can't imagine that there have been so many
> > bugs in the man page generation itself (which is the only thing that's
> > ensured to be working through the auto-builds).
> >
> >> I think it is OK as it is. Because disabling the build is as easy as
> >> GMX_BUILD_MANPAGES=no, and only effects developers and not users. But
> if you
> >> want me to add a check whether executing binaries is possible, I can do
> >> that.
> >
> > Actually, many advanced users use the git version of the releases --
> > partly because of the super-sluggish release cycle. I'm not saying
> > that the feature is not useful, but users shouldn't have to first
> > figure out how to make compilation not fail with a cryptic error that
> > doesn't even suggest a solution.
> >
> > Please do implement it If you (plural intended) think that the feature
> > is robust enough.
> >
> >> admin/.isreposource is not specific for github. It is for any download
> which
> >> is not using git to download source which didn't go through cpack.
> >
> > That is? Still sounds quite exotic, 2-3 official means of getting
> > Gromacs should be more than enough (git.gromacs.org, nightly source,
> > github as backup).
> >
> > Again, I don't mind a single hidden file, but IMO we shouldn't add
> > code, not even a single line just to support some exotic way to obtain
> > the source.
> >
> > --
> > Szilárd
> >
> >>  Roland
> >>
> >>>
> >>> --
> >>> Szilárd
> >>> --
> >>> gmx-developers mailing list
> >>> gmx-developers at gromacs.org
> >>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> >>> Please don't post (un)subscribe requests to the list. Use the
> >>> www interface or send it to gmx-developers-request at gromacs.org.
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> >> 865-241-1537, ORNL PO BOX 2008 MS6309
> >>
> >> --
> >> gmx-developers mailing list
> >> gmx-developers at gromacs.org
> >> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> >> Please don't post (un)subscribe requests to the list. Use the
> >> www interface or send it to gmx-developers-request at gromacs.org.
> --
> gmx-developers mailing list
> gmx-developers at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-developers-request at gromacs.org.
>
>
>
>
>

-- 
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20120830/682054f4/attachment.html>