[gmx-developers] Re: TNG format in Gromacs

[Roland Schulz]

On Tue, Apr 17, 2012 at 1:21 PM, Erik Lindahl <erik at kth.se> wrote:

> Hi,
>
> On Apr 17, 2012, at 6:58 PM, Roland Schulz wrote:
> >
> > Why does this require a scaled-down version? Why is not option to e.g.
> provide a custom Virtual File Layer? That would put the OS depending part
> in our hands.
>
> This far we've heard three different sizes of HDF5 today, ranging from
> ~50MB source to ~1MB source - I simply don't know how much it will be in
> practice!
>
Uncompressed source archive: 92M
Compressed source archive 5.6M
Src folder: 11M
Line of codes src folder: 135k
Lines of code in files with Windows #ifdefs: 9k


> Once it gets below 1MB uncompressed (preferrably a few hundred K, but
> that's no big deal) we're in the ballpark where it can be included in a
> library for other codes to use, but 50MB source isn't :-)
>
> What is the total size of _all_ the HDF5 code that would have to be
> included in a stand-alone library that does not have to be linked with
> anything else? Is that 130k lines, and could it be reduced further to make
> it easier to support as part of the file format library?
>
Yes I think 135k is the total size we would need (as long as we dont want
HL or C++ binding (would add each 7k)). I'm sure it would be possible to
reduce that, but I don't see an obvious or very quick way.

I think it is a minor point but I think it isn't a good idea to bundle
HDF5. As Christoph just wrote cmake allows to auto-install dependencies.
And if we ever need to make modifications to HDF5 we should provide them as
patches (as long as they aren't included upstream). This way users who
already have HDF5 can use it (without unnecessary downloading the bundled
version) and others have the comfort that dependencies are compiled
automatically (as they would with a bundled library).

> And even without. The OS depending part of HDF5 (otherwise it is also
> ANSI C) are a small part and it is OpenSource. So it is very much possible
> to fix ourselves. I doubt this is comparable to Charm++.
>
> I think this is the core discussion item. Would somebody be willing to
> support this code - including porting to new platforms that are considered
> important for Gromacs even if they are not yet HDF5 platforms?  *Hopefully*
> that will never be any significant amount of work, but I simply don't know
> HDF5 well enough to say, and we don't know what will happen in the future.
>
> However, we don't want a situation where developers add library
> dependencies because it is short-term convenient, and when that library
> later causes portability problems for somebody else nobody really feels
> responsible.
>

I completely agree that we don't want to save 100hrs of developer time now
by using an external library to have to invest 1000hrs in the long run to
make it portable on some unforeseeable and important architectures. On the
other hand if we can save 1000hrs now (only features the majority thinks
are really worth spending time on not just things nice to have when free)
and might need to invest 100hrs later than I think it is something we
should do (I'm not saying we know at this point - this has to be
researched). When this has been decided by the majority as the useful thing
than at this point it becomes (it least to a large percentage)
the responsibility of the person who wants portability for system X to make
it possible (with the option of commercial support)..


> Rossen has already volunteered to check NDF5 on a fujitsu system similar
> to Kei - I think another first obvious step is for somebody to create the
> smallest possible stand-alone version and see how well that compiles on all
> platforms we need, including nativeclient!
>
I can create an archive with the 130k lines and test that it compiles in
general. The disadvantage would be it wouldn't contain the tests which one
might want to run on the different systems to see that it actually works.
I'm also happy to test it on all architectures I have access to (nothing
really exotic).

Roland

>
> Cheers,
>
> Erik
>
>
>
>


-- 
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20120417/07706369/attachment.html>