[gmx-developers] Re: TNG format in Gromacs

[Erik Lindahl]

Hi,

On Apr 17, 2012, at 6:14 PM, Szilárd Páll wrote:

> a) The need for a new format which will *immediately* replace XTC as
> the default in Gromacs and seems to have requirements that pretty much
> exclude any external library (that is not as widespread as libc :).
> 
> b) It's not in line with the EU deliverable which requires a new
> library with certain specs to be written. Could it be that this is the
> classic case where the specification was created before the actual
> requirement engineering?

No, I don't think so, but the goals are different: I very much see the point for a file format that enables you to do fancy stuff, but if you want a file format that is supported by as many applications as possible you have to settle for the greatest common divisor.

> 
> c) The apparent need for ultimate portability to extremely rare and
> exotic architectures without accepting XTC as a fallback on these
> platforms (with conversion for post-processing and analysis). I might
> be wrong, but to me it seems that these extremely rare architectures
> are often more showcase platforms rather than the iron on which 99% of
> the science is carried out.

Well, windows was exactly such a very exotic architecture until we started using Gromacs in Folding at Home. In hindsight, the decision to rely on XDR libraries for all our IO was a bad one.

Just counting core-hours, I actually think windows is one of our most common Gromacs architectures today. Similarly, I think there is a clear possibility nativeclient could become a dominant platform quite soon if Google decides to push it for cloud services. It is very difficult to make predictions, in particular about the future. 

*Of course* does not mean a new default file format must be 100% portable without modifying the code. 

> Wouldn't it be beneficial to struck a balance
> between portability and effort/time required by accepting a short-term
> "compromise" (which isn't really a compromise if we don't consider
> ultimate portability a strict requirement :). XTC will have to be
> anyway maintained anyway so it could as well be kept as the
> alternative for platforms where the new format is not supported in
> early versions. So a file format that works on all reference platforms
> (that we can and should simply list and track) with XTC  as a fallback
> for the exotic iron should be an acceptable compromise, I'd say.

I think that's a perfectly valid short- and long-term balance *as long as there is a way to gradually expand support to other platforms as we need it*.  

The problem isn't the unimportant platforms, but when a new platform gets important for us, while it might not yet be important for the external library. What do we do when 10MB of source code fails to work on one of our reference platforms?


Cheers,

Erik