[gmx-developers] Re: TNG format in Gromacs

[Erik Lindahl]

Hi,

On Apr 17, 2012, at 4:47 PM, Roland Schulz wrote:

> 
> But now you are moving the goal-posts!  The aim of the present TNG-based project was NOT "fancy" IO, but a new default simple portable Gromacs trajectory format that (1) includes headers for atom names and stuff, (2) is a small free library that can easily be contributed to other codes so they can read/write our files, and (3) enable better compression.
> What I meant with "fancy" IO was that it is optional. These 3 things aren't required to run a simulation on an exotic platform (e.g. Kei) and to be able to analysis the results (after potentially converting).

No. A new default file format will not be "optional".

To be fairly blunt: this work is financed from an EU project with a deliverable to produce a small library that we can offer for inclusion in other codes and push as a standard MD file format, so that part is simply not negotiable.

> 
> Option 1) Up to 100k lines we have to write and support. And the code can only use the subset of HDF5 supported. 
> Option 2) Users on very exotic platforms have to keep using XTC and in post-production convert their files (only if they want to benefit of HDF5 advantages in analysis

> 
> I really don't see how Option 1 could win in any reasonable cost benefit analysis. :-)

Option (2) is not on the table. We simply are NOT introducing a new _default_ file format that does not work everywhere, since that does not fulfill the EU deliverable we have. In that case we are simply going with alternative (3), which is "ditch HDF5 completely" for TNG.  This might sound extreme, but if we have to keep support XTC indefinitely we will in practice never get a new default format - that would de facto still be XTC!

Again, let's separate things:

1) Optional file formats: Here I'm flexible and will accept almost anything.

2) Default core Gromacs file formats: We must be able to get these working on any platform with a reasonable amount of work, and we must be able to provide a small library for other programs to use.


If (2) is possible with a very scaled-down HDF5 implementation that somebody is willing to support and fix portability issues for, that could be an interesting alternative for the container, but if that isn't the case we cannot use HDF5 for (2) - but it would still be fine for (1).


However, I still haven't seen any discussion about the *concrete* features HDF5 would provide for the new XTC-like format?


Cheers,

Erik