[gmx-developers] Native endianess in TPR body

[Len Kimms]

Hello everyone,

while fooling around with the new (i.e. version 2020 rc1) TPR file format I noticed some strange behaviors that I don’t understand. As far as I understand the body of the new format is written by the `gmx::InMemorySerializer`. My following questions are basically about this module.

First it seems that the memory serializer writes the values in native byte order. This means that the body of TPR files differ between big- and little-endian systems. The XDR standard used before requires big-endian data. For me, a novice user, the new implementation seems to be less portable and robust. Endian swapping seems to be implemented but not currently used for TPR files.
Is this intentional, if so, why?

Second the individual bytes of a value are padded to 4 bytes per original bytes (each byte is packed as `char`). Therefore the size increases accordingly.
Do those padding bytes serve a special purpose?
Also regarding the padding bytes: Some bytes are not, like most others, padded with zeros. In some places they are padded with ones. At first glance this seem to happen to the second byte (big-endian) of a float. From some initial testing my best guess is, that this is caused by the union conversion in `CharBuffer`. With an `unsigned char` in the private union `u` those values would be zero padded.

In the attachment one could find example files from a big- and little-endian system as well as a file created with GROMACS 2019.
I also brought this to the attention of the MDAnalysis devs here:
https://github.com/MDAnalysis/mdanalysis/issues/2428

Best regards,
   Len
-------------- next part --------------
A non-text attachment was scrubbed...
Name: n3_be.tpr
Type: application/octet-stream
Size: 7956 bytes
Desc: not available
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20191226/9683b712/attachment-0003.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: n3_le.tpr
Type: application/octet-stream
Size: 7956 bytes
Desc: not available
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20191226/9683b712/attachment-0004.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: n3.tpr
Type: application/octet-stream
Size: 2196 bytes
Desc: not available
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20191226/9683b712/attachment-0005.obj>