[gmx-developers] Help needed with tng_io library

[Magnus Lundborg]

Hi Semen,

I found some extra time, so I will try to clarify some things below.

On 2014-08-07 12:16, Semen Esilevsky wrote:
> Dear All,
> I'm currently trying to add support for tng files to my code and I realised that I can't understand the logic of the library at all. I'm definetely doing something wrong, but can't figure out a mistake.
> The tng code is got from the latest git. Test trajectory is generated by Gromacs 5.0.
>
>
> I can open the tng file and can extract, for example, position exponential correctly, but after that...
>
>
> Problem #1: getting number of frames.
>
> int64_t n1,n2;
> tng_num_frames_get(trj, &n1);
> tng_num_frame_sets_get(trj, &n2);
>
> This gets
> n1 = 10761
> n2 = 1086
>
> The correct number of frames is 1086.
>
> What on Earth is returned from tng_num_frames_get()?
One thing that is slightly difficult with TNG is that different kinds of 
data can be written at different intervals, e.g. it can have positions 
written every 100th frame, forces every 200th frame, velocities every 
250th frame etc. To make this a bit more graspable in GROMACS every md 
step is considered a frame when writing the TNG file (it would be 
possible to let the n steps between frames be the greatest common 
denominator of the written data types as well). The default when writing 
TNG from a GROMACS simulation is to keep 100 frames of the most 
frequently written data in each frame set. When generating a TNG from 
trjconv (like the file you provided) it is not as easy to predict what 
data to expect and the output frequency of it. Therefore a frame set is 
created every time data is written - this will probably be changed in 
the future to facilitate more efficient compression from trjconv too.
>
> Problem #2: Reading past the end of file.
>
> stat = tng_util_particle_data_next_frame_read(trj, TNG_TRAJ_POSITIONS, &values,
>                                                    &datatype, &frame, &physical_time);
>
> This function should return false when no more frames are available, right? It doesn't do this and reads forever returning garbage in values when called in a loop. stat is always true. How should I use it correctly to finish reading at the end of file?
> Also concerning the strange thing with the number of frames and frame sets - what it reads actually each time?
>
> Problem #3: tng_util_pos_read_range() behaves insane
>
> int n=0;
> bool stat = true;
> while(stat){
>      stat = tng_util_pos_read_range(trj,n,n, &ptr, &len);
>      n++;
> }
>
> This always stops at n=10 (why?!), while the number of frames is 1086! What I'm doing wrong here?
I will have a look at this, and will try to give more details next week 
when I've got more time.
>
> I've also tried VMD plugin from pre-release VMD 1.9.2. It is said that this plugin is contributed by Gromacs developers, so it should work in principle. It doesn't work because of the bug #2 - it reads forever beyond the end of file.
This bug was discovered a few days after the release and is fixed in the 
TNG library in the TNG repository. The fix is included in the 
release-5-0 branch of GROMACS, but I don't think there has been any bug 
fix release. I also think it will be fixed in the next VMD pre-release.
>
> The test tng file is here: https://drive.google.com/file/d/0Bx_ng_72VH8BQlFsTGJOVC1oMjQ/edit?usp=sharing
>
>
> Any help is appreciated!
I hope the answers help you at least a bit. Please tell me if anything 
is unclear and I will try to explain it better.

Kind Regards,

Magnus
>
> Sincerely,
> Semen
>