[gmx-users] why does trjcat take so much memory?

[Roland Schulz]

Chris,

one quick look at the code suggest that the reason is that the memory isn't
freed correctly. read_first_frame is allocating the memory for one frame but
in most cases it isn't freed. Thus you only see this error with many small
files.

Since it is open source the best way to get it fixed, is to quickly do it
yourself ;-)

Roland

On Wed, Oct 13, 2010 at 1:40 AM, <chris.neale at utoronto.ca> wrote:

> Thank you for the insight Mark.
>
> To get it on the record, this is fairly annoying for trjcat (not trjconv).
> If anyone else has run into this, then please do post because a memory
> efficient cat mechanism will only be coded if there is sufficient interest.
>
> But again, thanks for your good answer Mark, probably you are correct.
>
> Chris.
>
> ----- Original Message -----
> From: chris.neale at utoronto.ca
> Date: Wednesday, October 13, 2010 10:55
> Subject: [gmx-users] why does trjcat take so much memory?
> To: gmx-users at gromacs.org
>
> [Hide Quoted Text]
> I have 28,000 .xtc files, each having a single frame and each
> 150K. If I run du -hs on the directory containing these .xtcs, I
> get 4.4 GB. Nevertheless, when I run trjcat -f *.xtc -o
> ../tot.xtc , my memory consumption goes over 11 GB and then I
> run out of available memory.
>
> Sure, I could find a larger memory system, but why does it take
> =3x more memory than the individual files took in disk space?
> I'd guess that the increased space reflects trjconv converting the
> compressed .xtc format back to normal precision, and then back to
> compressed. To cope with all its possibilities, trjconv probably needs to
> either keep the loaded frames in memory, or use a two-pass algorithm. It
> does the former.
>
> Mark
>
>
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use thewww interface
> or send it to gmx-users-request at gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>



-- 
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-users/attachments/20101013/f142758d/attachment.html>