[gmx-developers] Re: flushing files

[Erik Lindahl]

Hi,

File flushing has been a huge issue to get working properly on AFS and other systems that have an extra layer of network disk cache. We also want to make sure the files are available e.g. on the frontend node of a cluster while the simulation is still running.

I think the proper solution is rather to have a separate IO thread so the disk operation can take all the latency in the world without delaying the run.

You are more then welcome to play with it (but not in the release branch!) - you might already have an account on the AFS-equipped clusters here, or we can arrange it!

Alternatively, sync with Sander and he might be able to test new code on AFS.

Cheers,

Erik 


On Oct 12, 2010, at 23:26, Roland Schulz <roland at utk.edu> wrote:

> Erik, Berk,
> 
> you added flushing of trn, xtc and ern before the checkpointing functionality had been added. The additional flush can add quite a bit of unnecessary time especially with parallel file systems and/or MPI-IO. Am I right that with checkpointing it is not necessary anymore?  The checkpointing is flushing the file before writing the checkpoint. Also gmx_fio_check_file_position is flushing the file before checking whether the file is too large for gmx_off_t
> Roland
> 
> -- 
> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> 865-241-1537, ORNL PO BOX 2008 MS6309
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20101013/b3b50ec3/attachment.html>