Re: Performance issue when trashing files (backtraced to fsync)

[Alexander Larsson <alexl redhat com>]

On Tue, 2009-08-11 at 17:27 +0200, Mark wrote:
> On Tue, Aug 11, 2009 at 4:46 PM, Alexander Larsson<alexl redhat com> wrote:
> > On Tue, 2009-08-11 at 16:09 +0200, Mark wrote:
> >> Hi,
> >>
> >> A few days ago i deleted thousands of files with just nautilus. That
> >> went fine but horribly slow. doing the same with the rm command was
> >> way faster.
> >> I made a bug report about it yesterday with my results from that
> >> moment: http://bugzilla.gnome.org/show_bug.cgi?id=591363
> >>
> >> So i tried to trace this issue down. First i added dozens of debug
> >> messages to the function: g_local_file_trash in the file glocalfile.c
> >> and that resulted in one function that sucked up time.
> >> The function was: g_file_set_contents (infofile, data, -1, NULL);
> >
> > So, the issue here is that for each file we trash we write a small
> > keyfile with some information about the file. Additionally we use atomic
> > create to ensure that we get a unique filename for the trash (since two
> > trash operations with the same basename could happen at the same time).
> >
> > This leads to a call to g_file_set_contents() with an existing file.
> > This causes us to use fsync to avoid losing the existing file due to a
> > possible system crash.
> >
> > Clearly this is way to expensive in this case, going from nine seconds
> > to almost two minutes.
> 
> It's not nine seconds. It's 0.9 seconds (1 second if you will).
> 
> >
> > There are various possible solutions:
> >
> > 1. never fsync and risk data crash
> > 2. add some way in the API to say we don't want to fsync and use this
> >   for "unimportant" files
> > 3. Extend the checking for existing file case to check if the existing
> >   file is 0 bytes (would be the case here) and in that case avoid
> >   fsync.
> >
> > Clearly we should do at least 3, which will fix this case (and other
> > similar tempfile cases). However, given the extremely bad performance
> > here we should maybe add the extra API in 2 allowing apps to avoid the
> > cost when needed? Its kinda ugly to expose that to users, but the
> > performance cost is pretty ugly too...
> >
> > Opinions?
> 
> What i find strange is moving files or folders form location x to
> location y is rapid but moving them to the trash is slow.. why is
> that?

Moving a file doesn't fsync, its only writing the trash info file that
is slow.

> And isn't moving a file as simple as changing a value in the inode
> table (i only know it exists and holds file information but i don't
> know the details of it)?

Well, move is quite a bit more complex than that, but yes, its quite
fast, as is a file write. The only slow part is fsync.