Re: Performance issue when trashing files (backtraced to fsync)

[Alexander Larsson <alexl redhat com>]

On Tue, 2009-08-11 at 16:09 +0200, Mark wrote:
> Hi,
> 
> A few days ago i deleted thousands of files with just nautilus. That
> went fine but horribly slow. doing the same with the rm command was
> way faster.
> I made a bug report about it yesterday with my results from that
> moment: http://bugzilla.gnome.org/show_bug.cgi?id=591363
> 
> So i tried to trace this issue down. First i added dozens of debug
> messages to the function: g_local_file_trash in the file glocalfile.c
> and that resulted in one function that sucked up time.
> The function was: g_file_set_contents (infofile, data, -1, NULL);

So, the issue here is that for each file we trash we write a small
keyfile with some information about the file. Additionally we use atomic
create to ensure that we get a unique filename for the trash (since two
trash operations with the same basename could happen at the same time).

This leads to a call to g_file_set_contents() with an existing file.
This causes us to use fsync to avoid losing the existing file due to a
possible system crash.

Clearly this is way to expensive in this case, going from nine seconds
to almost two minutes.

There are various possible solutions:

1. never fsync and risk data crash
2. add some way in the API to say we don't want to fsync and use this
   for "unimportant" files
3. Extend the checking for existing file case to check if the existing
   file is 0 bytes (would be the case here) and in that case avoid
   fsync.

Clearly we should do at least 3, which will fix this case (and other
similar tempfile cases). However, given the extremely bad performance
here we should maybe add the extra API in 2 allowing apps to avoid the
cost when needed? Its kinda ugly to expose that to users, but the
performance cost is pretty ugly too...

Opinions?