Re: fsync in glib/gio
- From: Michael Meeks <michael meeks novell com>
- To: Alexander Larsson <alexl redhat com>
- Cc: gtk-devel-list gnome org
- Subject: Re: fsync in glib/gio
- Date: Fri, 13 Mar 2009 10:23:13 +0000
Hi Alex,
On Fri, 2009-03-13 at 08:38 +0100, Alexander Larsson wrote:
> If you want to you can make all i/o sync by mounting it as such. But
> thats of course really slow. Generally the gio file write operations are
> used for saving files, and people sort of expect that when save returns
> the file is ok on disk.
Sure - which is basically what ext3 provides with it's default ordered
mode right ? metadata hits the disk after data - ok, so all pending data
writes were flushed as well as a side-effect(?) but ... ;-)
> And to make matters worse, its perfectly ok for a filesystem (ext4, xfs
> and btrfs do this atm) to reorder the metadata and the data writes such
> that writing to a temp file, renaming over target file and then crashing
> can end up with an empty file. This happens if metadata was saved but
> not the new file contents, and the window for this is about a minute, so
> its not a small race condition.
Sure, sure - and I guess this is why they had to make 'fsync' not suck
for these filing systems.
> So, you save and the system hangs 10 seconds later. What do you expect?
> Ideally the new file, less problematic the old file. But, if you're left
> with *no* version of the file I'd be pretty pissed off.
Yep; of course this is not good.
> With the data=ordered setting in ext3 (the default), any fsync will
> result in all dirty data being flushed, not just the data in that file.
> This can be pretty expensive if there is a lot of outstanding I/O.
Sure - on ext3 it's a 'sync()' call effectively, with the added bonus
that this has a terrible effect on other applications trying to use the
I/O subsystem to eg. read audio to play your mp3, or swap, or allocate
memory, or ... ;-)
Calling 'fsync' regularly on ext3 will bring your system to a grinding
halt, quite regularly, cf. my comments on hair-loss etc. Lots of our
users will be on these systems, and last I checked [ did you poke your
kernel guys on this ? ] this was not recommended.
> However, this is only a problem if such an operation happens often, and
> file saving is just not that common.
Sure; although amusingly, if it happens often ~enough (say every few
hundred ms) - then we almost move the OS into a semi-synchronous writes
mode, and have somewhat less to write each time, and so don't suffer
multi-second glitches in the I/O subsystem ;-)
> And if something constantly is saving something that is a problem
> for multiple other reasons too and should be fixed.
Oh - well, of course there is regular autosave, and setting of gconf
settings [ we set ~200+ keys on login amazingly ;-], then of course the
IM infrastructure will want to make sure your IM logs are *really*
on-disk each time you get a message, the E-mail client for every mail
message ;-) pretty soon if everyone calls 'fsync' we end up in a fairly
bad place (IMHO).
> Of course, not all file writes are saves. For example, it could be
> nautilus copying 10000 files. This is why I added the ASYNC_WRITE flag
> and used it in the file copy case.
I guess.
> It occurs often enought that there were several people in the ubuntu
> ext4-eats-my-data bug that had it happen to them multiple times.
Nasty indeed, and this is the only solution. I suppose the nutshell of
my concern is this:
* can we in some (evil or otherwise) way avoid hurting
desktop performance, interactivity and playback for
all ext2/3 systems, and still keep ext4 users happy ? :-)
Of course, perhaps I'm just way off / out of date wrt. my dislike of
fsync, let me hunt down some kernely people for a sane 2nd opinion.
Regards,
Michael.
--
michael meeks novell com <><, Pseudo Engineer, itinerant idiot
[
Date Prev][
Date Next] [
Thread Prev][
Thread Next]
[
Thread Index]
[
Date Index]
[
Author Index]