Re: GNOME VFS and FUSE.

[Joe Shaw <joeshaw novell com>]

Hi,

On Fri, 2006-07-07 at 11:47 +0100, Darren Kenny wrote:
> According to Robert Love[1] the prospect of FUSE replacing GNOME VFS was well
> received at GUADEC. I was present, and took part from a Solaris standpoint, in
> the discussion about this at Alex's talk about the future of GNOME VFS.
>
> I certainly didn't think that it was well received - more a controversial
> change I would think. It seemed to me that there was a split in the room.

I rather disagree.  I thought the discussion was positive.  I think most
people left the meeting with both the positives and negatives of using
FUSE, and there was interest in investigating and prototyping the
feasibility of the solution.  This was the best possible outcome in my
mind.

> I don't see why we should push out a perfectly good GNOME VFS implementation,
> with a rich API, to be replaced with a POSIX based file API that would result
> in some weird uses of ioctl()s and the like to access meta-information. Not to
> mention the kernel context switching that would result from such calls.

I find this paragraph to be rather ironic, since gnome-vfs was designed
by Ettore in 1999 to be a virtual duplicate of the POSIX API, being what
most people were familiar with.  The two additions of course were
dealing with URIs instead of paths (allowing essentially any backend
source, like HTTP or SMB) and an asynchronous API.  After Eazel took it
over it did evolve a bit, mostly to suit the needs of Nautilus, but I
feel that saying that the GNOME VFS API is "rich" is quite an
overstatement.

For ioctl()s, I don't remember the exact context, but I thought it was
fairly well agreed upon that most out-of-band control could be handled
using D-BUS, which would provide a richer and more type-safe API.

Lastly, the kernel context switching is something that needs to be
addressed.  This is why a prototype would be helpful; we can measure
these things.  After the talk, though, Robert and I discussed this a
bit, and we think that the situation is not measurably worse than it is
now.

Essentially, the only 3 VFS modules anyone uses today are the SMB one,
the sftp one, and the HTTP (webdav) one:

        * The SMB code exists entirely within the gnome-vfs daemon for
        licensing reasons.  That means that any gnome-vfs app needs to
        communicate with the daemon and there is a context switch
        anyway.
        
        * The sftp module calls out to ssh, so there is fork and exec
        overhead and a context switch anyway.
        
        * The HTTP module only makes sense for webdav, not for general
        HTTP streaming.  As was pointed out by Bastien, Alex, and others
        in the talk, for that you really want to be using an HTTP
        library like neon, curl, or soup.  On systems where context
        switching might even remotely matter, it seems like this method
        won't be used enough to show up.
        
Even on systems where context switches are expensive, like ARM, the
overhead is probably not likely to even show up on a profile.  There are
doubtless many other areas where optimization work would be better
focused.  That said, this is the reason why we need a prototype before
we can really move on this.

> I makes more sense to me to fix/address the "concerns" that people have with
> the GNOME VFS API - and these mainly seem to be down to complexity - or a
> thinking that it's too difficult to use - where does this come from? If it's
> really like this, then it seems we need to provide a simple version of the
> API for people that need it.

I don't see this as an argument in favor of either technology.  We need
these changes *anyway*.  We need a stream API, we need good asynchronous
IO in our platform.

> An other issue that was raised was the reliability of GNOME VFS, in that there
> was a risk of data-loss - this is surely a bug and should be verifiable
> through a thorough test-suite - if it still exists. I would think the most
> likely way that there could be a risk of data loss is in the realms of network
> based communication - and I can't see how FUSE would be any different here.

I totally agree.  A test suite is essential in either case.  Again, I
don't see this as a strength of the VFS position.

> Personally, I would prefer to see us move with FUSE layered on top of GNOME
> VFS as the solution for the legacy applications - why should GNOME apps have
> to suffer a base API like POSIX - which you have to admit is quite likely to
> be wrapped in a more "user-friendly" API again, since it'd be so complex to
> get simple meta-data or even decent error return codes, and we've back to
> where we started with another GNOME VFS later.

The important distinction is that (now that we have a better idea what
we're doing) we'll be dealing with file descriptors and streams, which
will give us a lot more flexibility over the GNOME VFS APIs we have
today.  As I tried to make clear during the talk, these are pieces of
the platform we desperately need anyway, totally independent of
gnome-vfs vs. FUSE.

I too would like to see a GNOME-VFS FUSE module.  At the very least it
gives us an interim solution, and ultimately might turn out to be the
correct one in the long run.

> I mentioned at GUADEC that there wasn't a FUSE implementation for Solaris - I
> was wrong, there is an Open Solaris project that is working on porting it[2]
> but given that it's GPL licensed it's highly unlikely that it will be included
> as a base Solaris component - and I would think this could be an issue for any
> commercial distribution where there are some proprietary elements in it
> (specifically anything that is licensed from a 3rd party under non-disclosure
> agreements).

FreeBSD has FUSE as well, and while I don't know its lineage, I have to
believe that it is BSD-licensed rather than GPL.  Maybe it could be a
basis, or maybe someone can just ask the author about relicensing it. :)

I could expound for hours on the topic of GNOME and system integration,
but I'll try to shorten it to merely one hour: I no longer think it is
adequate for GNOME to say that we'll blindly run on any OS that exposes
a POSIXy API.  System integration is essential to the user experience.
To see an example of this, just try plugging in a piece of hardware into
a Linux desktop two years ago and try it on one from today and you'll
see one example of why it's so important.

I would like to see Sun take a leadership position on this, like they
did with accessibility and usability 6 years ago, but sadly they've been
notably absent from the various aspects of Project Utopia over the past
2+ years.

Ultimately, however, if system vendors and developers aren't willing to
hold up their end of the bargin, it doesn't serve anyone's interests --
especially the broader GNOME community's -- to stop driving innovation
in this area.

If I can make one suggestion to people/organizations/companies out there
who have an interest in the free desktop: get a system/kernel engineer
on your desktop team.  It's something I pushed heavily at Ximian for 4
years before we actually got one.  That is Mr. Love's role, and in that
time he's largely responsible for the uevent and inotify systems in the
Linux kernel.  Thanks to him, Kay Sievers and Greg KH, hotplug is sane
and scalable on Linux from the perspective of userspace.  At least in
the Linux world, kernel developers are generally pretty clueless about
the needs of userspace environments and apps, but once they understand
our perspective, they can really tackle our issues. :)

Joe