Streams API [Was: Re: Plans for gnome-vfs replacement]

[Hans Petter Jansson <hpj novell com>]

On Wed, 2006-09-20 at 10:07 +0200, Alexander Larsson wrote:
> On Tue, 2006-09-19 at 16:24 -0400, Hans Petter Jansson wrote:

> > Of course, not all applications need such functionality, but that's
> > just a matter of wrapping this in a higher-level, more rigid API.

> In my reading of code using gnome-vfs i haven't seen a single use of
> anything like this. Could you perhaps describe some real-life examples
> of how/where this would be used?

Well, a peer-to-peer filesharing application is the most extreme example
I can think of. Such programs typically need encryption, rate control,
ad-hoc protocol implementations, lots of TCP and UDP sockets and in
decentralized networks, routing facilities. Now, we don't need to
provide all of this, especially not in glib, but having the basic
building blocks there would be nice.

Another example is instant messaging applications, which require
encryption (typically an SSL layer), rate control (so pasted text can be
sent slowly, lest you trigger the server's anti-flood filter) and ad-hoc
protocols. In fact, I think Gaim is a good example of how the platform's
lack of a good, low-level async I/O API leads to broken applications:

Gaim (at least version 1.5, which I inspected) has UI that blocks on
network I/O, poor error handling and convoluted protocol code. It also
features an interesting approach to "async" DNS lookups: Fork a child
process and do lookups there, using pipes to communicate with it.
Unfortunately, pipe communication was done using blocking posix calls,
so the end result was a main process that could still block on DNS
lookups.

Also, Evolution could use it.

It's possible that we're talking past each other here, by the way. I was
thinking of the streams API as lower-level than VFS, so not something
you'd see a need for perusing gnome-vfs consumers.

My mental image is:

[posix] <- [Async I/O] <- [Streams] <- [VFS]

> >From just reading your post, without deeper insight into what you mean I
> would be hesitant to add something like that to glib. Its not clear that
> applications need it, and its not clear the the solution we create is
> good enough for the applications that need something like it (in fact, I
> think its likely that such apps want to do their own thing).
> 
> [...]

I wasn't proposing it specifically for glib, at least not the
element-based stream model. I thought that was still up in the air.

I think we can revisit this later, as the next part is more important to
me anyway:

> > Also, what's your thinking on supporting asynchronous DNS lookups,
> > representing IP addresses, TCP/UDP sockets, etc? We currently don't
> > have any clear platform solution for that.

> These are interesting questions, and some applications would surely
> benefit from such code (although others will consider it bloat). I don't
> want to intermix these questions with the vfs layer though. They are not
> really strongly related, only TCP sockets are slightly related to
> streams, and not really in a very interesting way (its just another
> possible stream implementation).

If you're going to keep implementations of network protocols in glib
(which I assume you'd have to for some VFS modules), you'll have to
implement this anyway. Why not make it available to applications?

I'm making the assumption that you'll keep VFS modules in glib because
you mentioned it would have "odd dependencies" earlier.

-- 
Hans Petter

        "...you cut a piece, and as you're taking it off of the fork the
        main piece of the filet may be floating up out of the plate, but
        you stab it with your fork and put it back down in, and if you
        do it carefully, the surface tension of the gravy will keep it
        in place."
                                                   - Russell Schweickart