GDBus/GVariant plans for next GLib release

[David Zeuthen <david fubar dk>]

Hey,

At the GNOME summit this weekend, Ryan Lortie, Matthias Clasen, Will
Thompson and myself had a hallway conversation about how to get D-Bus
functionality into the GLib stack. This mail is an attempt at
summarizing what we talked about.

(Unfortunately the scheduled talk on the summit for GLib/D-Bus was on
Sunday where neither Matthias or I was available. Not ideal, but since
we reached consensus we asked Ryan to run that session and solicit
feedback.)

There was an amazing amount of consensus; here's a brief list

 - We want some level of usable D-Bus binding in the GLib stack for
   the next release. We also agreed to avoid doing any C specific
   type/object mapping as part of the first push.

 - Everyone would like to use GVariant as the message encapsulation
   data type (as opposed to e.g. DBusMessage or some wrapper around
   DBusMessage).

 - We want convenience/utility for the "hard" parts of D-Bus, e.g.
    - mainloop integration
    - async name owning
    - async name watching
    - async listening to signals
    - exporting objects

 - We want to use the GIO async patterns, e.g. GCancellable,
   GAsyncReadyCallback, GAsyncInitable. E.g. everything should
   be async and we should deter people from using sync calls
   by suffixing the _sync to the method name.

 - Ideally make insane things impossible to do

   - e.g. only allow proxy creation for unique bus names - not
     well-known names. Instead people will discover they should
     be using e.g. g_bus_watch_name().

 - Strong consensus about using libdbus-1 for stability/portability
   reasons. And also, so a single process can nicely interoperate with
   D-Bus bindings for other languages (e.g. shared object tree)

 - Hide the fact that we are using libdbus-1

 - Make the code available in libgio (perhaps with the libdbus-1 using
   implementation in e.g. GVfs through GIOExtensionPoint - maybe not
   worth the effort)

 - Ensure the provided code is thread-safe / can be used from threads
   - e.g. deliver replies/signals/method-invocation in the thread where
     the method was called from, signal was subcribed to etc.
   - this may^Wwill include fixing libdbus-1 bugs

Since the GDBus stuff I've been working on already does this most of
this, thanks to Havoc's laundry list

 http://lists.freedesktop.org/archives/dbus/2009-February/010988.html

in particular, we agreed that I should a) make it use GVariant; b) rip
out the C object mapping. The past few days, I've been working on this
and the work is now nearing completion.

See

 http://people.freedesktop.org/~david/gdbus-20091014/
 http://cgit.freedesktop.org/~david/gdbus-standalone/

for the current API and code. The best way to see how the code works is
to read the API docs and look at the examples

http://cgit.freedesktop.org/~david/gdbus-standalone/plain/gdbus/example-own-name.c
http://cgit.freedesktop.org/~david/gdbus-standalone/plain/gdbus/example-watch-name.c
http://cgit.freedesktop.org/~david/gdbus-standalone/plain/gdbus/example-watch-proxy.c
http://cgit.freedesktop.org/~david/gdbus-standalone/plain/gdbus/example-server.c

and also the test cases

 http://cgit.freedesktop.org/~david/gdbus-standalone/tree/gdbus/tests

This API isn't yet complete - the main outstanding items include

 - D-Bus peer connections

   - The obvious thing here is to do a GDBusServer (wrapping DBusServer)
     class and add a constructor to GDBusConnection that takes a D-Bus
     address.

   - It might be nicer to just pass a GIOStream and use the new
     socket functionality in GIO. This requires some (perhaps trivial)
     libdbus-1 changes. We can always add this and we probably
     want the former *anyway*.

 - Misc convenience API

   - It might be nice to have methods on GDBusProxy to invoke
     methods. Might be nice to have also have varargs methods on
     GDBusProxy that use the GTypeSerializer functionality of
     GVariant.

   - Various obvious stuff (e.g. g_bus_watch_name_on_connection())

There's also a couple of bugs in libdbus-1, mostly related to threading,
that needs fixing. I will file bugs and get them fixed.

Some notes about GDBus in general

 - The API and code is minimal on purpose and has an extensive
   test-suite (12K LOC, three new classes, 3.5K LOC tests).

   I believe, even with the current code, that nice C type mappings
   (like those generated by EggDBus, e.g. see
   http://people.freedesktop.org/~david/eggdbus-20091014/eggdbus-eggdbusbus.html
   http://people.freedesktop.org/~david/eggdbus-20091014/tests-testfrob.html
   for example (yes, this is generated code with comments)) are
   possible to do.

   The API does not expose a GDBusMessage type since that would cause
   unwanted API that would assist in using the API as a whole in a
   wrong way (e.g. exposing on-wire details like message headers
   can be nasty). Instead, the API is based on invoking/handling methods
   and receiving/emitting signals.

   While it might be possible that people use D-Bus in ways this API
   doesn't cover, I think it is the right decision to cut the cake
   this way.

 - Some people have mentioned the possibility to port e.g. dbus-glib
   to run on top of this code. I am opposed to that idea for at least
   two reasons

   - first of all, that would mean exposing libdbus-1 and adding other
     API that might be unwanted. While this could be done in a non-invasive
     manner I don't think it's worth the effort.

   - similarly, we didn't port gnome-vfs2 to sit on top of GIO. I might
     be comparing apples and oranges here... but don't underestimate how
     quickly people ported apps to GIO.

   That said, if we can add/tweak API (within reason) to make the
   transition easier for people I'm all for it. But it should not be
   our main goal.

 - I decided to represent wire-errors (e.g. DBusError) using GError
   since that type is widely used both in C and language bindings. There
   are facilities for recovering the D-Bus error name. We also support
   transparent passing of any GError (a feature used in e.g. GVfs).

 - The libdbus-1 on Win32 / OS X story. Gah.

   So either we a) restrict this to UNIX (put headers in gio-unix-2.0)
   or; b) we just tell people they need to use e.g. WinDBus on Win32,
   D-Bus from the Fink ports on OS X or whatever.

   Anyway, I handwavely suggest the latter and also just getting
   mainline libdbus-1 to work on Win32 and OS X. Havoc says it's a week
   worth of work or something. It might be more, it might be less, I
   don't know.

   (Right now I'm tempted just to do the libdbus-1/Win32 myself...)

Some notes on GVariant

 - GVariant is very pleasant to use

 - GVariant needs to be LGPL2 (I believe Ryan has fixed this already)

 - GVariant doesn't use en_US spelling. I think this is a deal-breaker,
   it needs to read e.g. serialize instead of serialise. Anyway, I'm
   da_DK myself but...

 - GVariant is a superset of a subset of the D-Bus protocol (!)

   1. GVariant is a superset because it handles e.g. 'maybe' types

   2. GVariant doesn't yet handle the new 'h' type (e.g. unix fd)

   Now, 1. has been discussed extensively on this list. I don't
   have any position on it, really. I guess it would be nice to
   either get the GVariant type extensions into the D-Bus protocol
   or disable them before merging GVariant into GLib. I don't know
   how that would affect other GVariant users.

   We probably need to make GVariant support the 'h' type. I guess
   it shouldn't be too much work but the devil is in the details.
   Ryan?

 - GVariant is not protocol compatible with the D-Bus protocol. This
   is not a deal-breaker, in my view, but it means there's a
   encoding/decoding step involved. 

   Ryan assures me that the current D-Bus protocol encoder/decoder (I
   lifted the one from DConf, did some fixes - see gdbusconversion.c
   in the GDBus tree) can be optimized - do some binary thing or
   whatever. Personally I don't care too much - if you have a lot
   of throughput, you probably shouldn't use D-Bus (use e.g. sockets
   instead) - but YMMV.

 - GVariant is very pleasant to use
   - Good job Ryan!

Gosh, that was a long mail. Sorry about that. To add insult to injury,
I'd like to ask if people can reply to separate parts of the mail using
separate mails rewriting the Subject line as needed. Thanks!

Thanks,
David