GNOME ABI review

[Havoc Pennington <hp redhat com>]

Hi,

I'd like to propose a huge GNOME-wide effort over the next twelve
months.

In conversations with various people I've been raising the issue that
some of our ABIs simply aren't ready to be supported for 5, 10, 15
years. As the desktop really begins to get somewhere, and ISVs start
to port, we need to solve this problem. Sun raised the same issue at
the GUADEC advisory board meeting, citing some of the same ABIs that
I'm worried about.

By "ABI" I mean something very general, including file formats and
locations, protocols, and so forth. Anything that, when changed, would
break an application relying on it.

For example, the recent move from ~/.gnome-desktop to ~/Desktop
probably counts as an ABI change.  Changing how menus are constructed
probably counts.  As desktop Linux catches on, more and more people
will scream loudly when this kind of thing changes.

Not all ABIs need to be part of the platform; ABIs such as libwnck,
libegg, and so forth are not an issue. People have to go out of their
way to depend on these (defining -DABI_NOT_STABLE, forcing shared libs
to be built, etc.).

So some points - 

===
First, what should we be looking for in the stable ABI?

Here are some ideas to start the discussion:

1. It must allow apps to do everything they need to do to be
   good applications.

2. It should be a coherent whole; only one undeprecated facility for
   doing each task, each facility logically related to the others,
   etc.

3. It should allow alternative implementations to integrate properly
   with the desktop. This includes an ABI-incompatible GTK+ 3.0 if we
   ever do that; Swing; WINE; OpenOffice; or whatever.

4. It should be recommendable and actively maintained. If someone says
   "this API sucks" we should say "we're working on improving it" or
   "here is how to use it" but not "yeah everyone knows that library
   is broken, you are dumb for using it"

   If the core GNOME developers avoid the ABI, that's a good
   indication that we're wrong to suggest it to others.

5. It should be extensible and allow us to make UI progress.  Should
   not hardcode the current desktop's UI any more than required.

6. It should be sufficiently well-defined that as we iterate and
   improve it, we can make a reasonably educated guess about what will
   and won't break applications.

7. It should be in encapsulated chunks when possible so we can
   deprecate and replace one bit without having to deprecate and
   replace everything that depends on it. Implementation details 
   should be kept as just that.
  
   (To be resolved: is it going to fly to do the Pango trick 
   with Xft1 -> Xft2, where we kept the dependency out of DT_NEEDED
   and later changed the dependency?)

8. It should be standards-compliant. If we need to break the 
   ABI to conform to standards (or create a standard that 
   needs to exist) then that will suck.

===
Second, what is the timeframe for addressing ABI issues?  Nat suggests
that GNOME 2.6, maybe 2.8 will be the latest time we can fix these
things.  That's about a year from now!

The real answer isn't an exact date but "when we really start to get
serious ISVs and large deployments" - which we at least hope will be
soon, and could come at any time.

This is sooner than many have been thinking. We aren't talking about
GNOME 3.0. We need to fix the ABI without breaking it.

===
Third, how would we deploy ABI fixes without breaking the ABI?

I believe we would do it as follows:

 - introduce fixed ABIs in parallel, keeping current ABI working

 - do not try to mass-port GNOME itself to new ABIs
   (remember the dogfood rule and the time-based release)

 - fudge it for things like ~/Desktop, because we can 
   get away with that for now

 - keep new ABIs as small and simple as we can

Improved ABIs should not be half-baked, overengineered, or needless
rewrites. However, replacing some things that work for us now may be
needed.

===
Fourth, what does it mean to "support" an ABI?

If we leave in a suboptimal ABI, what problems does that cause for us? 

We have to keep it working; that may make it exceptionally difficult
and labor-intensive to fix UI or architectural issues, or may stick us
with an unfortunate amount of bloat.

Some of this is inevitable since software is just like that. But right
now we're overextended and some damage control is in order.  Many
issues are well-known and just waiting to be fixed.

This is of special concern to the OS vendors, but getting stuck with
bad ABIs will impact everyone hacking on GNOME who has to work within
the confines of the historical baggage.

===
Fifth, isn't this focus on platform instead of UI?

Yes it is. However, I think the platform has come back to the top of
the priority queue, or at least parts of it have. GNOME 2.6/2.8 are
looking reasonable UI-wise, but our broken ABIs are going to make it
hard to evolve the UI in the future if we don't address them.

This is not glamorous platform-building, it's cleaning up
messes. Though incidentally, some cleanup and de-bloat should make it
a lot easier to evolve the platform in the future.


===
Sixth, isn't it a criticism of GNOME to bring this up?

Yes, but keep in mind that GNOME is outstandingly good in this area
compared to nearly all other open source projects, and we do have a
solid core ABI. The issue here is loose ends, and raising our
already-high standards.

And we shouldn't have to break the ABI we have now in the process of
future-safing our platform.

It's a stronger criticism of GNOME if we can't recognize the need 
to address these issues.


=== 
Finally, given my guidelines, what are some of the problematic
ABIs?

I'll make my list, some will inspire flames. This is intended to start
a discussion about where we're going. These are not in priority order.

The point is that we need to think hard about everything in the
platform.

1. Drag-and-drop/Cut-and-paste formats

 We need to catalog and document the DND/CNP datatypes and their
 details. This registry needs to be kept up-to-date as new types are
 added.

2. Menu system

 The vfolder format shipped in 2.2, 2.4 doesn't do what app developers
 want, has a broken implementation, and isn't compatible with the
 freedesktop.org spec.

 To consider this ABI issue solved we not only need to implement the
 new spec, but also create a conformance test suite for it so we don't
 end up having to support nonconformant behavior long-term.

3. Applets/System Tray Icons

 The current applet/tray-icon split just doesn't make any sense.
 Mark's attempt to clarify the difference and how to use a tray icon
 floundered on massive disagreement. This ABI clearly needs to change
 at some point if only to clarify the UI that results when you do 
 various things with your tray icon.

 In my opinion, applet ABI should be stripped of all dependencies and
 become a protocol specification built around XEMBED and some IPC
 mechanism (possibly just X events, but whatever is general). Then we
 can change our desktop architecture significantly, while keeping
 applets working, and let applets be implemented by lots of different
 toolkits.

4. GConf

 GConf could be reduced to a much smaller API with fewer dependencies,
 without a rewrite; we keep the implementation of "gconfd" since that
 doesn't export ABI, but introduce an alternative client side ABI with
 less random stuff in it and clearer semantics.

5. All UI/widget stuff above GTK+

 Many, many GNOME developers want to wholesale deprecate the UI stuff
 above the GTK+ level. This is nearly feasible with GTK+ 2.4, with
 only session management and possibly the menu system missing.  GTK+
 2.6 should make it fully feasible.

 We should not be letting people use something we know we're going to
 kill off. The flamewar needs to be had and the decision made.

6. Libraries that just aren't production quality

 libgnomecanvas, libart come to mind; they can work fine if you
 understand the pitfalls, but. For those two in particular we're also
 expecting Cairo/Xr to replace them.

7. MIME system

 App developers need to install their MIME types and it needs to
 work; the mechanism needs to be a standard across Linux desktops.
 
8. VFS

 gnome-vfs has a sprawling API with UNIX-looking semantics. However, 
 the backends don't conform to UNIX semantics (or really any semantics
 at all). Moreover, we make up URI schemes, and otherwise suck.

 The big ugly problem is that different apps (OpenOffice, etc.) 
 see different sets of usable URIs - this simply _must_ be solved
 in some way.

 A better gnome-vfs might have a very minimal required interface for
 each backend, such as just listing files and getting their contents
 in a single large block. Then additional interfaces discovered via a
 queryInterface()-style approach would add features such as file
 modification, streaming read/write, getting file permissions, file
 copy, or whatever.
 
 Then the ABI is extensible and we can know when a backend is breaking 
 the spec and when it isn't.

9. Online Help

 How do you install it and how do you display it? 
 Needs to be well-specified, well-documented, and kept working.

10. Nautilus views/extensions

 Are we happy with these interfaces? I'm not that familiar with them.


Undoubtedly we can think of more items that need a hard look. Please
correct my mistakes in the above list.

So that's my mega-troll of the month - but seriously, I hope everyone
will give it serious thought.

Havoc