Serialization in libgobject

[Tim Janik <timj imendio com>]

Hi All.

the request for providing object and strcuture serialization/deserialization
in libgobject has come up a number of times:
  http://mail.gnome.org/archives/gtk-devel-list/2001-January/msg00128.html
  http://mail.gnome.org/archives/gtk-devel-list/2004-October/msg00123.html
  http://bugzilla.gnome.org/show_bug.cgi?id=157734
and as part of the Buzztard (http://www.buzztard.org) project:
  http://www.buzztard.org/index.php/Gobject_serialisation

basically, i don't think that GObject is going to get (de-)serialization
functionality any time soon.

there are multiple reasons for this:
a) the GType system is not self-contained enough to recursively (de-)serialize
   nested structures and objects (e.g. POINTER or BOXED types).
b) serialization requirements with regards to uniqueness, inter-object
   references, error handling, etc. are highly application dependent.
   to provide just a rough idea, i'll outline a number of issues that'll have
   to be decided upon to tailor serialization to a specific application, which
   usually contradicts other application scenarios.
b.1) should the target stream contain binary or text representations of the
     values?
     - binary would be neccessary for remote-protocol implementations;
     - text is neccessary to support human readable/editable files, as is the
       case for config files or some project files.
b.2) how are incompatibilities between object types/implementations and the
     file format going to be handled?
     - strict rejections of incompatibilities between file format and type
       interfaces is not acceptable for storage of first-class application
       data, e.g. project files.
     - ignoring incompatible serialization bits or lax automated conversion
       can incur critical data loss or produce bogus deserialzation results
       for the user.
b.3) how's file versioning going to be handled?
     - file versioning is crucial for first-class application data, but can be
       overkill for config files and may be an unaccaptable burden for remote
       protocol implementations.
     - also, with file versioning, converters may need to be implemented, and
       an extra abstraction layer needs to be provided to disentangle file
       versions from interface implementation versions. that in turn requires
       extra application specific mediation code.
b.4) how is data being handled that isn't reflected by the object property
     interface?
     somewhat related are cases where proper setting of object data requires
     proeprties to be set in a certain order. this occours when property
     setters have side effects apart from just storing a value in a data slot
     (such as setting the number of cols/rows on a table).
b.5) how is storage of defaulting properties handled?
     - some properties may be important to not save when defaulting.
     - some properties may be important to save regardless of defaulting.
b.6) how are properties classified to distinguish between ones that are used
     at the GUI, as programming interface or to reflect serializable object
     state, and any combinations thereof?
     he's an archive email partly addressing the issue:
       http://mail.gnome.org/archives/gtk-devel-list/2001-June/msg00460.html
b.7) how are object and structure pointers being handled?
     - can/should they be saved by reference or recursively by value?
     - and when restoring, factories and lookup mechanisms are needed to
       resolve references. also, when restoring circular references on object
       trees, properties can not anymore be restored in order.

these are some of the issues we ran into when developing serialization and
deserialization of object trees and nested structures for Beast
(http://beast.gtk.org) where GValue serialization is used for project files
and a remote protocol implementations.

other implementations that should be inspected in this context are python
pickling:
  http://docs.python.org/lib/module-pickle.html
and java serialization:
  http://java.sun.com/developer/technicalArticles/Programming/serialization/
both of which do a good job on *some* of the issues outlined above but fall
short on others.

the border line is, serialization/deserialisation is far from being esily 
solved generically. and any non-generic implementation of this should better
make sure to define exact usage cases to cover, otherwise it will fall short
on most applications and overdo central abstractions.

while the issues mentioned under (b) are mostly accountable for the complexity
involved in the topic, (a) is the real killer for any serious attempt at object
serialization in GLib. as a result, i think it's best for applications that
have real serialization needs to implement the mechanisms on their own, tailored
to their specific usage scnearios, like the GStreamer
(http://gstreamer.freedesktop.org/), Buzztard (http://www.buzztard.org) and
Beast projects have done it.

---
ciaoTJ