Re: Serialization in libgobject

[Stefan Kost <ensonic hora-obscura de>]

Hi Tim,

thanks for the extensive comments and detailed concerns. Don't you think
it would be good to keep the buzzgilla issue open. Its an enhancement
request after all and there seems to be several people wanting to have
it. The bugzilla issue could serve as a tracker item. And now maybe some
more people know about thsi and add their thoughts and ideas.

@Andrew: is you implementation public? If so where can I find the
sources, if not can you make the serialisation part public?

Maybe we can find a SoC student for the next summer to look at the
solutions used in our application as come up with a good proposal that
suites most apps.

Stefan

Am Mittwoch, den 24.05.2006, 13:08 +0200 schrieb Tim Janik:
> Hi All.
> 
> the request for providing object and strcuture serialization/deserialization
> in libgobject has come up a number of times:
>    http://mail.gnome.org/archives/gtk-devel-list/2001-January/msg00128.html
>    http://mail.gnome.org/archives/gtk-devel-list/2004-October/msg00123.html
>    http://bugzilla.gnome.org/show_bug.cgi?id=157734
> and as part of the Buzztard (http://www.buzztard.org) project:
>    http://www.buzztard.org/index.php/Gobject_serialisation
> 
> basically, i don't think that GObject is going to get (de-)serialization
> functionality any time soon.
> 
> there are multiple reasons for this:
> a) the GType system is not self-contained enough to recursively (de-)serialize
>     nested structures and objects (e.g. POINTER or BOXED types).
> b) serialization requirements with regards to uniqueness, inter-object
>     references, error handling, etc. are highly application dependent.
>     to provide just a rough idea, i'll outline a number of issues that'll have
>     to be decided upon to tailor serialization to a specific application, which
>     usually contradicts other application scenarios.
> b.1) should the target stream contain binary or text representations of the
>       values?
>       - binary would be neccessary for remote-protocol implementations;
>       - text is neccessary to support human readable/editable files, as is the
>         case for config files or some project files.
> b.2) how are incompatibilities between object types/implementations and the
>       file format going to be handled?
>       - strict rejections of incompatibilities between file format and type
>         interfaces is not acceptable for storage of first-class application
>         data, e.g. project files.
>       - ignoring incompatible serialization bits or lax automated conversion
>         can incur critical data loss or produce bogus deserialzation results
>         for the user.
> b.3) how's file versioning going to be handled?
>       - file versioning is crucial for first-class application data, but can be
>         overkill for config files and may be an unaccaptable burden for remote
>         protocol implementations.
>       - also, with file versioning, converters may need to be implemented, and
>         an extra abstraction layer needs to be provided to disentangle file
>         versions from interface implementation versions. that in turn requires
>         extra application specific mediation code.
> b.4) how is data being handled that isn't reflected by the object property
>       interface?
>       somewhat related are cases where proper setting of object data requires
>       proeprties to be set in a certain order. this occours when property
>       setters have side effects apart from just storing a value in a data slot
>       (such as setting the number of cols/rows on a table).
> b.5) how is storage of defaulting properties handled?
>       - some properties may be important to not save when defaulting.
>       - some properties may be important to save regardless of defaulting.
> b.6) how are properties classified to distinguish between ones that are used
>       at the GUI, as programming interface or to reflect serializable object
>       state, and any combinations thereof?
>       he's an archive email partly addressing the issue:
>         http://mail.gnome.org/archives/gtk-devel-list/2001-June/msg00460.html
> b.7) how are object and structure pointers being handled?
>       - can/should they be saved by reference or recursively by value?
>       - and when restoring, factories and lookup mechanisms are needed to
>         resolve references. also, when restoring circular references on object
>         trees, properties can not anymore be restored in order.
> 
> these are some of the issues we ran into when developing serialization and
> deserialization of object trees and nested structures for Beast
> (http://beast.gtk.org) where GValue serialization is used for project files
> and a remote protocol implementations.
> 
> other implementations that should be inspected in this context are python
> pickling:
>    http://docs.python.org/lib/module-pickle.html
> and java serialization:
>    http://java.sun.com/developer/technicalArticles/Programming/serialization/
> both of which do a good job on *some* of the issues outlined above but fall
> short on others.
> 
> the border line is, serialization/deserialisation is far from being esily 
> solved generically. and any non-generic implementation of this should better
> make sure to define exact usage cases to cover, otherwise it will fall short
> on most applications and overdo central abstractions.
> 
> while the issues mentioned under (b) are mostly accountable for the complexity
> involved in the topic, (a) is the real killer for any serious attempt at object
> serialization in GLib. as a result, i think it's best for applications that
> have real serialization needs to implement the mechanisms on their own, tailored
> to their specific usage scnearios, like the GStreamer
> (http://gstreamer.freedesktop.org/), Buzztard (http://www.buzztard.org) and
> Beast projects have done it.
> 
> ---
> ciaoTJ
> _______________________________________________
> gtk-devel-list mailing list
> gtk-devel-list gnome org
> http://mail.gnome.org/mailman/listinfo/gtk-devel-list