GLib type system

[Tim Janik <timj gtk org>]

hey all,

pretty much since i returned back from GUADEC, i've only been spending
my time on imlpementing the new GLib type system which i just finished
this morning. it then took me a good three hours to port BSE over to use
the GLib type system instead of the BSE one, half of that was a huge
sed job.
for the ones interested in details, the new type system is sitting in
the beast module under bse/glib-gtype.[hc].
(don't confuse this with the upcoming GObject system or the GParam stuff,
i'm only talking g_type_* currently).
for this mail, i want to unveil some of the new API bits and compare them
to the type system in Gtk+ which has the best chance of people being
familliar with it.

the key features of the new system are:
- derived types are completely dynmaic, i.e. can be implemented by
  plugins that get unloaded/reloaded on demand
- we have multiple interfaces for object-alike types, fully dynamic
  as well
- we have fundamental type id "branches", that is, sequential ids are
  now handed out per fundamental type. with this we can have compile
  time definitions of certain derived type ids (e.g. for GTK_TYPE_WIDGET,
  GTK_TYPE_CONTAINER, etc. for general speedups)
- support for object hirarchies is generic which means that based
  on the new type system, anyone can implement his own object hirarchy
  that doesn't need to start out with GObject, GtkObject or BseObject.
  the type system calls such types "instantiatable", since objects are
  instances of a certain class.


the plugin support
==================

in order for the type system to demand-load and unload plugins, a very
simple plugin interface has been established, it is really only that:
a _programming interface_, so the type system just knows about a plugin
pointer and methods it can apply to it:

typedef void (*GTypePluginRef)               (GTypePlugin    *plugin);
typedef void (*GTypePluginUnRef)             (GTypePlugin    *plugin);
typedef void (*GTypePluginFillTypeInfo)      (GTypePlugin    *plugin,
                                              GType           g_type,
                                              GTypeInfo      *info);
typedef void (*GTypePluginFillInterfaceInfo) (GTypePlugin    *plugin,
                                              GType           interface_type,
                                              GType           instance_type,
                                              GInterfaceInfo *info);
struct _GTypePlugin
{
  GTypePluginVTable     *vtable;
};
struct _GTypePluginVTable
{
  GTypePluginRef                plugin_ref;
  GTypePluginUnRef              plugin_unref;
  GTypePluginFillTypeInfo       complete_type_info;
  GTypePluginFillInterfaceInfo  complete_interface_info;
};

once a certain type has to be instantiated, the type system will call
plugin_ref (plugin) and complete_type_info (plugin, type, &type_info).
the later function then has the duty to fill in the type_info structure.
after all instances of such a type are destroyed, plugin_unref (plugin)
will be called. note, that the plugin poiner has to remain valid, since
the same procedure is repeated if a new instance of type is created.
also, multiple calls to plugin_ref() may occour before plugin_unref()
is called, so the plugin implementation has to provide some means of
a reference counting system (if g_module_*() is used for the actuall
implementation, it does reference counting for g_module_open() and
g_module_close() out of the box already).
complete_type_info() is the function that fills in the type info
structure, i.e. class_size, class_init, etc. functions for
instantiatable and classed types.
complete_interface_info() has to fill in the bits, necessary to
implement a specific interface for a certain instantiatable type,
analogous to complete_type_info().
a plugin that is only used for instantiatable types doesn't need
to provide complete_interface_info(), similarly, a plugin that is
only used for interface implementations doesn't need to provide
complete_type_info().


type registration
=================

fundamental types are always static, derived types and interface
implementations can be static or dynamic, thus the new API:

GType g_type_register_static       (GType                       parent_type,
                                    const gchar                *type_name,
                                    const GTypeInfo            *info);
GType g_type_register_dynamic      (GType                       parent_type,
                                    const gchar                *type_name,
                                    GTypePlugin                *plugin);
GType g_type_register_fundamental  (GType                       type_id,
                                    const gchar                *type_name,
                                    const GTypeFundamentalInfo *finfo,
                                    const GTypeInfo            *info);

g_type_register_static() is pretty analogous to gtk_type_unique(), just
the type name moved out of the type_info structure.
g_type_register_dynamic() uses plugin to retrive the type_info at a
later point, as outlined above.
g_type_register_fundamental() registers new fundamental types, type_id
may be anything from 3 to 255, however, certain type ids are reserved
for certain purposes, use of them is discouraged:

/* Predefined Fundamentals
 */
typedef enum
{
  /* standard types, introduced by g_type_init() */
  G_TYPE_INVALID,
  G_TYPE_NONE,
  G_TYPE_INTERFACE,

  /* GLib type ids */
  G_TYPE_ENUM,
  G_TYPE_FLAGS,
  G_TYPE_PARAM,
  G_TYPE_OBJECT,

  /* reserved type ids, mail gtk-devel-list@redhat.com for reservations */
  G_TYPE_BSE_PROCEDURE,
  G_TYPE_GLE_GOBJECT,

  /* the following reserved ids should vanish soon */
  G_TYPE_GTK_OBJECT,
  G_TYPE_GTK_ARG,
  G_TYPE_BSE_OBJECT,
  G_TYPE_BSE_ENUM,
  G_TYPE_BSE_FLAGS,
  G_TYPE_BSE_PARAM,

  G_TYPE_LAST_RESERVED_FUNDAMENTAL
} GTypeFundamentals;

in the final implementation, g_type_init() will probably call the
initialization functions for G_TYPE_ENUM, G_TYPE_FLAGS, G_TYPE_PARAM
and G_TYPE_OBJECT right away. libraries/applications that want to
register new (non-reserved) fundamental types can do so with
G_TYPE_LAST_RESERVED_FUNDAMENTAL+. to avoid clashes, the next free
fundamental id can be retrived with G_TYPE_FUNDAMENTAL_LAST (i might
make that an actuall function, so it looks less constant). besides
the normal type_info structure, fundamentals have to provide a
fundamental info structure as well, which looks like:

typedef enum
{
  G_TYPE_FLAG_CLASSED           = (1 << 0),
  G_TYPE_FLAG_INSTANTIATABLE    = (1 << 1),
  G_TYPE_FLAG_DERIVABLE         = (1 << 2),
  G_TYPE_FLAG_DEEP_DERIVABLE    = (1 << 3)
} GTypeFlags;
struct _GTypeFundamentalInfo
{
  GTypeFlags          type_flags;
  guint               n_collect_bytes;
  GTypeParamCollector param_collector;
};

the implementation of n_collect_bytes and param_collector usage is still
standing out, it'll come with the parameter collector for varargs of
the GParam implementation. implementation of a param_collector is
optional for new fundamental types. the type_flags member is actually
more interesting here, it may contain a combination of the GTypeFlags 
values, where
G_TYPE_FLAG_CLASSED
	indicates a classed type, that means that the type
        system can create (and destroy) one class structure
        for this type.
G_TYPE_FLAG_INSTANTIATABLE
	indicates an instantiatable type, so the type system can
	create (and destroy) one class of this type, as well as
	multiple instances. specification of G_TYPE_FLAG_INSTANTIATABLE
	without G_TYPE_FLAG_CLASSED is an error.
G_TYPE_FLAG_DERIVABLE
	means that new types can be derived from this fundamental type,
	i.e. allowing for a flat type hirarchy.
G_TYPE_FLAG_DEEP_DERIVABLE
	means that new types can be derived from derived types of
	this fundamental type, up to unlimited depth.
	specification of G_TYPE_FLAG_DEEP_DERIVABLE without
	G_TYPE_FLAG_DERIVABLE is not usefull.

after an interface type has been registered (say GtkRadio), actuall
implementations can be added to existing instantiatable types (e.g.
GtkCheckButton and GtkMenuItem) with:

void  g_type_add_interface_static  (GType                       instance_type,
                                    GType                       interface_type,
                                    GInterfaceInfo             *info);
void  g_type_add_interface_dynamic (GType                       instance_type,
                                    GType                       interface_type,
                                    GTypePlugin                *plugin);

usage is analogous to the type registration functions.

on to the new GTypeInfo structure, it is meant to cover the needs
of instantiatable, classed and interface types:

typedef void   (*GBaseInitFunc)         (gpointer       g_class);
typedef void   (*GBaseDestroyFunc)      (gpointer       g_class);
typedef void   (*GClassInitFunc)        (gpointer       g_class,
                                         gpointer       class_data);
typedef void   (*GClassDestroyFunc)     (gpointer       g_class,
                                         gpointer       class_data);
typedef void   (*GInstanceInitFunc)     (GTypeInstance *instance,
                                         gpointer       g_class);
struct _GTypeInfo
{
  /* interface types, classed types, instantiated types */
  guint16               class_size;

  GBaseInitFunc         base_init;
  GBaseDestroyFunc      base_destroy;

  /* classed types, instantiated types */
  GClassInitFunc        class_init;
  GClassDestroyFunc     class_destroy;
  gconstpointer         class_data;

  /* instantiated types */
  guint16               instance_size;
  guint16               n_preallocs;
  GInstanceInitFunc     instance_init;
};

class_size indicates the interface vtable size for interface types,
and the class size for classed and instantiatable types.
base_init() and base_destroy() will be executed each time a class
or interface of this type or of derived types is created. this is
usefull to override values in the class structure that got copied
over from its parent class.
note that providing an implementation of base_destroy() is usefull
even for static types, it will be executed if classes of derived
dynamic types are destroyed.
class_init() and class_destroy() will be executed only for
creation/destruction of this very type's class. specification of
class_destroy() for static types is an error.
both functions get a class_data pointer passed to them, this is
mainly intended for enumeration classes that will share the same
class initialization function for different enumerations.
instance_size indicates the size of the instance structures for
this type which will be initialized with a call to instance_init().
n_preallocs is a hint to the type system to optimize allocations
for instances of this type, powers of 2 are a good choice for
types that often carry many instances (e.g. 16 for GtkLabel).
the g_class pointer for instance_init() points to the real class
of the instance that is to be initialized, this is so that
instance_init() functions of parent types that initialize a
child type's instance still have access to the instance's real
type.

to round up type registration, here's the GInterfaceInfo structure:

typedef void   (*GInterfaceInitFunc)    (gpointer       g_iface,
                                         gpointer       iface_data);
typedef void   (*GInterfaceDestroyFunc) (gpointer       g_iface,
                                         gpointer       iface_data);
struct _GInterfaceInfo
{
  GInterfaceInitFunc    interface_init;
  GInterfaceDestroyFunc interface_destroy;
  gpointer              interface_data;
};

interfaces of an instantiatable type will be initialized as soon
as the type's class is created, they will be destructed shortly
before the type's class is destructed. invocation of
interface_init() and interface_destroy() with interface_data
is analogous to class_init() and class_destroy() with class_data
for classes. specification of interface_destroy() for interfaces
that are added statically to an instantiatable type is an error.


new type functions
==================

similarl to

gboolean g_type_is_a (GType type,
                      GType is_a_type);

which checks identification and anchestry of a type (just like
gtk_type_is_a() does),

gboolean g_type_conforms_to (GType type,
                             GType iface_type);

checks whether an interface type iface_type has been added to
an instantiatable type. thus, type "conforms to" iface_type.

type names are now stored as quarks internally, access to the
quark is provided with:

GQuark g_type_qname (GType type);

since sequential ids now increment per fundamental type branch,
a new function is supplied to retrive the next free sequential
id of a fundamental branch:

guint g_type_fundamental_branch_last (GType type);

to implement class destruction, the type system has to keep track
of how long a class is in use, so they are now reference counted:

gpointer g_type_class_ref               (GType     type);
void     g_type_class_unref             (gpointer  g_class);

classes that already exist and need to be retrived without incrementing
their reference count (e.g. the parent class of a class a reference is
being held on) can be "peeked" at with:

gpointer g_type_class_peek (GType type);

to remind programmers of doing clean reference counting for classes,
gtk_type_parent_class() now exists as:

gpointer g_type_class_peek_parent (gpointer g_class);

the new counterpart for gtk_type_children_types() to retrive the child
types of a given parent type is:

GType*   g_type_children (GType  type,
                          guint *n_children);

and a similar function exists to find out what interfaces an instantiatable
type conforms to:

GType*   g_type_interfaces (GType  type,
                            guint *n_interfaces);

to figure the next child type of a given parent type in a branch of
the type hirarchy, specified by the parent type and one of its descendants,
the following new function is supplied:

GType    g_type_next_base (GType type,
                           GType base_type);

static quarked data pointers can now be associated with registered type ids,
they will never get destructed, wich is why no destroy notifier is contained
in the new API:

void     g_type_set_qdata               (GType                   type,
                                         GQuark                  quark,
                                         gpointer                data);
gpointer g_type_get_qdata               (GType                   type,
                                         GQuark                  quark);

to show the actuall intend of these functions - e.g. BSE uses
them to associate a type blurb with each type it introduces.
internally, the type system may use this in the future for
specific data which is used only by a subset of the type nodes,
e.g. the plugin pointer.


Outstanding issues
==================

- GLib will have a seperated implementation for enum and flags types
  and will also provide a more flexible enum/flags parser that should
  be easily usable by applications (this is mostly done for BSE already).
- i still have to cook up an alternate interface for GtkTypeQuery,
  most probably there will be accessor functions for instance_size and
  class_size, with a comment noting that their sizes *may* change
  during runtime (this happens when registered dynamic types
  are recompiled during runtime and get reloaded, whether bindings
  actually want to acocunt for this is more than questionable).
- i don't see good ways of supporting gtk_type_set_chunk_alloc(), it
  might just die.
- i may take volountary submissions of ported gtk_type_describe_heritage()
  gtk_type_describe_tree(), but these functions will probably just also die.
  GLE and BSE already provide nice command line front-ends to query
  and print out type system branches which can be ported over in <20
  minutes to GLib (for bsequery specifically, that would only involve
  deleting the correct code portions).
- gtk_type_set_varargs_type() and gtk_type_get_varargs_type() will
  just die as well, AFAIK, only GLE uses them anyways. their
  functionality is to be superceeded by the GParamCollector.


Schedule
========

the next thing i want to get going is porting over BseEnum and
BseFLags types to GLib, after that, actually moving the new GType
stuff from BSE into GLib and porting Gtk+ over to the new type
system seems like a reasonable step.
once that transition has been performed, i'll probably come back
here with a proposed GObject implementation (still excluding
the GParam stuff though), and then attempt to get GtkObject and
BseObject going on top of that (not that we *have* to finish this
for Gtk+ 1.4 - the type system suports multiple instantiatable
hirarchies now).
and then? well, if we got that far before 1.4, we can see if
GParam, GSignal and GClosure (in this order necessarily) will
fit in a timely fashion.



ok, i think that's enough for now, eventhough i left all the casting
and checking macro stuff out, this email is already longer than the
new type system's header file ;)

as a secluding note, it shouldn't be too hard to port the Gtk+ 1.3
type system over to this, i did the same thing for BSE quick enough.
for Gtk there'll not even be a need for an sed job, most gtk_type_*
function will simply be wrappers around the g_type_* ones, preserving
85% source compatibility.

the bits that will be biting us are:
- bindings can't simply mirror the type ids in a one-dimensional
  array anymore, they either have to keep track of branch expansion,
  or simply use g_type_set_qdata() to store their own stuff.
  (AFAIK, only gnome-guile is actually affected by this)
- GtkArg type ids will not be fundamental types anymore, i still
  have to figure how great the impact on this one is.

Disclaimer:
though it's pretty early, i'm already up for quite some time, so
despite public expectation, one may find errors in this email ;)

---
ciaoTJ