Re: Introspection API

I have written up a draft spec for the format of the binary metadata
to back up the repository api which I posted earlier. I hope to have
some initial code implementing both of these soon.

Comments highly appreciated,


GObject binary metadata

GObject metadata is stored in a variable with the name _G_METADATA in the shared 
object it describes. The format of the data is strongly influenced by the Mozilla
XPCOM format. 

Some of the differences to XPCOM include:
- type information is stored not quite as compactly (XPCOM stores it inline in 
  function descriptions in variable-sized blobs of 1 to n bytes. We store 16 bits
  of type information for each parameter, which is enough to encode simple types
  inline. Complex (e.g. recursive) types are stored out of line in a separate
  list of types.
- String data is stored in a string pool, string references are stored as 
  offsets relative to the start of the metadata.


The metadata has the following general format.

metadata ::= header, directory, interfaces, types, annotations, string pool

directory ::= list of entries

entry ::= interface type, name, namespace, offset

interface ::= function|callback|struct|value|object|interface|enum|flags|boxed

annotations ::= list of annotations, sorted by offset 

annotation ::= offset, key, value


We describe the fragments that make up the metadata in the form of C structs (although 
some fall short of being valid C structs since they contain multiple flexible arrays).

Header (48 bytes)

struct Header 
  gchar[16] magic;
  guint8    major_version;
  guint8    minor_version;
  guint16   reserved;

  guint16   n_entries;
  guint16   n_local_entries;
  guint32   directory;
  guint32   types;
  guint32   annotations;
  guint32   string_pool;

  guint32   size;
  guint32   namespace;

magic:    The string "GOBJ\nMETADATA\r\n\032". This was inspired by XPCOM, 
          which in turn borrowed from PNG.

          The version of the metadata format. Minor version changes indicate 
          compatible changes and should still allow the metadata to be parsed 
          by a parser designed for the same major_version.
          The number of entries in the directory. 

	  The number of entries referring to interfaces in this blob. The
	  local entries must occur before the unresolved entries.

          Offset of the directory in the metadata. 
          FIXME: need to specify if and how the directory is sorted

types:    Offset of the list of types in the metadata.

          Offset of the list of annotations in the metadata.

          Offset of the string pool in the metadata. The strings are stored 
          as zero-terminated strings

size:     The size of the metadata.

          Offset of the namespace in the string pool. 

Directory entry (16 bytes)

struct DirectoryEntry 
  guint32 type;
  guint32 name;
  guint32 namespace;
  guint32 interface;

type:     The type of interface this entry points to:
          1 function
          2 callback
          3 struct
          4 object
          5 interface
          6 enum
          7 flags
          8 boxed
          9 value

name:     The name of the entry.

          For references to interfaces which are in a different metadata blob,
          this contains the namespace to look in. For references to interfaces 
          in the same blob, it is 0.

          For references to interfaces in the same blob, this is the offset of 
          the interface in the metadata. For references to interfaces which are 
          in a different blob, this is 0.

All interfaces pointed to by directory entries start with the same layout for 
the first 8 bytes (the reserved flags may be used by some interface types)

struct InterfacePrefix {
  guint16 type; 
  guint   deprecated : 1;
  guint   reserved   :15;
  guint32 name;

type:     An integer specifying the type of interface, see DirectoryEntry 
          for details

          Wether the interface is deprecated

name:     The name of the interface.

The SignatureBlob is shared between Functions, 
Callbacks, Signals and VirtualFunctions.

SignatureBlob (4 + 8 * n_arguments bytes)

struct SignatureBlob 
  guint16           return_type;

  guint             may_return_null              : 1;
  guint             caller_owns_return_value     : 1;
  guint             caller_owns_return_container : 1;
  guint             reserved                     : 5;

  guint8            n_arguments;
  ArgBlob[]         arguments;

          Describes the type of the return value. See details below for TypeBlob.

          Only relevant for pointer types. Indicates whether the caller
          must expect NULL as a return value.

          If set, the caller is responsible for freeing the return value
          if it is no longer needed.

          This flag is only relevant if the return type is a container type.
          If the flag is set, the caller is resonsible for freeing the container,
          but not its contents
          The number of arguments that this function expects, also the length 
          of the array of ArgBlobs

          An array of ArgBlob for the arguments of the function.

FunctionBlob (12 + x bytes)

struct FunctionBlob {
  guint16       type;  /* 1 */

  guint         deprecated     : 1;
  guint         is_setter      : 1; 
  guint         is_getter      : 1;
  guint         is_constructor : 1;
  guint         reserved       : 2;
  guint         property       :10;

  guint32       name;
  guint32       c_name;
  SignatureBlob signature;

c_name:   The symbol which can be used to obtain the function pointer with dlsym().

          The function is deprecated.

          The function is a setter for a property. Language bindings may prefer to 
          not bind individual setters and rely on the generic g_object_set().

          The function is a getter for a property. Language bindings may prefer to 
          not bind individual getters and rely on the generic g_object_get().

          The function acts as a constructor for the object it is contained in. 

          Index of the property that this function is a setter or getter of in the
          array of properties of the containing interface

          Describes the parameter types and the return value type.

CallbackBlob (8 + x bytes)

struct CallbackBlob 
  guint16       type;  /* 2 */

  guint         deprecated : 1;
  guint         reserved   :15;

  guint32       name;
  SignatureBlob signature;

          Describes the parameter types and the return value type.

ArgumentBlob (8 bytes)

struct ArgBlob 
  guint32 name;

  guint   in                      : 1;
  guint   out                     : 1;
  guint   is_return_value         : 1;
  guint   null_ok                 : 1;
  guint   receiver_owns_value     : 1;
  guint   receiver_owns_container : 1;
  guint   reserved                :10:

  guint16 arg_type;

name:     A suggested name for the parameter. 

in:       The parameter is an input to the function

out:      The parameter is used to return an output of the function. 
          Parameters can be both in and out. Out parameters implicitly 
          add another level of indirection to the parameter type. Ie if 
          the type is uint32 in an out parameter, the function actually 
          takes an uint32*.

          The parameter should be considered the return value of the function. 
          Only out parameters can be marked as return value, and there can be 
          at most one per function call. If an out parameter is marked as 
          return value, the actual return value of the function should be 
          either void or a boolean indicating the success of the call.
null_ok:  Only meaningful for types which are passed as pointers.
          For an in parameter, indicates if it is ok to pass NULL in, for 
          an out parameter, wether it may return NULL. Note that NULL is a 
          valid GList and GSList value, thus null_ok will normally be set for 
          parameters of these types.

          For an in parameter, indicates that the function takes over ownership 
          of the parameter value. For an out parameter, it indicates that the 
          caller is responsible for freeing the return value.

          For container types, indicates that the ownership of the container, 
          but not of its contents is transferred. This is typically the case 
          for out parameters returning lists of statically allocated things.

          Describes the type of the parameter. See details below.

Types are specified by two bytes. If the high byte is zero, the low byte 
describes a basic type, otherwise the 16bit number is an offset relative to 
header->types and points to a TypeBlob. 

SimpleTypeBlob (2 bytes)

union SimpleTypeBlob 
    guint8 reserved;  /* 0 */

    guint  is_pointer :1;
    guint  reserved   :2;
    guint  tag        :5;
  guint16 offset;

          indicates wether the type is passed by reference. 

tag:      specifies what kind of type is described, as follows:
          0  void
          1  boolean (booleans are passed as ints)
          2  int8
          3  uint8
          4  int16
          5  uint16
          6  int32
          7  uint32
          8  int32
          9 int64
         10 uint64
         11 float
         12 double
         13 string  (these are zero-terminated char* and assumed to be 
                     in UTF-8, for other data, use uint8[])
         14 GString 

         For string and GString, is_pointer will always be set.

offset:  Offset relative to header->types that points to a TypeBlob. 

TypeBlob (4 or more bytes)

union TypeBlob
  ArrayTypeBlob        array_type;
  InterfaceTypeBlob    interface_type;
  OneParameterTypeBlob one_parameter_type;  
  TwoParameterTypeBlob two_parameter_type;  
  ErrorTypeBlob        error_type;  

ArrayTypeBlob (4 bytes)

Arrays have a tag value of 20. They are passed by reference, thus is_pointer is 
always 1.

struct ArrayTypeBlob 
  guint          is_pointer      :1; /* 1 */
  guint          reserved        :2;
  guint          tag             :5; /* 20 */
  guint          zero_terminated :1;
  guint          has_length      :1;
  guint          length          :6;  

  SimpleTypeBlob type;

          Indicates that the array must be terminated by a suitable NULL value. 

          Indicates that length points to a parameter specifying the length of the
          array. If both has_length and zero_terminated are set, the convention 
          is to pass -1 for the length if the array is zero-terminated. 
          FIXME: what does this mean for types of field and properties ?

length:   The index of the parameter which is used to pass the length of the 
          array. The parameter must be an integer type and have the same direction 
          as this one. 

type:     The type of the array elements.

InterfaceTypeBlob (4 bytes)

struct InterfaceTypeBlob 
  guint   is_pointer :1; 
  guint   reserved   :2;
  guint   tag        :5; /* 21 */
  guint8  reserved;

  guint16 interface;

Types which are described by an entry in the metadata have a tag value of 21. 
If the interface is an enum of flags type, is_pointer is 0, otherwise it is 1.

          Index of the directory entry for the interface.

OneParameterTypeBlob (4 bytes)

GLists have a tag value of 22, GSLists have a tag value of 23. They are passed 
by reference, thus is_pointer is always 1.

struct OneParameterTypeBlob 
  guint          is_pointer :1; /* 1 */
  guint          reserved   :2;
  guint          tag        :5; /* 22 or 23 */
  guint8         reserved;

  SimpleTypeBlob type;

type:     Describes the type of the list elements.

TwoParameterTypeBlob (8 bytes)

GHashTables have a tag value of 24. They are passed by reference, thus is_pointer 
is always 1.

struct TwoParameterTypeBlob 
  guint          is_pointer :1; /* 1 */
  guint          reserved   :2;
  guint          tag        :5; /* 24 */
  guint8         reserved;

  guint16        reserved;

  SimpleTypeBlob type1;
  SimpleTypeBlob type2;

type1:    Describes the type of the keys in the table. This will most commonly be 
          int or string.

type2:    Describes the type of the value in the table. This may be void*.

ErrorTypeBlob (4 + 4 * n_domains bytes)

struct ErrorTypeBlob
  guint           is_pointer :1; /* 1 */
  guint           reserved   :2;
  guint           tag        :5; /* 25 */
  guint8          reserved;

  guint16         n_domains;

  guint32         domains[];

          The number of domains to follow

domains:  Offsets of ErrorDomainBlobs describing the possible error domains.

ErrorDomainBlob (10 bytes)

struct ErrorDomainBlob
  guint32        name;
  guint32        get_quark;
  guint16        error_codes;

name:     The name of the error domain

          The symbol name of the function which must be called to obtain the 
          GQuark for the error domain.

          Index of the InterfaceBlob describing the enumeration which lists
          the possible error codes.

PropertyBlob (8 bytes)

struct PropertyBlob
  guint32        name;

  guint          deprecated     :1;
  guint          readable       :1;
  guint          writable       :1;
  guint          construct      :1;
  guint          construct_only :1;
  guint          reserved       :11;

  SimpleTypeBlob type;

name:     The name of the property. 

          The ParamFlags used when registering the property.

type:     Describes the type of the property.

SignalBlob (8 + x bytes)

struct SignalBlob 
  guint32       name;

  guint         deprecated        : 1;
  guint         run_first         : 1;
  guint         run_last          : 1;
  guint         run_cleanup       : 1;
  guint         no_recurse        : 1;
  guint         detailed          : 1;
  guint         action            : 1;
  guint         no_hooks          : 1;
  guint         has_class_closure : 1;
  guint         reserved          : 6;

  guint16       class_closure;
  SignatureBlob signature; 

name:     The name of the signal.

no_hooks: The flags used when registering the signal.

          Set if the signal has a class closure.

          The index of the class closure in the list of virtual functions
          of the interface on which the signal is defined.

          Describes the parameter types and the return value type.

VirtualFunctionBlob (8 + x bytes)

struct VirtualFunctionBlob 
  guint32       name;

  guint         must_chain_up           : 1;
  guint         must_be_implemented     : 1;
  guint         must_not_be_implemented : 1;
  guint         is_class_closure        : 1;
  guint         reserved                :12;

  guint16       signal;
  SignatureBlob signature;

name:     The name of the virtual function.

          If set, every implementation of this virtual function must
          chain up to the implementation of the parent class. 

          If set, every derived class must override this virtual function.

          If set, derived class must not override this virtual function.

          Set if this virtual function is the class closure of a signal.

          The index of the signal in the list of signals of the interface 
          to which this virtual function belongs.

          The signature of the virtual function.

FieldBlob (8 bytes)

struct FieldBlob 
  guint32         name;

  guint           readable : 1; 
  guint           writable : 1;
  guint           reserved : 6;
  guint8          bits;

  SimpleTypeBlob type;

name:     The name of the field.

writable: How the field may be accessed.

bits:     If this field is part of a bitfield, the number of bits which it
          uses, otherwise 0.

type:     The type of the field.

ValueBlob (32 bytes)

Values commonly occur in enums and flags, but we also allow them to occur
in interfaces or freestanding, to describe constants.

struct ValueBlob
  guint16 type;  /* 9 */
  guint   deprecated : 1;
  guint   reserved   :15;
  guint32 name;

  guint32 short_name;
  guint32 value;

          A short name for the value;

value:    The numerical value;

StructBlob (12 + 8 * n_fields + x * n_functions)

struct StructBlob 
  guint16      type; /* 3 */
  guint        deprecated : 1;
  guint        reserved   :15;
  guint32      name;

  guint16      n_fields;
  guint16      n_functions;

  FieldBlob    fields[];   
  FunctionBlob functions[];  

          The lengths of the arrays.

fields:   An array of n_fields FieldBlobs. The fields must be described in
          the order in which they occur in the struct.

          An array of n_functions FunctionDescriptors. The described functions 
          should be considered as methods of the struct. 


All GType registered types are described by an InterfaceBlob. Some parts of 
a descriptor will be zero, depending on what kind of type is being described. 
Interfaces have no parent and no fields, enums and flags have only values, boxed
types may have fields and functions.

struct InterfaceBlob
  guint16 type;  /* 4 for object, 5 for interface, 6 for enum, 7 for flags, 8 for boxed */
  guint   deprecated   : 1; 
  guint   reserved     :15;
  guint32 name; 

  guint32 type_name;
  guint32 type_init;

  guint16 n_interfaces;
  guint16 n_values;
  guint16 n_fields;
  guint16 n_properties;
  guint16 n_methods;
  guint16 n_signals;
  guint16 n_virtual_functions;

  guint16 parent;
  guint16 interfaces[];
  ValueBlob           values[];    
  FieldBlob           fields[];
  PropertyBlob        properties[];
  FunctionBlob        methods[];
  SignalBlob          signals[];
  VirtualFunctionBlob virtual_functions[];

          The name under which the interface is registered with GType.

          The symbol name of the get_type() function which registers the type.

          The lengths of the arrays.

parent:   The directory index of the parent interface. This is only set for 

Up to 16bits of padding may be inserted between the arrays to ensure that they
start on a 32bit boundary.

          An array of direcory indices which for objects point to the implemented 
          interfaces, while they point to prerequisited for interfaces. For enums,
          flags or boxed types this array is empty.

values:   Describes the constant values defined in this interface. 

fields:   Describes the fields. For enums, flags and interfaces this array is

          Describes the methods, constructors, setters and getters. For enums
          and flags, this array is empty.

          Describes the properties. For enums flags and boxed types, this array 
          is empty.

signals:  Describes the signals. For enums flags and boxed types, this array 
          is empty.

          Describes the virtual functions. For enums flags and boxed types, 
          this array is empty.

struct AnnotationBlob
  guint32 offset;
  guint32 name;
  guint32 value;

offset:   The offset of the interface to which this annotation refers. Annotations are
          kept sorted by offset, so that the annotations of an interface can be found
          by a binary search.

name:     The name of the annotation, a string.

value:    The value of the annotation (also a string)

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]