Re: Another defs file draft

[Marius Vollmer <mvo zagadka ping de>]

Hi,

I'm happy that you are still thinking about the defs files.  I have
not been following the discussion, and I don't think I'll be able to
contribute to it much in the future, but this seems like a good
occasion to put in some comments.  I'm making them partly from the
guile-gtk point of view, but more from a general language design
perspective.  I'm sure we can use everything you guys come up with in
the guile-gtk package, and I'm certain that any remaining glitches can
be elegantly solved because the defs file syntax is so extensible.

One important thing is of course that not only the syntax is described
very precisely, but also the meaning of the various constructs.  This
draft is the only part of the whole defs file discussion that I
actually read, so I might be missing a lot, but from the text below, I
think that the meaning of the defs file expressions could need some
more words.

Havoc Pennington <hp@redhat.com> writes:

> The overall syntax is:
> 
>      (type-of-thing-being-defined  name-used-to-refer-to-this-thing
>        (attribute-name  attribute-value-depending-on-the-attribute)
>        (attribute-name  attribute-value-depending-on-the-attribute)
>        (attribute-name  attribute-value-depending-on-the-attribute))

I think this is a very nice fundamental structure.
 
> ===
> (module module-name
>   (submodule-of module-name)) ;; submodule is optional

About the naming of defined features: you seem to want to have a
hierarchical name space, which is a very good thing.  I think this
concept deserves to be defined idependently from the rest of the defs
file syntax so that you don't need special attributes like
`submodule-of' or the `optional-module-name-if-different' part in the
`parent' attribute of `object'.  Maybe this way:

    Features are named by a sequence of symbols:

        name ::= (SYMBOL-1 SYMBOL-2 ... SYMBOL-n)

    A name of this form is termed a `absolute name'.  There are also
    `relative names', which are converted into absolute names in the
    following fashion.

    Statements are read and `evaluated' (whatever that means for defs
    files) in a sequential fashion and at any one time, there is one
    name that denotes the `current module'.  This current module is
    established by various forms in various way.  See the individual
    descriptions for details.  At the beginning of a defs file, the
    current module is the empty list of symbols, `()'.

    A relative name consists of a single symbol:

        name ::= SYMBOL

    This form of a name is equivalent to the absolute name that is
    formed by appending SYMBOL to the current module name.

With this notion of names, the `module' statement would read:

    (module MODULE-NAME)

    Sets the current module name to MODULE-NAME.

    Ex:

    (module Gdk)              ; A relative name, expanding into "(Gdk)"
    ;; later, same file
    (module Rgb)              ; A relative name, expanding into
                              ; "(Gdk Rgb)"
    ;; later
    (module (Gtk))            ; A absolute name, cancelling the 
                              ; "(Gdk Rdg)" prefix.

Maybe it is useful to also have

    (module-end)

    Set the current module name to the one active before the last
    `module' statement.  (This should of course act like a stack).

> ===
> 
> (type
>  (alias some-unique-identifier)
>  (in-module module-name)   ;; optional, gchar* is not in a module
>  (gtk-type-id gtk-type-system-id) ;; optional, absent if this is not
>                                   ;; in the type system
>  (is-parametric boolean)          ;; optional default to #f
>  (in-c-name name-of-symbol-in-C)
>  (out-c-name name-of-symbol-in-C)
>  (inout-c-name name-of-symbol-in-C))

I think this should be

    (type NAME
      (gtk-type-id ...)
      ...)

with `alias' and `in-module' removed.  the module issue is taken core
of by the name construct.  When we want to have real aliases for
features, a dedicated statement should be defined:

    (alias NAME-NEW NAME)

    This makes the feature named NAME also available as NAME-NEW.

Or, to conform to the general pattern from above

    (alias NAME-NEW
      (old-name NAME))

> Ex: (type
>      (alias string)
>      (gtk-type-id GTK_TYPE_STRING)
>      (in-c-name "const gchar*")
>      (out-c-name "gchar**")      ;; actually I'm not sure how strings work out/inout
>      (inout-c-name "gchar*"))

I think much care needs to go into describing what exactly in, out and
inout parameters are.  Who manages the memory, who is allowed to
modify what storage and if pointers are allowed to be retained beyond
the function call, etc.
 
>  (type
>      (alias list)
>      (gtk-type-id GTK_TYPE_POINTER)

Is this true?  I think it would be preferable to either extend the
Gtk+ run-time type system to really support GLists, or to not declare
a gtk-type-id for lists.

>  ;; This one would be implied by the (object) def for GtkWidget I
>  ;; think - (type) is only required for types that are not implied
>  ;; by other definitions, such as int/boolean/etc.
>  
>     (type
>      (alias GtkWidget)
>      (in-module (Gtk))
>      (gtk-type-id GTK_TYPE_WIDGET)
>      (in-c-name "GtkWidget*")
>      (inout-c-name "GtkWidget*")
>      (out-c-name "GtkWidget**"))
> 
> "Type" bindings are automatically assumed for objects, boxed types,
> etc. as defined below.
> 
> The alias field is used to refer to the type later on.
> 
> Whenever a type alias can be used, it is also possible to use the
> keyword "native", which implies that the type in question is too
> C-specific to represent. Then a c-declaration will typically be
> available for use.

Can you give an example for this?

> C types containing [] or () are function pointers or arrays. For
> arrays that don't specify a size, we just treat them as pointers. For
> function pointers, we need special (type) syntax/attributes of some
> kind, but since there basically aren't any of these right now in the
> libs we care about we can just ignore them. For arrays that specify a
> size ditto, you would handle them by adding an (array-size) attribute
> or something or using the "native" keyword and skipping the (type)
> stuff.

I have some stuff for composite types in guile-gtk, which is not fully
debuggeg yet, but I like it anyway.  Maybe you want to get some
inspirations from it.  Here is the (minimal, existing) documentation
from the guile-gtk/README:

    Composite Types and `call-by-reference'
    ---------------------------------------

    Guile-gtk has (still experimental) support for composite types
    like lists and vectors.  On the Gtk+ side, the types GSList
    (singly linked list), GList (doubly linked list), and counted and
    fixed length vectors are supported.  On the Scheme side, you can
    use Scheme lists and Scheme vectors.  You can mix these types
    freely.  That is, you can use a list on the Scheme side to
    represent a vector on the Gtk+ side, and vice versa.

    Composite types have modes associated with them.  They can be
    `in', `out', or `inout'.  When a composite type is marked with a
    `in' mode, the Gtk+ side is not allowed to make changes to the
    contents of the composite.  When it is `out', it must initialize
    the contents before it returns and is not allowed to read it prior
    to initialization.  Consequently, mode `inout' indicates that the
    Gtk+ side may both read and write the composite freely, and that
    the contents has been initialized by the caller.  On the Scheme
    side, mode `out' turns off type checking for the contents of
    composites and the initial values of the composite passed to Gtk+
    will be undefined.

    The default mode is `in'.

    The available variations are:

      [ *.defs file syntax ]             [ C side function arguments ]
      [ Comment ]

      (slist <type> [<mode>])            GSList *lst
      For singly linked lists.  Scheme owns the memory of the list nodes.

      (list <type> [<mode>])             GList *lst
      For doubly linked lists.  Scheme owns the memory of the list nodes.

      (cvec <type> [<mode>])             int len, <type>* vec
      For counted vectors.  The length of the Scheme composite is passed
      to the Gtk+ function.  Scheme owns the memory of the vector.

      (cvecr <type> [<mode>])            <type>* vec, int len
      For counted vectors, reverse order of arguments.  Scheme owns the
      memory of the vector.

      (fvec <type> <len> [<mode>])       <type>* vec
      For fixed length vectors.  The Scheme composite must be of length <len>.
      Scheme owns the memory of the vector.

      (ret <type>)                       <type>* vec
      Abbreviation for (fvec <type> 1 out)

    These composite types are also intended to be used for
    call-by-reference arguments.  Neither Scheme nor C really has
    these call-by-reference arguments, so guile-gtk wont either.  In
    C, you would pass a pointer to the desired object; in Scheme you
    use a one-element list or a vector of length one.  For guile-gtk,
    these types are modelled with a `fvec' composite type of length
    one and mode out, or equivalently but shorter with the `ret' type.

    For example, when you have a Gtk+ function with a prototype like

        void gtk_foo (char **strptr);

    that deposits some string in *STRPTR (whose memory should be taken
    over by the caller), you can wrap it like this:

         (define-func gtk_foo
           none
           ((ret string) strptr))

    Usage is a little cumbersome from Scheme.  This code snippet

         (let ((strptr (list #f)))
           (gtk-foo strptr)
           (car strptr))

    would yield the returned string.

As a real life example, here is the declaration of the
gdk_pixmap_create_from_xpm function (I use GdkWindow for GskPixmap and
GdkBitmap because we don't have name aliases in the guile-gtk defs
files yet):

    GdkPixmap* 
    gdk_pixmap_create_from_xpm (GdkWindow  *window,
                                GdkBitmap **mask,
                                GdkColor   *transparent_color,
                                const gchar *filename);

    (define-func gdk_pixmap_create_from_xpm
      GdkWindow
      ((GdkWindow window)
       ((ret GdkWindow) mask (null-ok))
       (GdkColor transparent_color (null-ok))
       (string filename)))

You see that although the `transparent_color' parameter is actually
modified by Gtk+, it is not delcared as `out', because the `mode'
concept only applies to composites.  You might want to do better.

> ===
> (object object-name
>    (in-module module-name-list)

`in-module' would not be necessary.

>    (parent object-name optional-module-name-if-different)

This would just be `(parent NAME)'.

>    (abstract boolean-is-abstract-class) ;; omit for default of #f
>    (c-name name-of-the-object-in-C)
>    (field (type-and-name type-alias-of-struct-field name-of-struct-field)
>           (access read-or-write-or-readwrite)))
>    
> 
> Ex: (object Widget
>       (in-module (Gtk))
>       (parent Object)      ;; could say (parent Object (Gtk))
>       (abstract #t)
>       (c-name GtkWidget)
>       (field (type-and-name GdkWindow* window) (access read)))
                              ^^^^^^^^^^

Wouldn't that be

        (field (type-and-name (Gdk GdkWindow) window) (access read))

because we are referring to GdkWindow which is supposedly defined as

    (module (Gdk))

    (boxed Window
      (c-name "GdkWindow*")             ; note the "*"
      (ref-func window_ref)
      (release-func window_unref))

> It also implies a module that is the name broken into parts:
>  (module CTree
>    (submodule-of Gtk))

Hmm, wouldn't it be better to go just state the name explicitely in
already `broken' form?  Like

    (object (Gtk CList)
      ...)

implies

    (module (Gtk))
    
    (type CTree
      ...)

But maybe it is just better not to try to be too clever.  Not implying
anything about the current module might be cleaner.  Then people might
write something like

    (module (Gtk))

    (object CList
      ...)

where the `object' form gets translated into a corresponding `type`
form of

    (type CList
      ...)

If the defs files are meant to be largely auto generated, then
convenience should make way for cleaner semantics anyway.

> ===
> 
> (function function-name
>   (in-module module-name-list) ;; "static methods" go in their
>                                ;;  object's module
>   (is-constructor-of object-type-alias) ;; optional, marks a constructor
>   (c-name function-name)
>   (return-type return-value-type) ;; defaults to void
>   (caller-owns-return boolean-value) ;; defaults to #f
>   (can-return-null boolean-value) ;; defaults to #t
>   (parameter in-or-out-or-inout 
>       (type-and-name parameter-type-alias parameter-name)
>       (type-parameter name-of-contained-type) ;; optional, requires parametric type
>       (c-declaration "c-type-and-name")) ;; c-declaration only required
>                                          ;; if the type alias is "native"
>   (varargs #t) ;; has varargs at the end
> )
> 
> Ex:
>   (function init
>     (in-module (Gdk Rgb)
>     (c-name gdk_rgb_init)))
> 
> Ex: 
>   (function new
>     (in-module (Gdk Rgb Cmap))
>     (is-constructor-of GdkRgbCmap)

I would prefer to refer to the defs file name of GdkRgbMap here, which
is likely `(Gdk Rgb Cmap)'.

>     (c-name gdk_rgb_cmap_new)
>     (return-type GdkRgbCmap)

Likewise.

>     (caller-owns-return #t)   ;; perhaps this could be implied by is-constructor-of

I think it is best not to imply too much.

>     (parameter in (type-and-name array-of-guint32 colors))
>     (parameter in (type-and-name gint n_colors)))
> 
> Ex:
>   (function config_set_set_handler
>    (in-module (Gnome))
>    (c-name gnome_config_set_set_handler)
>    (parameter in (type-and-name native func)
>                  (c-declaration "void (*func)(void*)"))
>    (parameter in (type-and-name gpointer data)))  
> 
> ===
> (method method-name
>   (of-object object-name module-name)
>   ;; retval/arg attributes as for (function), but with first parameter
>   ;; omitted for non-constructors

Can methods be constructors?

> ===
> 
> (boxed boxed-name
>   (in-module modname)
>   (c-name c-name)

I think the C name should include the "*" pointer marker when
appropriate.

>   (ref-func func-to-increase-refcount)
>   (copy-func func-to-copy)
>   (release-func func-to-destroy-or-decrement-refcount)
>   (field (type-and-name type-alias-of-struct-field name-of-struct-field) (access access-rule)))
> 
> It is never OK to use memcpy() to copy a boxed type, or use
> malloc()/free() to alloc/free one.
> 
> Ex:
> 
>  (boxed Pixmap
>    (in-module (Gdk))
>    (c-name GdkPixmap)
>    (ref-func pixmap_ref)
>    (release-func pixmap_unref))

I think the `c-name' should really be "GdkPixmap*" (maybe with quotes)
because a GdkPicmap is actually used that way in C code.  `Boxed'
types are _always_ treated as handles.  There might be types like

    typedef struct {
        ...
    } *foo_handle;

that already include the pointer-nature, while others (like GdkPixmap)
don't.  The implied type definition would change accordingly.

> An "object" declaration automatically implies the type definition:

"boxed"?

> (type
>   (alias concat-module-elements-and-boxed-name)
>   (in-c-name pointer-to-c-name)
>   (out-c-name pointer-to-pointer-to-c-name)
>   (inout-c-name pointer-to-c-name))
> 
> Ex: 
>  (type (alias GdkPixmap) 
>        (in-c-name GdkPixmap*) 
>        (out-c-name GdkPixmap**) 
>        (inout-c-name GdkPixmap*))
> 
> 
> ===
> 
> (struct struct-name
>   (in-module modname)
>   (c-name c-name)
>   (field (type-and-name type-alias-of-struct-field name-of-struct-field) (access access-rule)))
> 
> Unlike a boxed type, a struct type can be copied with memcpy() and
> allocated on the stack or with g_malloc().

This should probably _not_ contain the pointer, because it is not
treated as a handle.
 
> (typedef new-name
>   (in-module module)
>   (c-name c-full-name)
>   (orig-type alias-of-orig-type))

Oops, here is the `alias' statement, more or less.



I hope this is not too much off-track.

- Marius