Re: Ideas about improving gtk-doc

[Brian Cameron <Brian Cameron Sun COM>]

Matthias/Stefan:

First of all, I'd like to highlight that it isn't critical that all
the suggestions that I've made be implemented.  While I would personally
like to see them all addressed, I'll be delighted if we make any of the
changes.  I think that I've mentioned that my main reason for encouraging
these sorts of improvements in gtk-doc is because these changes would help
to make the interface documentation that gtk-doc creates more in-line with
the sort of interface documentation that is required internally at Sun
during our ARC (Architecture Review Committee) process.

I understand that this reason, in and of itself, is not a good reason to
make changes to a tool like gtk-doc.  However, I believe that some of the
interface documentation that Sun's ARC requires is of general use.  So I
think it is useful to explain why Sun's ARC thinks this additional
documentation is useful and to see if the community agrees that adding
this sort of additional information to the tools makes sense.

That's why I say that it isn't really necessary that all the suggestions
I'm making get implemented.  I'm expecting the community will likely
pick-and-choose which ones also make sense to the community.  The
closer we can align the internal Sun process with the community external
process, the easier and better things will be for us at Sun.  By working
with the community to improve gtk-doc to better document interfaces that
it currently neglects (such as environment variables and structures),
then the community benefits by better and more complete developer
documentation (or at least the infrastructure to make better
documentation possible).

This is really an experiment here at Sun.  So far, the ARC process hasn't
translated into many suggestions back to the community about things
that ARC considers important that the community could, perhaps, do better.
I think it is a healthier relationship for Sun to have with the
community to at least make sure that issues that ARC brings up are issues
that the community is made aware of.  This way the community can decide
which, if any, of their ideas also make sense to the community.

I realize that by exploring such topics, I risk coming off as a person
who is just trying to push Sun's ideas on the community.  I'd like to
make sure that I highlight that this really isn't my intent.  I'm really
just trying to make sure that the things Sun's ARC community brings up
are at least brought to the attention of the community.  My hope is that
if Sun and the community find that they agree on such issues, that Sun's
ARC community might become more involved with interacting more directly
with the community in general.  I think such a transformation here at
Sun would be a good way for Sun to better contribute to the GNOME project.


I am going to respond to both Matthias' and Stefan's comments in one email
to reduct traffic.


Matthias says:
> > My understanding is that the gtk-docs are built from the .sgml files
> > that are normally stored in the "tmpl" directory of a module.  These
> > files have a structure that looks like this (using gdk's code as an
> > example for STRUCT, ENUM, and FUNCTION):
>  
> You can document things in the templates, but generally we prefer to 
> document things inline in the sources now (at least for glib, pango and
> gtk+). Only the short and long descriptions still *have* to be provided
> in the templates.

Thanks for the clarification.  The format of the comments is the same,
though, regarless of whether you document in the .sgml file or in the
code, or am I still lacking in my understanding of how things work?


Matthias says:
> > 1. Currently the "Deprecated" tag only highlights information about
> >    the deprecation, such as:
> >
> >    * Deprecated: Use gdk_draw_layout() instead.
> >
> >    Changing this to also include information about what version the deprecation
> >    happened would be useful.  For example, if a function was deprecated in
> >    version 2.6, the tag could look like this:
> >
> >    * Deprecated: 2.6: Use gdk_draw_layout() instead.
> >
> >    Then the version information could be included along with the descriptive
> >    text.
>
> Do you win much by providing that information ? If an application is
> supposed to be working with, say 2.2, it has to be compiled against 2.2
> anyway, therefore the developer will likely have 2.2 (and thus also the
> 2.2 docs) installed somewhere anyway.

I think information about what version an interface becomes deprecated is
very useful.  If I am planning to upgrade my application from 2.4 to 2.6
it is useful to be able to refer to the docs, and have them highlight
what I need to worry about.  It doesn't seem like much additional hassle
to include the version information along with the text.

Here at Sun, we need to provide our ARC (Architectural Review Committee)
with documents that highlight all interface changes between releases, so
having this information come from gtk-doc would make this process easier.
It seems that such information would also be generally useful.  As you
say, the best way to do this currently is to manually compare two
versions of the gtk-docs.  Considering that www.gtk.org only keeps the
latest version of the docs available online, this has to be done by
manually comparing the docs that are installed to two different machines
with different versions of the software installed.  This is a real
hassle.


Stefan says:
> I am neither see much advantage of it. You can still just add this information
> if you want. As a developer I will continuosly upgrade to newer version of the
> underlying list. Thus when I spot depecated things I will migrate the code.
> Afterward it doesn't matter if it is deprecated since 2.0 or 2.2 ...

You are assuming that a developer is constantly upgrading to a newer version
of the code.  While this might be the most common case right now in the Linux
community, it isn't uncommon for some applications to need to support code
that has been distributed to several different versions of the codebase.  I
might be a developer who needs to maintain code built on GNOME 2.0, 2.2, 2.4,
and 2.6.  Being able to clearly see what interfaces I should be using in each
release would be useful to such a person.


Matthias says:
> > 2. Highlight change to data structures and enumerations:
> >
> >    Perhaps this could be done by adding a new keyword to the list so a new element
> >    could be highlighted by:
> >
> >    @height:Since 2.6: the height of the rectangle.
> >    @height:Removed 2.6: the height of the rectangle.
> >    @GDK_WINDOW_FOREIGN: Since 2.6: foreign window (see gdk_window_foreign_new())
> >    @GDK_WINDOW_FOREIGN: Removed 2.6: foreign window (see gdk_window_foreign_new())
> >
> >    The scripts for checking structures should be smart enough to verify ABI
> >    compatibility.  In other words, ABI is only not broken if elements in data
> >    structures are added to the end of the structure.  If enumeration values
> >    or data structure elements are removed, or if elements in data structures
> >    are added in anywhere but the end, this should be highlighted as an ABI
> >    breakage.
>  
> I wouldn't mix abi checking with documentation. I can see us adding
> struct size checks for important public derivable classes to the gtk abi
> checking, but there is no way to make this kind of abi checking
> completely waterproof. 

That's a good point.  I withdraw the suggestion that gtk-doc should check for
ABI compatibility.  I think it would be enough if structure data elements could
be marked with the following keys:

  + Since - indicating an addition
  + Removed - indicating a removal
  + Modified - indicating a change, perhaps of data type.

Does this seem reasonable?  The docs could then just highlight the differences
and it would be up to the end-user to dig into the code look further to determine
if an ABI breakage has occurred.  If a breakage has occurred and is intentional,
it could be detailed in the structure description.


Stefan says:
> ABI checking is really not what gtk-doc should be concerned with. Apart I
> understand that it might be usefull to support "Since:" and "Deprecated:" for
> members as well.

Agreed.  I think simply supporting Since and Deprecated for members of a
data structure and for enumerations would be useful.


Matthias says:
> >    In the generated gtk-docs, additional text highlighting changes would look
> >    something like this:
> >
> >    enum GtkFooEnum
> >    typedef enum
> >    {
> >       GTK_FOO_VALUE_1
> >       GTK_FOO_VALUE_3  (Added 2.6)
> >    } GtkFooEnum
> >
> >    GTK_FOO_VALUE_2  Removed 2.4
>
>
> We can't remove public APIs anyway, only deprecate them. And if they are
> not public, they shouldn't show up in the docs

Is this true for all libraries that use gtk-doc?  I assumed that it could be
used to document volitale libraries that do not guarantee ABI stability from
release-to-release.

I'm happy if we use the term "Deprecated" instead of removed if gtk-doc
doesn't want to support such volitale libraries.


Matthias says:
> See above. Instead of figuring out ways to document abi breakage, we
> should just avoid breaking abi.

Agreed.  As I said above, I withdraw the suggestion that we should try
to make gtk-doc document ABI breakage.  I still feel it would be useful
to be able to document when elements are added or deprecated to enums or
structures


Matthias says:
> > 3. Add a keyword "Stability" which will be used much like "Since", "Deprecated", etc.
> >    Valid values would be:
> >
> >    Stable   - the interface is not going to change until the next major release
> >               (2.0 to 3.0 for example).  It will not change in a minor release
> >               (like 2.2 to 2.6)
> >    Unstable - the interface may change in a minor release.
> >    Obsolete - the interface exists, but should not be used since it is obsolete,
> >               typically when the function has been deprecated.
> >    Private  - the interface exists only for private use and should not be used
> >               for general consumption.  An example would be a gdk function that
> >               can be used by gtk, but should not be used by any other library.
> >
> >    Such stability levels would be displayed along with the function information
> >    in the generated gtk-docs.
>
> I'm not sure this is really worth adding to the docs in any formal way.
> You can always add information about this informally. Obsolete doesn't
> seem to add anything beyond Deprecated, and private apis should simply
> be left out of the docs.
> For unstable, one could think about utilising the defines we use in some
> places for apis which are considered less stable, e.g.
> GDK_PIXBUF_ENABLE_BACKEND or PANGO_ENABLE_BACKEND.


I guess for an example of a function that might be marked as Private is
gtk_widget_reset_rc_styles, which has a description "This function is not
useful for applications".  This is an example of a public interface that
is for a specific use and not intended to be coded to.  How should this
sort of function be documented in the gtk-docs?  Should it really be left
out?  Adding Stability levels allows the docs to specify which functions
applications should consider using, and give the user an idea of how
stable the interface is.

Also, you are suggesting that all libraries that use gtk-doc should only
document public and stable interfaces.  But I imagine that there must be
some libraries (perhaps eel for example) where functions change from
release-to-release and stability isn't guaranteed.  Being able to specify
which interfaces are Stable vs. Unstable would allow people maintaining
such libraries to specify which interfaces are most volitale and which
ones are more likely to be stable.  This does seem generally useful.

I do agree that Obselete and Deprecated do mean the same thing, so that
classification could go away.  But I do see some value in being able to
specify Private or Unstable.

Stefan says:
> The problem here is who is the audioence of the docs. The public docs should
> only contain public API. I do document private methods in the same fashion as
> public API, but that docs is never extracted (unfortunately gtk-doc warns about
> that). These comments are purly for the package developers. It might be handy to
> build docs for them too.
> 'Obsolete' is the same as 'Deprecated' for me. And 'Stable' vs.'Unstable' is
> often just handled via package versioning.

Yes, you highlight one of the advantages about being able to document Private
interfaces.  It makes it possible to include information about private information
in the public docs, so that people developing on the module itself can easily
find the docs.  Users of the library can understand that Private interfaces
aren't intended for them.  This could be made more clear by separating Private
and Stable interfaces into separate pages.

Further, you are assuming that all libraries that use gtk-doc are intending
for all interfaces to be Stable.  I must assume that there are some libraries
that could benefit from documentation where the maintainers aren't wanting to
ensure stability from release-to-release.  Being able to mark functions as
Unstable allows people to use gtk-doc for libraries that are not guaranteed
to be ABI stable from release-to-release, and clearly mark which functions
are Unstable and which ones are Stable.  Some libraries, like eel, might
benefit from a documentation infrastructure that makes them think about
which functions should be classified into which bucket, and help to encourage
Unstable areas to migrate to more a more Stable playing field.


Matthias says:
> > 4. Add two new types, FILE and ENVVAR, for documenting files and environment
> >    variables.  They would also support the "Since" and "Deprecated" tags.
> >    They would work much in the same way as ENUM or STRUCT except that they
> >    would document a file or environment variable rather than a macro.
>
>
> We have documented them in a dedicated section in the gtk+ and glib api
> docs. Would you win much by moving these docs to the templates ? It
> might be a win to be able to document them in the source next to the
> functions using them, but for that you need to invent a nonconflicting
> hash key syntax, so that you can stuff them in the global symbol hash.

I think the biggest advantage of moving them to the templates would be that
it would be easier to document such interfaces, so libraries that haven't
gone to the trouble to document them in separated dedicated sections could
more easily start documenting these interfaces.

Stefan says:
> The beaty of the gtk-doc approach is that it mearges with normal docbook
> documents. So you can include such list as separate section (you could even
> generate those with little tools).
> Generally it is true that the behaviour of a method could be affected by
> external entities, such as files, environment vars, databases, network
> conections, ..., but that sould be informally documented (too much variety)

I think you are missing something.  The issue is not that environment variables
or file locations could affect the function interfaces (which is true), but
that environment variables and file locations are interfaces deserving of
documentation as much as function interfaces.

Users and developers might be reasonably interested to know what file locations
the code depends upon and how environment variables affect the way the code
runs.  I think making gtk-doc support the ability to document these interfaces
makes sense, especially since such documentation would really be quite similar
to the way gtk-doc already heandles other interfaces.

I mean you could make the same argument about macros, enumerations, or structures.
Why document them with gtk-doc when you could just document them directly in
docbook and maintain them that way.  The reason it isn't done that way is that
it is easier to document with standardized comments in the code.  Why not support
such easy documentation for interfaces such as file locations and environment
variables?  Adding such functionality seems fairly straightforward since gtk-doc
already supports documenting many different types of interfaces already.


Matthias says:
> > 5. It would be useful if gtk-docs generated an additional page that highlights
> >    all changes that are found in the gtk-docs.  This page would highlight all
> >    interfaces (functions, structures, enumerations, files, environment variables)
> >    deprecated, added, or changed in all previous releases.  This would serve as a
> >    reference page to anyone who is interested in seeing what interfaces have
> >    changed from release-to-release.
>
>
> Like http://developer.gnome.org/doc/API/2.0/gtk/ix02.html ?

Yes, although it would be useful to see more information on this page.  Being able
to identify which version each symbol was deprecated would be nice, being able to
see interfaces that have changed (additions to enumerations, changes to data
structures), being able to see interfaces that have been added with version numbers.

In other words, expanding this page from just a list of deprecated symbols to
a page that highlights change.


--

Brian