[gnome-love] Tinderbox (automatic daily build) for GNOME

[Sergio Villar Senin <svillar igalia com>]

Hi,

after fixing some issues we have set up a complete GNOME tinderbox
compilation for 2.14 moduleset. You can see it here
http://tinderbox.fisterra.org/all_trees.panel.html We want to
point out that we did the minimum changes required to the tools we use,
in order to have a first release.

Once this test has been completed we gathered some conclusions we want
to share with you. I'll try to list the changes we made and some
proposals for changes in the tools. Comments will be highly appreciated.

* Client changes

   * tinderbox
- build_shellscript: This script usually iterates over a list of trees
passed as arguments. We think it is not useful for the whole GNOME
compilation though. The reason is that the command that launches the
tinderbox loop would be "tainted" because it should include something
like "--tree :cairo:libxml2:libslt:.....:" up to more than 100 modules.
The second reason is that all the modules will send their results to the
same web page. We modified it in order to iterate over all the modules
of GNOME, considering that each module is a tree. We get the list of the
modules with the 'jhbuild list' command, that gives us the list of
modules sorted in the right order.
- buildcf: There is no need for the TreeData.pm file in the client
because we will not use any information that could be stored there
because jhbuild takes all data from the moduleset files. This would be a
new buildcf_jhbuild version.
- Sometimes, the build logs show weird symbols. Encoding issue?
- Despite the current error parsing rules are quite good there are still
some false positives, mainly related to compiler options
(-fno-exception) or file names that include words like error, exception
and so on. What about adding a gnome_parser to handle this process properly?
- Neither the client nor the server are moduleset-dependent, changing
the moduleset is as easy as change the value of a parameter.
- Currently it is not possible no begin the build loop at a specified
module (because we do a buildone over the whole list). Maybe it could be
done just skipping all the modules into the build_shellscript loop until
the specified module name was found. It would be also necessary to add a
new parameter to the build_shellscript to pass this module name.
- the build loop is not stopped even a basic library fails. You can
argue that if we haven't got glib we have nothing. Well, we think that
the tinderbox loop should not stop because it's probable that you have
installed into your jhbuild environment an older version of the library
that failed. This will allow the other modules to be compiled. In our
humble opinion the use of 'jhbuild buildone' isolates the errors within
each module.

   * jhbuild
- added support for independent phase execution for the buildone
command. This allows you to call for example jhbuild buildone -f gtk+
for only executing the configure phase for the gtk+ module. As I said
before we do not like our current implementation because it lacks some
design work. One important question for taking into account is that the
available STATES are different per module, for example a module packaged
into a tarball (without CVS access) will not have a STATE_CHECKOUT but
STATE_DOWNLOAD or STATE_UNPACK. We think it would be nice to have this
feature even for other projects.
- the use of "--no-interact" parameter is mandatory. This prevents the
compilation wait for an user input.

* Server changes

   * tinderbox
- TreeData.pm: this file is mandatory for the server. We modified it in
order to add dynamically the list of jhbuild modules to the VC_TREE
variable that stores all the available modules. It is also unnecessary
to have there information about branches or repository roots, the server
does only need the module names. For getting the list of modules, we
call 'jhbuild list' as in the client. There is an issue here, we must
ensure that both clients and server are using the same moduleset. Maybe
adding something like "moduleset: gnome-2.14" to the header of the log
files would help the server to discard client build logs that does not
belong to the moduleset that is being built. Another possibility is to
configure the server to receive compilation logs from clients that build
modules of different modulesets. This will imply more work because the
URL generation system should take into account the moduleset for
creating the URLs.
- Another interesting feature would be adding links to the phases inside
the full log in the summary page.
- The default outputs of the tinderbox are not quite good for a long set
of modules, it would be nice to generate another kind of view.
- We had some problems with the gtk+ module due to the '+' symbol. The
tinderbox server removed it when creating the required files for showing
the status pages. We fixed it adding the '+' symbol to the set of
allowed symbols, but we do not know if it will break builds in another
file systems different than ext3. Anyway IMHO there are some renames
spread over the tinderbox code that should be managed in an unique
location (for example the tinderbox still generates some URLs with "gtk"
instead of "gtk+").

We have still some little bugs to fix and the system lacks some features
that are a must for a real installation, like for example security. What
are the next steps? I think the following:

   * client packaging: for distributing client compilation
   * integration with measurement tools: lcov, memcheck (valgrind)...
   * integration with GNOME Bonsai (ViewCVS after the SVN migration?)
   * integration with GNOME Bugzilla
   * security concerns: remote administration of the server, client
authentication...

How much can it take? Client packaging should be not difficult, we could
start with a limited number of package types, .deb and .tgz for example.
We already have a tinderbox integrated with some measurement tools here
at Igalia, so it won't take also a lot of time. The last two points,
integration with remote Bonsai and Bugzilla, should give us not a lot of
problems (maybe Mike Taylor could give us more details). And finally the
security concerns. It's a risky task IMHO. I saw some options for doing
email authentication but no idea about how many time will it take.
Anybody has some experience related to this?

> From here we think that we have enough info to decide if this solution
is suitable for gnome continuous integration. Is there any other info
that should be added or answered to all the requirements Luis sent?

Comments? Opinions?

Regards.