Re: Announcement/RFC: jhbuild continuous integration testing

[Martin Pitt <martin pitt ubuntu com>]

Hello Tristan,

Tristan Van Berkom [2013-02-14  6:42 +0900]:
> Upon reading this particular part (and I noticed before you are
> using mostly jhbuild mechanics), it leads me to wonder, how
> granular exactly are these rebuilds ?

Right now, every 15 minutes. Sometimes longer, when the previous run
is still running.

> I think ideally it would be great if builds could be triggered by
> commit. In other words, commits are serialized chronologically and
> each and every commit should trigger an entire rebuild, each rebuild
> should build everything in the moduleset "up to the latest commit"...
> separately, one after the other.

That is indeed the long-term plan, but there's still some work to be
done before we can do that. The machine we are running this on has 64
2.7 GHz cores and 64 GB of RAM, that really isn't a bottleneck right
now. The main two problems right now are that the "jhbuild update"
stage takes some 5 minutes to update all the ~ 160 git trees, and
that jhbuild build doesn't parallelize at all, i. e. build modules
which don't depend on each other could build in parallel.

Once we solve both, and we dramatically reduce the time of one run
from "several hours" (which is currently needed if e. g. a glib change
happens, which rebuilds pretty much everything) to < 15 minutes.

> The way I imagine this works now (and this is a big assumption,
> correct me if I'm wrong), is that a commit in a given module triggers
> a jhbuild build, which would mean that:
>
>    a.) Several commits could have been made in a given module
>         by the time jhbuild actually runs... meaning we dont know
>         which of the given commits in that lapse of time actually
>         caused the fault.

That's right. It's massively better to know that one commit in these
15 minutes triggered it than "something in the past week", but still
not perfect as you say.

>    b.) Module "foo" triggers a rebuild... and while jhbuild builds,
>         it also pulls in new changes from module "bar", in this
>         case it's possible that a recent commit in module "bar"
>         caused another module "baz" to be effected,  but in the
>         end it's module "foo" who is blamed (since module "foo"
>         essentially /triggered a rebuild/)

That's a trickier thing. For most commits, one should actually be able
to build them independently, but sometimes those "in between" breaks
are inevitable. Say, you make an API change in a library and then
update your application to the new API, then in between you will get a
build failure. The next iteration should fix it again.
We have that problem independently of the frequency we build stuff of
course, as we can always hit a "bad time".

Martin
-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)