Re: Enabling builddir != srcdir by default in jhbuild

[Emmanuele Bassi <ebassi gmail com>]

On 1 June 2016 at 10:33, Sébastien Wilmet <swilmet gnome org> wrote:

> > A try server is not a simple thing to set up; and if the cost of
> > setting it up is non-negligible, it simply pales compared to the cost
> > of keeping it up. We can barely keep Continuous building as it is —
> > somebody would need to be funded full time just to keep it going for
> > each bug/feature/patch series that we'd want to test.
>
> With master/master-stable, developers, translators etc continue to work
> exactly as before. But GContinuous would automatically sets
> master-stable to commits that work (from the GContinuous point of view).
> And jhbuild would pick up the master-stable branch by default.
>
> It doesn't need more servers,

Oh, I can assure you: it does need a separate build machine — and not a VM.

> it needs development time. Currently
> GContinuous needs maintenance: tag modules manually to a previous
> commit, untag, etc etc. A lot of that manual work could be replaced by
> an automatic way. In the short term it needs more time to develop that,
> but in the longer term, it saves time and we would have something that
> works better.

This is wishful thinking, at best.

Continuous would have a lot less issues if people actually built and
tested locally *already*, using a set up that maps to what Continuous
does — hence the default of switching jhbuild to out of tree builds.

> With master/master-stable, no need to manually revert commits.

The only way this is going to work is if very, very few people can
push to master on git.gnome.org — which is obviously not going to be
the case.

If the workflow is:

 * people push to master-next
 * Continuous builds all 'master-next' branches
 * on success: ${SOMETHING} merges 'master-next' into 'master'
 * on failure: ${SOMETHING} sends an email to ${SOMEBODY} to fix 'master-next'

I can assure that these things will happen:

 * some maintainers will keep pushing to 'master', sidestepping the
whole automated build
 * some maintainers will try pushing to 'master-next', go through a
couple of issues (by and large with the build system) and stop
 * 'master-next' will break and *stay broken*, and 'master' will break
at random points because nothing runs 'master'

Unless you add a new machine that builds on 'master' as well, in which
case you now build everything twice and you have to unbreak two
separate builds, and you just made my life as a volunteer build
sheriff twice as hard, and I have not seen you volunteering for the
job either.

The same thing will happen if you have a master/master-stable split,
because now people will commit to master, break it anyway, and nobody
will look at 'master-stable' because it's a completely different
workflow than what Git and various tools and organisations use.

Obviously, on top of that, we have to write all the tooling that
implements the ${SOMETHING} above, and keeps it running; so now we
have:

 * two separate build bots
 * a set of tools that deals with checking and merging branches on git.gnome.org
 * a set of tools that deals with sending notifications to people
pushing commits that break either of those builds, and maybe files
bugs about it

And I have not seen anybody spend some time to work on the *existing*
Continuous builder to send an email to the people listed on the DOAP
file, or to the committer, whenever a module breaks.

There *are* various alternatives/steps to keep Continuous working:

1. have an 'integration'/'release-team' branch that only release team
members can use; this is merged (daily/nightly, or whenever the build
succeeds and the smoketesting runs)
2. limit pushing to the 'master' branch to people listed as
maintainers on the DOAP file; translations go to a 'i18n' branch that
gets merged manually, and all features/bug fixes happen on topic
branches
3. have a try server that automatically builds all topic branches in a project
4. have a try server that builds patch series on request (maybe with
Bugzilla integration, so that we get results as Bugzilla comments)

Any of the above can be implemented separately, and it'd be a great
thing already; considering the amount of people that care about
building GNOME at the moment, and the amount of work necessary to get
this show on the road, you'll forgive me if I don't hold my breath.

In general, though, GNOME is *not* a simple system; it's made of many,
many moving parts that are developed on different cadences, and those
moving parts are interconnected in non-trivial ways. Even Firefox,
which is at least a couple of orders of magnitude simpler than GNOME,
and has try servers, build sheriffs, and automated testing running on
different platforms, breaks a build to the point of requiring a
revert, and that happens at least once a month. The only large
difference is that Firefox has a dedicated team of people that keep
the build and testing under control.

Ciao,
 Emmanuele.

-- 
https://www.bassi.io
[ ] ebassi [ gmail com]