Re: [ Revised Proposal ] Continuous Builds in GNOME

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

On Fri, 2016-06-17 at 07:55 +0200, Milan Crha wrote:
> On Thu, 2016-06-16 at 14:16 +0900, Tristan Van Berkom wrote:
> >   o The integration branch of every module is continuously
> >     reconstructed based on that module's master branch.
>       Hi,
> as the 'integration' branch targets only jhbuild, then it doesn't
> make
> sense to add it at each git module, no?

I dont think it targets only jhbuild, but jhbuild is an obvious
plausible consumer of it - rather it is the stomping grounds for
automated CI to be developed and the reference point for automatically
filing bugs against modules for broken commits; jhbuild is the consumer
of this which eases workflow for new comers.

>  Thinking of it, instead of
> playing with a jhbuild only, you propose to add one more layer (and
> place of possible issues) between the git master <-> jhbuild
> connection. The proposal, as I understand it, is:
>   git master <-> git integration <-> jhbuild
> where the git integration can be sometimes skipped in the jhbuild.
> But
> as you already have a functionality to skip some commits inside the
> jhbuild,

I dont believe you have any such functionality in jhbuild.

At best, you can specify the sha1 of the commit you want to build of a
given module, this would not let you single out one commit, omitting it
from history in integration and at least attempt to apply subsequent
commits without that one included, re-including that commit in the
correct place in history only when CI infrastructure would find
integration unbroken.

>  and the integration branch is meant to do exactly that, then
> why to have one software which would eat only space on a disk

I think disk space is really not a concern here, integration branches
would be rebuilt, they should not be accumulative, in the usual case
where no commits need to be omitted; the ref is just a ref to master.

If disk space is an issue somehow (because git will normally keep
history even when branches are re-written, the old sha1's still exist),
a mirror could also be an option, but I think branches are better.

>  in each
> git module with a new branch history, instead of writing the
> automated
> build tool which would cooperate directly with the jhbuild setup? You
> also want to add some restrictions on the git integration branch,
> which
> is even more unnecessary work.

Honestly, I see where you are going, but I tend to disagree; managing
this sort of thing in a jhbuild xml file, who's state needs to be
managed ideally by a cooperation of gnome-continuous build bots
(probably multiple in tandum to test on all supported arches) sounds
like a lot of work, not to mention it does not allow for singling out
commits, it can only block integration on the first integration
breaking commit.

> Not having the branch will make things easier for the developers and
> new comers too, as there will be no confusion for them. Having
> somewhere a super-detailed description of the integration branch
> intention is nice, but it'll be easy to forget of it during the time.

The thing is, either way; your work flow will not be interrupted at all
with this approach, it offers greater flexibility for CI systems to be
improved on looking forward and IMO actually seems less complex in a
first implementation than trying to manage this all in a single jhbuild
moduleset.

I realize my answer is quickly written and I would like to keep my mind
open and receptive, but I would also like to hear a more detailed
counter proposal on your end too, weighing the benefits of managing
this in a jhbuild moduleset vs a unified project wide git branch, how
much work it would really take and how this alternative route would
impact our workflow; for instance the first thing I can see is that I
would have to regularly refresh my integration modulesets if I wanted
to have a working build - while making jhbuild simply target
integration by default would not involve this extra step to confuse new
comers either.

This is not a super detailed proposal, but lets start with a base
minimum of at least trying to cover all the bases.

Best,
    -Tristan