Re: [ Revised Proposal ] Continuous Builds in GNOME

[Emmanuele Bassi <ebassi gmail com>]

So, I want to stop this thread on my side because not only is making
me not want to contribute anything ever again, but it also is
absolutely pointless busy work that relies on two people architecture
astronauting without the minimal intention of building anything or
even consensus.

I consider the matter that started this thread — ensuring that jhbuild
does non-srcdir builds by default — solved, and any talk about
continuous integration in GNOME absolutely moot until somebody comes
up with an actual CI server and tool that is *not* Continuous, and
shows that it works well enough that we can start talking about commit
and integration policies. I also consider the matter of fixing the
Continuous builds solved, in the sense that I'll continue tagging
badly behaving modules and file bugs. That is, obviously, if I can
muster caring about this stuff — something that I keep finding hard to
conjure out of thin air and my limited amount of time, given the
feedback I've received.

Instead of talking about these two matters, let me talk to you about
the only two tools we have at our disposal, and why I (possibly
naively) thought they were important enough to the GNOME project to
raise their continued working state as an important topic.

Let's start with jhbuild. I personally don't care if you're a
long-standing contributor to the GNOME project and you feel like you
don't need to use jhbuild because you either don't give a crap about
your system's state, or because you're competent enough to deal with
it; jhbuild *is* the tool that we provide to newcomers to the project.
It's the major blocker for people that come to use and wish to build
and contribute to GNOME. Since GNOME is a free software project and
contributors are a slowly renewable resource, introducing barriers
like having stuff *not* build is a perfect way to stagnate the
contributors pool, and turn enthusiasts and people willing to
contribute into people that will negatively impact the perception of
GNOME — and with good reason. Our Git repositories, and jhbuild, are
part of our developer outreach. Ensuring that potential new
contributors can follow a simple step tutorial without the build
crapping its pants two thirds into the process is the way we grow our
contributor's base. As much as I wish to retain one existing
maintainer, I'd rather have 10 new contributors every month.

Let's move on to Continuous. As much as it's a build bot, and a
continuous delivery testing mechanism, Continuous is much more, and
much simpler, than that: it's our Minimum Viable Product. It's the
closest thing we have to present to OSVs and OEMs and say: "You can
take this list of modules, with the given commit ids, and build
options; build them; and plop the resulting file system tree into a
product — and we can guarantee you that it'll boot, log in, and give
you the exact same results we seen". You literally have *no* idea how
powerful this is. It allows companies to create a whole operating
system without having to deal with the whims and policies of Linux
distributions. It gives us the ability to QA what we create. It gives
Linux distributions the ability to determine a baseline for their own
packaging, and check if something is a bug they introduced or if it's
upstream. Continuous is GNOME, and GNOME is Continuous.

Both of these things are so important because they demonstrate that
GNOME is bigger than the sum of its parts.

To return to the original topic: Milan, I understand you don't like
any policy imposed on your projects. That's perfectly understandable,
so I will consider Evolution and all its dependent modules as external
dependencies, and treat them as such. I'd even be happy to only build
the stable branches in Continuous, so we can keep the burden at a
minimum, and we'll ensure that GNOME does not use unstable API until
it's been released, but for the time being I'll just keep tagging
stuff — both in Continuous and in Jhbuild.

Tristan: I understand you don't give a crap about GNOME as a holistic
project; you've made it *painfully* clear over the years. I'd like to
point out that without GNOME working as a single unit nothing we do,
or did up until now, really matters in the grand scheme of things. The
*only* successful thing out of GNOME is the GNOME desktop environment
and its whole stack of technologies; everything else has been a
failure, mostly caused by this inane obsession of considering the
GNOME platform as a bag of loosely coupled projects that somebody has
to work hard into identifying and keep working together, granting the
position of gatekeeper to a bunch of middle men. This position is
ignorant of the history of the project, especially of its failures;
it's insulting of the work of hundreds of people who want to keep this
thing working and keep it approachable to newcomers; and completely
invalidated by the reality of what happens every day in the commercial
space. I also don't understand your opposition to keeping the only
integration branch we have, the 'master' branch, building constantly,
considering this is a basic engineering practice, but at this point I
can barely muster the strength to even care. In any case, as I said to
Milan, I'll gladly consider any project you maintain at the same level
as an external dependency, and thus just tag it. Feel free to go
through the Continuous manifest and Jhbuild moduleset and point out
which modules you still maintain.

As I said above, I consider this matter resolved, and I won't be
contributing any more time or energy to this thread.

Ciao,
 Emmanuele.


On 18 June 2016 at 06:45, Tristan Van Berkom
<tristan vanberkom codethink co uk> wrote:
> First,
>   I've been on vacation this week and have been able to put time into
> this, with the hope that next week I will be able to focus on $dayjob
> without regretting too much not being able to take part in this
> important debate. I hope we are going somewhere. Also, in closing in
> this mail, I will just highlight my motivations for bringing up this
> proposal in the first place.
>
> Sri,
>   Please just gloss over the technical stuff and read the end, I think
> you are in a position to effect change and this is written with you in
> mind (from the "Why do I think..." heading).
>
> On Fri, 2016-06-17 at 15:17 +0200, Milan Crha wrote:
> > On Fri, 2016-06-17 at 17:11 +0900, Tristan Van Berkom wrote:
> > > I dont believe you have any such functionality in jhbuild.
> > >
> > > At best, you can specify the sha1 of the commit you want to build
> > > of
> > > a given module, this would not let you single out one commit,
> > > omitting it from history in integration and at least attempt to
> > > apply
> > > subsequent commits without that one included, re-including that
> > > commit in the correct place in history only when CI infrastructure
> > > would find integration unbroken.
> >       Hi,
> > I never thought of a single commit removal from a set of commits. To
> > be
> > honest, that sounds even more crazy (not 'crazy' like 'insane'). :)
> > We are talking about master, the development version of the software.
> > If you'd like to tell me that you are at commit X and feature which I
> > added at commit X-3 does not work for you, then trying to figure out
> > why exactly you do not see that feature is a real nightmare and waste
> > of time for both the reporter and the developer, especially when you
> > want to remove only single commit from a history. Not talking that
> > following commit can "build on top of the previous commit", which is
> > quite usual.
> >
> > > The thing is, either way; your work flow will not be interrupted at
> > > all with this approach, ...
> > See above.
>
> Milan,
>     I think that if your commit broke integration with the wider GNOME
> ecosystem, that will matter to you and you will be quite aware of which
> commit in master broke it and why until it is fixed. I think we have to
> ensure that people take integration breaking commits seriously and
> ultimately, an integration breaking commit would block any stable
> release.
>
> That said, in your above scenario; a disparity between master and
> integration is a serious issue, but consider the alternative:
>
> If a new comer wants to get involved in GNOME, they probably have a
> specific itch to scratch, say with Evolution or EDS; they will either
> be able to build everything up to EDS & Evolution easily; have an easy
> experience, and be able to submit a patch to bugzilla, or, they will
> fail to build and get side tracked because the bleeding edge of master
> for all GNOME modules doesn't build.
>
> In other words (and this has absolutely nothing to do with using
> jhbuild or not) when you build everything from master and it breaks
> before you are even able to work on the issue you wanted to work on,
> this is discouraging; this means that we are offloading our technical
> debts to newcomers, forcing them to deal with our bugs before being
> able to even submit a patch they wanted to contribute.
>
> Would you prefer that a potential EDS contributor be able to submit a
> patch to bugzilla, even if it does not match up *exactly* against
> master ? Or would you prefer that your potential contributor get
> discouraged while trying to build EDS against the latest gcr,
> libsecret, libgdata or such, or just get bogged down trying to fix an
> EDS bug to adjust to some API churn in a dependent GNOME library ?
>
> I think receiving a patch for what the contributor wanted to work on is
> preferable, even if you happen to fall on a patch that doesn't apply
> exactly against master, in the 1% of the time that EDS integration is
> broken and lagging behind.
>
> [...]
> > I said earlier in the thread that the current situation works for me
> > the best. There is clearly stated what the continuous builds from, at
> > which exact commit it "stopped updating" certain module, because of
> > some breakage, and everything references the real master branch. That
> > the continuous uses jhbuild might be just a coincidence, even it
> > sounds
> > like the moduleset is targeted for the Continuous builds, when it can
> > influence an environment for any developer using the jhbuild (I do
> > not
> > use it, this is how I understood how it works in the jhbuild world).
>
> I'll just note again this is not about jhbuild particularly, jhbuild in
> master tries to build everything from master, sometimes it breaks and I
> think that's natural in the present state of affairs.
>
> gnome-continuous does *not* build jhbuild modulesets, it uses it's own
> json format for, also building everything from master:
>
>     https://git.gnome.org/browse/gnome-continuous/tree/manifest.json
>
> This is about getting the bleeding edge, or something as close as
> possible to the bleeding edge to be always buildable; for two reasons:
>
>   a.) To quickly be able to identify exactly which commits are blocking
>       the integration of every bleeding edge in GNOME
>
>   b.) To allow new contributors to be able to submit a patch against
>       our projects without getting bogged down in solving our own
>       technical debt.
>
> I can see that you think the approach I propose is wildly complex, I
> don't really agree on that; but; I think we do agree that we should
> choose the path of least resistance, the least complex path to
> satisfying all the requirements of this endeavor. But we really have to
> solve all the requirements in order to provide a reasonable alternative
> to policing commits directly in master and introducing revert noise in
> transitional periods.
>
> At this point I wont go over the technical merits of which solution
> will work, as pointed out this is not a jhbuild only issue so managing
> this in a special jhbuild module set is not really a viable alternative
> either, I'm happy to debate other simpler alternatives if provided.
>
> Instead, as I don't know what time will be afforded me to continue this
> conversation, I just want to highlight my own motivations for spending
> time on this proposal.
>
>
> Why do I think policing master is a hostile move ?
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> The original proposal works, the "lets force everyone to work closer
> together" and make master always buildable, well, it almost works
> except for the few cases where we make things difficult for
> refactorings and API churn where things will inevitably break, but
> overall it works - but it _only_ works if we all share the same vision.
>
> Why do I consider this to be a hostile approach ? This is a deep
> question, it's not _only_ because it means someone can step in and
> revert things forcibly, after an extremely short period of breakage (a
> day or two ?).
>
> No, I consider this to be hostile because we would be dictating that
> every project's priority and primary objective should be to create a
> unified desktop (or tablet ?) experience. We are basically assuming a
> sort of "play with the desktop team or don't play at all" mentality,
> and this makes me sick, really.
>
> Now, I'm sorry to pick on clocks again but really, since the early 90's
> we have had thousands of incarnations of clock applets, binary clocks,
> fuzzy clocks, we've seen it all really. Designing a cute new clock app
> for the beauty of the desktop is a nice thing to do, and it's an easy
> thing for young blood to work on - I hope to turn these new gadget
> developers into strong engineers over time, but I want the best of the
> best to *want* to play on our team.
>
> Why do people devote their time to working on some free software ? I
> mean asides from those who are lucky enough to get paid to do so, why
> do people devote their time to FOSS ?
>
> My answer; people look up to us, to the individual projects and their
> maintainership, people want to write gedit, people want to write GTK+,
> people want to participate in a project that tries to be the absolute
> best at what it does.
>
> Sometimes being the best text editor is not alway inline with adding an
> awkward headerbar which only makes sense if integrated into the GNOME
> desktop environment, for example. In these cases we need to be
> understanding that the reason we have gedit in the first place is
> because of the good graces of Paulo Borelli et al, and that they
> probably know what is best for a good text editor; we have to be
> understanding that they probably want to not *only* be the best text
> editor in GNOME, but just the best free software text/code editor that
> exists for the featureset they provide.
>
> By taking a stance that: "If your project is a part of GNOME, then
> automatically you care about the GNOME Desktop Environment more than
> anything else" is discouraging to those developers who just want to
> make the best software that exists, this is after all the spirit of
> FOSS, this is what attracts us in the first place - I can't think of a
> worse move for GNOME than to try to bring all of these contributors
> inline, and sacrifice their idealism for the sake of a fancy new
> headerbar. Now I'm picking on headerbars but really; they are really
> fine and nice; but we have to introduce things like this which make the
> GNOME experience better without for example, sacrificing the user
> experience of using our best softwares under KDE or other environments
> (headerbars in Ubuntu are getting better but were a bit of a disaster
> at first).
>
> In closing, I think we have become rigid over time and that by pushing
> the "GNOME Desktop Environment" agenda so vigorously in recent years we
> have become unwelcoming to those great minds and idealists which just
> want to create the best software that exists. Trying to reconcile
> everything under the GNOME umbrella to be always buildable in master is
> like a final nail in the coffin, it's where we draw the line and what
> was once a great ecosystem of amazing software becomes; merely a
> desktop environment with modules that are completely only self serving.
>
> Best Regards,
>     -Tristan
>
> _______________________________________________
> desktop-devel-list mailing list
> desktop-devel-list gnome org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list



-- 
https://www.bassi.io
[ ] ebassi [ gmail com]