BuildStream proposal - take two (shorter version)

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

Hi all,

    So GUADEC is upon us, and, as last time I sent this proposal out it
was overly detailed[0], I'll send out a less wordy proposal which I
hope everyone can easily digest.

We do have plans and ideas which go beyond the basic scope of the
proposal but let's take baby steps and talk about these separately.


The Short Proposal
~~~~~~~~~~~~~~~~~~
BuildStream is a new meta build system for building arbitrary stacks of
software and deploying built software in various ways. Our focus has
been on deterministic and reproducible builds, and decoupling the build
system from deployment mechanisms, while also providing a developer
friendly user experience.

BuildStream was also designed with GNOME's software building needs in
mind.

For more general information about BuildStream, see some of my previous
blog entries and documentation linked from our project page here[1].

For this proposal, our hope is to:

  o Replacing JHBuild as the developer tool for:

    - Building and publishing GNOME releases
    - Hacking on GNOME modules in general

  o Also use BuildStream to build and publish the GNOME Flatpak SDK and
    Runtime, using the same build metadata used for building the rest
    of GNOME.

As a first step, over a month ago I had setup a conversion process
which continuously takes the latest modulesets and creates a
BuildStream project which can be used to build the latest GNOME (3.26)
modulesets with BuildStream. See my blog post here[2] for full details
of how that works along with some instructions in case you want to try
it for yourself.

Currently I am also working on a Flatpak SDK deployment of GNOME
modules built from the same BuildStream project, this is not yet
integrated into the automated conversion scripts mentioned above yet,
though.

We hope that we can establish some consensus on this now at GUADEC, and
that we can also discuss plans to migrate our builds of GNOME and
GNOME's Flatpak SDK to be built with BuildStream using the same build
metadata (or "BuildStream project") moving forward.


Activities at GUADEC
~~~~~~~~~~~~~~~~~~~~
At GUADEC on Sunday, I will give a more in depth talk about
BuildStream. In that talk I will focus more on the reasoning and
driving requirements behind the creation of this new meta build system,
and then generally how it can benefit GNOME's specific building needs.

I will not go deep into the plumbing during this talk and probably wont
have time to make more than a small demo, but will try to keep enough
time for some Q&A here.

Further, on Wednesday (The last day of the "Unconference" portion) we
will have a BoF and prepare a hands on workshop where we can discuss in
more detail how this all comes together and where this is going.

Of course, catch any of us (Jürg Billeter, Sam Thursfield, Jonathan
Maw, Tristan Maat or myself) in the hallway during the conference days
and we will love to talk about this too.


I feel like this proposal is missing a few more pages of real detail
(I'm naturally verbose that way), but instead of covering too much
ground at once I'll leave it to you to ask questions and I can then
fill in the blanks later :)

Best Regards,
    -Tristan

[0]: https://mail.gnome.org/archives/desktop-devel-list/2017-April/msg00071.html
[1]: https://wiki.gnome.org/Projects/BuildStream
[2]: https://blogs.gnome.org/tvb/2017/06/01/continuous-bst-conversions-of-gnome-modulesets/