Dogfooding discussion

[Richard Maw <richard maw codethink co uk>]

For anyone unfamiliar with the term, dogfooding means to use your own product,
see https://en.wikipedia.org/wiki/Eating_your_own_dog_food.

We've let our team develop how they feel most comfortable.
This predictably has not resulted in using BuildStream to develop BuildStream,
since it's a new tool to us when we start
and then changing afterwards is extra effort so a reason to change is needed.
It's been decided that now we have a reason, and that is dogfooding.

The purpose of this E-Mail is to state our intent
and solicit feedback on how we propose to go about it.

It is not our intention to advocate everybody develop BuildStream this way,
for now we're only concerned with our team,
though we hope we'll be able to recommend it as a superior experience
after ironing out kinks in the workflow.

We're interested in feedback,
since the risk of dogfooding is optimising for our use-cases,
so if our approach isn't close enough to what other users are doing
we could make the experience worse.

---

The key points of how we intend to implement our new workflow are:

1.  We use a container with BuildStream installed in it.

    This is for practical reasons of developers have a preferred OS
    which BuildStream may not be packaged for natively,
    and in the absence of continuous packaging effort for that OS
    the alternatives of holding back BuildStream's dependencies from updating
    or people on that OS not using the latest BuildStream aren't acceptable.

    In the absence of effort to build for other container runtimes
    we're going to be using docker, since we already build images
    and have the `bst-here` script to build on from.

2.  We develop BuildStream in a workspace.

    We currently work with a clone of buildstream.git,
    making changes there and `pip install`ing as appropriate.

    We would instead have a project that includes a buildstream.bst
    and any dependencies necessary, and use `bst workspace open`
    to have our clone of buildstream.git.

    This implies either automation of changes to buildstream.git
    causing updates to the project that contains buildstream.bst
    and us pulling changes from automation into our workspace,
    or that updates to buildstream.git will imply that we now
    are responsible for updating the project that contains buildstream.bst.

3.  We use `bst shell` to test BuildStream.

    Instead of having our container include all the dependencies
    that are necessary to run buildstream.git's test suite,
    we instead include those by defining them in .bst files.

    We use `bst shell` for both running the test suite and test builds,
    using `--mount` to bring the test project and your local caches in.

4.  Having a Merge Request accepted results in container updates.

    We currently have CI jobs to create Docker containers
    with the latest version of BuildStream.

    Users of the `bst-here` script are already doing this to some degree.

    We may want to have a wrapper script for `bst`
    that periodically checks if there's an update to the container
    and notifies so that the developer can opt to update their container.

    While we can know to wait for our own changes to be included in a container
    we won't so easily notice other people's changes if we're busy on a task,
    and it would be too disruptive for updates to be mandatory.

---

There's also a related argument that we should be publishing distro packages.

This could take the place of using a container, but not immediately
since the current container building approach involves installing from source.

There are benefits and drawbacks of using a container
which may be worth considering when packages exist as an alternative,
but we can defer a decision on that to some degree,
since we could incorporate packages into the container build
by having a CI job between running the tests and building the container
that publishes to a package repository,
which the container can add as a package source to install from.

---

Since we'll need a base system capable of hosting BuildStream to use `bst shell`
it's possible we could provide containers based on that instead of a distro.

We could incorporate dogfooding of the freedesktop-sdk or Baserock,
into our dogfooding story if we were to build our buildstream.bst on top.

I suspect since we may end up needing to do most of this to support
testing buildstream.git with `bst shell`,
it may be a relatively small amount of extra effort to use it for our container.