GNOME.org
 

Re: Dogfooding discussion

On Wed, Jun 20, 2018 at 02:56:58PM -0400, Tristan Van Berkom wrote:

Hi Richard,

On Wed, 2018-06-20 at 17:22 +0100, Richard Maw via Buildstream-list wrote:

For anyone unfamiliar with the term, dogfooding means to use your own product,

see https://en.wikipedia.org/wiki/Eating_your_own_dog_food.


We've let our team develop how they feel most comfortable.
This predictably has not resulted in using BuildStream to develop BuildStream,
since it's a new tool to us when we start
and then changing afterwards is extra effort so a reason to change is needed.
It's been decided that now we have a reason, and that is dogfooding.

The purpose of this E-Mail is to state our intent
and solicit feedback on how we propose to go about it.

It is not our intention to advocate everybody develop BuildStream this way,
for now we're only concerned with our team,
though we hope we'll be able to recommend it as a superior experience
after ironing out kinks in the workflow.

We're interested in feedback,
since the risk of dogfooding is optimising for our use-cases,
so if our approach isn't close enough to what other users are doing
we could make the experience worse.


Hmmm, ok so my thoughts in general are: Making use of BuildStream in
the daily activities of developing BuildStream is good, and I think the
appropriate amount of that will fall into the right place if we let it
happen at the right time.

By just building BuildStream with BuildStream, we cover a very small
surface of the BuildStream featureset; it really is the freedesktop-sdk
project (and GNOME project) which ensures we get fairly good usage
coverage of our features. If developers need a real project to test the
BuildStream feature they are working on, they *really* should be using
freedesktop-sdk or gnome-build-meta as a sample.

As you mention, this could cause us to optimize for our own use cases;
worse than this, we could invent our own problems by contorting our
workflow into something which requires BuildStream, causing friction in
other areas of our development process (or inventing unnecessary
features).


Our team doesn't develop freedesktop-sdk or gnome-build-meta though,
so bringing them into our workflow for dogfooding purposes
would be the same contortion you are concerned about.


From my understanding one of the use-cases for BuildStream

is to develop a single application,
so there would be value in us developing BuildStream with BuildStream.

If there's other developers working on both freedesktop-sdk or gnome-build-meta
and BuildStream then that should work as an appropriate counter-balance
shouldn't it?


Some comments inline...


---

The key points of how we intend to implement our new workflow are:

1.  We use a container with BuildStream installed in it.

    This is for practical reasons of developers have a preferred OS
    which BuildStream may not be packaged for natively,
    and in the absence of continuous packaging effort for that OS
    the alternatives of holding back BuildStream's dependencies from updating
    or people on that OS not using the latest BuildStream aren't acceptable.

    In the absence of effort to build for other container runtimes
    we're going to be using docker, since we already build images
    and have the `bst-here` script to build on from.


I think we need to keep our install story on supported platforms
working well (which we also need to use it).

I know we have the `bst-here` script but I rather discourage it's use;
I think it is used on OSX seems to be mostly a workaround for not
having BuildStream work natively on OSX.

We of course use Docker on gitlab to ensure that we always have strong
control of the input of tests (and to diversify the platforms on which
we're testing).


2.  We develop BuildStream in a workspace.

    We currently work with a clone of buildstream.git,
    making changes there and `pip install`ing as appropriate.

    We would instead have a project that includes a buildstream.bst
    and any dependencies necessary, and use `bst workspace open`
    to have our clone of buildstream.git.

    This implies either automation of changes to buildstream.git
    causing updates to the project that contains buildstream.bst
    and us pulling changes from automation into our workspace,
    or that updates to buildstream.git will imply that we now
    are responsible for updating the project that contains buildstream.bst.


This feels like adding "build" and "deploy" to a cycle which is
currently just "edit" and "test" (using the `pip3 install --user -e .`
approach).

Doing a lot of work from within a `bst shell` open on a workspace
sounds a bit more likely...


I'm not sure I get your meaning here.


From my understanding we're intending

that others should be able to use `bst shell` to debug and test,
so if it's harder to do that than `pip3 install --user -e .`
we've got some work ahead of us to make it easier.


3.  We use `bst shell` to test BuildStream.

    Instead of having our container include all the dependencies
    that are necessary to run buildstream.git's test suite,
    we instead include those by defining them in .bst files.

    We use `bst shell` for both running the test suite and test builds,
    using `--mount` to bring the test project and your local caches in.


But as I understand it, this is currently not possible (maybe it will
be possible with future kernel developments ?).

At least there needs to be some research towards whether this is even
possible: as I understand it Flatpak cannot be run in a Flatpak, and as
we use the same container technology, I expect that BuildStream cannot
run a build launched from inside a `bst shell`.


It works if either unprivileged user namespaces are permitted by your kernel
or for the inner invokation of `bwrap` we use `--unshare-user`.

We can emulate the kind of sandbox we use in `bst shell` by running:

    bwrap --unshare-pid --die-with-parent --bind / / --unshare-net \
          --unshare-uts --hostname buildstream --unshare-ipc --chdir / \
          --proc /proc --tmpfs /tmp --dev /dev --dev-bind /dev/full /dev/full \
          --dev-bind /dev/null /dev/null --dev-bind /dev/urandom /dev/urandom \
          --dev-bind /dev/random /dev/random --dev-bind /dev/zero /dev/zero \
          --remount-ro / --unshare-user --uid 0 --gid 0 /bin/sh

Attempting the same command in the created sandbox resulted in:

    bwrap: No permissions to creating new namespace, likely because the kernel does not allow non-privileged 
user namespaces. On e.g. debian this can be enabled with 'sysctl kernel.unprivileged_userns_clone=1'.

It worked after dropping the `--unshare-user` from the command.

<snip>

---

There's also a related argument that we should be publishing distro packages.

This could take the place of using a container, but not immediately
since the current container building approach involves installing from source.

There are benefits and drawbacks of using a container
which may be worth considering when packages exist as an alternative,
but we can defer a decision on that to some degree,
since we could incorporate packages into the container build
by having a CI job between running the tests and building the container
that publishes to a package repository,
which the container can add as a package source to install from.


I think I would prefer to have not ever gone into the territory of
recommending using BuildStream within a container; it may be practical
for people in some scenarios (like Docker in GitLab), but there should
not be a need for launching it from a container as a tool on your own
host.


I agree it shouldn't be *needed* but it is convenient,
so ideally we'd provide some level of support for the work-flow.

We're suggesting we go with a container to start with
since it's something we already know works
so we can start dogfooding with less up-front effort.

It doesn't preclude doing the work to have proper packages.


---

Since we'll need a base system capable of hosting BuildStream to use `bst shell`
it's possible we could provide containers based on that instead of a distro.

We could incorporate dogfooding of the freedesktop-sdk or Baserock,
into our dogfooding story if we were to build our buildstream.bst on top.

I suspect since we may end up needing to do most of this to support
testing buildstream.git with `bst shell`,
it may be a relatively small amount of extra effort to use it for our container.


On a similar subject, I'd like to get coverage for `bst source-bundle`, 
it would be nice if we used the freedesktop-sdk project to assert that
it's actually useful for the purpose it was created for:
  o Run source bundle on the dependencies you need for a system with
    BuildStream
  o Build the cross built runtime for another arch, plus kernel
    and init script
  o Boot the cross built runtime in a VM, the init script launches
    the build script generated by `bst source-bundle`
  o Once that compiles successfully in it's vm...
  o Run the BuildStream test suite on that built in that emulator

I think we're going to get dogfooding along the way, and in the right
places; I just don't think we should rally our efforts under the banner
of dogfooding.


That sounds like too long a task to not automate,
at which point we wouldn't get any feedback on the user experience,
which we're hoping to improve by dogfooding
by building and testing BuildStream in BuildStream.
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]