Re: [BuildStream] BuildStream 2.0 - must have's

[Chandan Singh <chandan chandansingh net>]

Hi Will,

Thanks for getting back. Some comments inline.

On Mon, Jan 27, 2020 at 2:26 PM William Salmon
<will salmon codethink co uk> wrote:
> Not worrying about releases has seen bst2's development really take off,
> its amazing to see how far it has come and I am really excited about it
> stabilizing to the point were it is ready for use with more projects and
> being able to release bst2!

Me too!

> > Sandboxing
> > ----------
>
> I like that we want to move as much sand box code out of core as
> possible and moving to BuildBox as fast as possible should be high up on
> our priorities but dose this need further changes to public API or are
> there some other blockers that means we need to wait on this for bst2?
> Can further work be in bst2.0.x or bst2.x? I imaging there may well be a
> good reason to wait for this, but I know a lot of work has already been
> done.

This is essentially captured in
https://gitlab.com/BuildStream/buildstream/issues/719.

There has been significant effort in this area, but there's still work to be
done. Probably the most hairy will be supporting workspaces and interactive
shells.

> > Plugins
> > -------
>
> Plugins are really important so its not surprising that our community
> has spent a lot of time thinking and talking about plugins (not lease
> myself) because we all really care about them, as getting them right is
> key to bst's future success.
>
> Apart from actually moving them out and giving them a bit of a shake up,
> are we blocked on actually getting on and doing this? We have put in
> place a lot of infra around them that I believe is now working well enough?

The infrastructure is mostly there, although we can improve how we run plugin
tests.

The main work left here is to actually do the splitting up of the repos, but
that should probably happen _after_ the plugin API has been more or less
stabilized. This is important because we are still working on the plugin API,
and making breaking changes.

> An area of discussion that we often stumble over is plugin distribution,
> once all the plugins live out side of the core then there is a little
> more work required to build a hello world project.
>
> Many bst projects already require additional plugin installation and
> those wishing to provide packages for bst may well chose to bundle up
> some of the once "core" plugins in to the base package so i think we
> still have some good options.
>
> Dose the plugin distribution story need to block bst2 any further? I
> presume if we need to have a more integrated plugin fetching system it
> would be Additional API rather than braking and so fit in bst2.x?

I don't think the plugin distribution needs to, or is, blocking bst 2.0. This
was discussed at length in the past and we decided to not have a baked-in plugin
distribution mechanism. We agreed that we would continue with the current
approaches, i.e. either submodules or python packages.

The primary reason for not doing our own fetching system was that it wasn't
obvious how it would really work, and support dependencies etc. If someone has
nice ideas around this, please share them on the list, but I don't consider this
part a blocker.

> > CLI
> > ---
>
> We have made a lot of CLI changes since bst1.2 but looking at [1] I see
> that of the outstanding issues there are quite a few issues that have
> ended up being controversial. I wonder which of our outstanding issues
> we wish to block bst2 on?

Other than unplanned changes, this is captured in
https://gitlab.com/BuildStream/buildstream/issues/1068.

> > Performance
> > -----------
> [...]
>
>
> I have not been following the Artifact/performance work, other than to
> note that the core performance has made some leaps and bounds since 1.2,
> it has been great to see Ben et al. working very hard and delivering
> remarkable improvements, Do these have any other outstanding work that
> would block bst2?

I'm not aware of any performance-related blockers at this time. Obviously this
is one of the areas that we can never call "done", but I don't think there are
any blockers either.

> I don't want to put too much time presser on ourselves as I am sure lots
> of things will come out of the wood work as more projects jump to
> bst2/preview. However it has been noted that Ubuntu's next LTS release
> is this spring [2] so if we got a wriggle on then we might be able to
> get bst2 released in time!
>
> I think getting in to 20.04 would be a nice thing to aim for as it would
> help bst2 adoption but I don't think it is anyway near as important as
> getting bst2 right.

I can sympathize with the desire to get into next release. But I don't think
that can be promised at this time. If we finish all the blockers in time, and
are happy with the state of the API, then sure. But, what will be worse is if we
make an incomplete release and have to break the API two months into bst 2.0.

Having said that, I am quite glad to see that people are excited about
BuildStream 2.0 :)

Thanks!
Chandan