Re: [BuildStream] BuildGrid and BuildStream 1.4 and API concerns

[Sander Striker <s striker striker nl>]

Hi,

On Sun, Dec 2, 2018 at 12:26 PM Jürg Billeter <j bitron ch> wrote:

Hi Tristan,

On Sun, 2018-12-02 at 16:08 +0900, Tristan Van Berkom via BuildStream-list wrote:
> On Fri, 2018-11-30 at 16:51 -0500, Sander Striker wrote:
> > -1.  Using remote execution is optional.  It is implemented around
> > a protocol.  There are multiple implementations of the protocol beyond
> > BuildGrid: BuildBarn, BuildFarm, Scoot and Google's commercial Remote
> > Build Execution service.  Each BuildStream user or group of users are
> > free to chose which, and which stability guarantees they are
> > comfortable with.
>
> This interoperability is simply untrue until the specification
> stabilizes, and our implementation of the REAPI as I understand it will
> remain a step ahead of the specification for the foreseeable future.

While there is a risk that incompatible tweaks will still be applied to
REAPI v2, the upstream goal is that this won't happen anymore (until
REAPI v3). And I hope OS and machine architecture values can be
standardized by end of Q1/2019, well in time for BuildStream 1.6.

>
> Today for example, BuildStream will have to require workers which
> clearly advertise their capabilities in terms of OS and machine
> architecture of the execution environment, tomorrow it might be
> something else, like full support for all file attributes on the remote
> execution cloud.

For OS and machine architecture requirements it's clear that we can't
expect all remote execution deployments to support all values. I.e.,
BuildStream needs to be able to report an understandable error if the
platform requirements of the project can't be met. This generally also
applies to local builds (e.g. if user namespaces are not available).

In my opinion, it would make sense to follow the same approach also for
other execution environment requirements such as file attributes. I.e.,
specify the requirements as REAPI platform properties and the execution
service will return a FAILED_PRECONDITION error if the server/workers
can't fulfill the requirements.

While this obviously would still require server side upgrades for
projects that have these additional platform/sandbox requirements, I
think we can implement it in such a way that projects that don't need
these features will still work with existing remote execution servers.

E.g. in the case of multi-UID support I also expect that additional
host features will be required for local builds (subuid support) and
thus, this feature should anyway be opt-in.

This approach should work for arbitrary future requirements even with
existing remote execution servers. BuildGrid currently ignores the
requirements specified in the Platform message but this is a bug¹ and
hopefully fixed soon.

In conclusion, I think it may make sense to document remote execution
as experimental for BuildStream 1.4 but the plan would definitely be
for this to be temporary. I.e., I'd expect remote execution to be fully
stable by BuildStream 1.6 and interoperable with other servers such as
RBE.

+1.  This message articulates my concerns well.  I simply don't want us to give stability guarantees too early.  I regret not having been more vocal around when BuildStream went 1.x.x - I've now been bitten enough by the burden of early stability guarantees that I don't want to see us repeat something similar which even extends beyond BuildStream itself.

 

Cheers,
Jürg

Cheers,

Sander

PS.  I've been under the weather since the weekend.  Slowly getting back to things.

¹ https://gitlab.com/BuildGrid/buildgrid/issues/104