GNOME.org
 

Re: [BuildStream] Buildtrees in 1.4 release

Hi all,

I've been meaning to make a statement over the past couple weeks, as
we've been seeing the pressure rising, but only finally got around to
writing it up today:

    https://mail.gnome.org/archives/buildstream-list/2018-December/msg00039.html

Hopefully backing out of our rash decision of advancing the release
cycle in mid-cycle will relax the situation.

I should add, this is not only for the sake of build trees, we planned
to make a lot of breaking CLI changes which ideally we will never,
ever, ever do again; and we should land these mostly simple changes
once and for all in 1.4; what is killing us in landing these features
is not the functionality itself but the tiresome bike shedding which
surrounds them, unfortunately.

But back to the topic...

On Fri, 2018-12-07 at 00:58 +0100, Sander Striker via BuildStream-list wrote:

Hi,

On Tue, Dec 4, 2018 at 1:04 PM Tom Pollard via BuildStream-list <buildstream-list gnome org> wrote:

Hi all,

On 30/11/2018 06:28, Tristan Van Berkom via BuildStream-list wrote:
<snip>

A few things to say here:
 
   * Enabling build tree creation to unblock the surrounding feature
     work was the first thing we did when early branching for 1.2
 
     Which means we have had more than an entire release cycle to
     make this work perfectly - having to back this out now, would be
     very sad.


After working on features around build trees I obviously share this
sentiment, regardless of my position of them being included in the release.


As one of the original requesters of the feature, and having seen us
wreck our brains on how it was supposed to work since January, I can
relate.  However, I don't think we should introduce behavior that
would hurt users more than they would gain.  And especially not when
we are anticipating a behavioral change in the following release.


The creation of build trees is so deeply entrenched into the code that
I really don't think backing it out is an option, or rather, it is the
most expensive option.

As everyone knows I am spiteful of any talk about "experimental flags",
lets use it at least as a strawman here: If we were even to consider an
experimental flag where everything was "optionally turned on", this is
even more technically challenging in my view than adding the missing
optionalities that are in mid flight (not only are downloads and
uploads, and possibly even creations optional, but the whole thing is
optional again - as well as dependent features which should "disappear"
when the experimental flag is turned off).

Realistically speaking, implementing any remaining optionality and
committing to it, is much less expensive than either backing out the
feature and its dependents, or making the whole thing optional.


   * I am doubtful of the importance of any option for *creation* of
     build trees, especially now that we have local cache cleanup
     in a robust enough state.


I think you're forgetting about remote execution.  If we
unconditionally create the build trees it means we need to
unconditionally capture these files on the worker, get them uploaded
into CAS.  And then we happily pull all of this stuff down into the
client in the current implementation.  And then we create the local
artifact, which we then decide not to push?
Clearly this is wasteful on resources, and not being able to turn
that off seems just plain wrong.  Especially if you know that you are
not going to, or can not, use the build trees.


I am somewhat forgetting about remote execution, and while arguments
have been made so far that remote execution is added but should be
advertised as experimental for 1.4, I agree it would be nice to have
remote execution still "work well" in it's experimental state.

That said, the point I want to make here, is that making creation of
build trees optional is a commitment, but I think not making the
creation of build trees leaves the door open to making it optional
later.

As such, I think that considering this early commitment as a blocker of
the feature, on the sole grounds that a new experimental feature might
perform better, is the wrong move.

That is not to say that the optionality of creating/caching build trees
is not otherwise beneficial, just that we can postpone committing to
this optionality if the only reason to commit to it is for a feature
that is still in the process of materializing.


     In other words, the local cache only needs to be large enough
     to store the sources, build trees and build results for a single
     full build - this is the same amount of local disk space required
     for a full build using say, JHBuild.


 

     Has anyone even raised an issue or proposed that *creation* of
     the build trees should be optional ?


My choice of words may not have been perfect, but I would
consider https://gitlab.com/BuildStream/buildstream/issues/566 to be
it.  In that we also explicitly state that "Yes we are certainly on
the same page here - I do want to see this used a lot before
introducing an escape route, but agree that we should design one at
some point before freezing things in 1.4."  That was three months
ago.


Yes, this was always discussed in terms of making the uploads optional,
not the creations. If there is something in the comments which speaks
of making creations optional as well as uploads, then I'm sorry that it
escaped me.
 
[...]

I don't disagree that we should refactor complexity.  However, I
don't think we should be making a trade between a bit of complexity
vs changing the feature to now require build trees everywhere instead
of having them be an optional for certain use cases.


The optional upload is already a work in progress, and we need to
decide on whether the creation is optional, and if that optionality is
tied into the same configuration as the optionality of uploads or not.

We have time to complete this optionality, and this optionality is less
expensive than an additional blanket optionality (experimental flag
strawman above) or backing it out entirely, I really think pushing
forward is the smallest effort of any of the choices.


That said, all of this talk is secondary to the elephant in the room;
which is that very soon the build trees are *going to be mandatory*
dependencies for elements which run `make check` on the build trees the
necessarily depend on:

    https://mail.gnome.org/archives/buildstream-list/2018-November/msg00042.html

While postponing optionality is possible if the negative impact is not
too bad in local execution (no need to commit while experimental
features are still in flux), the conversation of how this commitment
impacts the future plans for the testing elements I think is
inescapable.

We need a statement of how to reconcile:

  * I have a project with elements which explicitly depend on build
    trees in specific elements

  * I want to turn off build trees completely

Unfortunately, saying later that "turning off build trees" only means
"turning off the build trees which are not required" is very unwieldy
in practice; making the distinction of "a required build tree" is
almost impossible, it is like the having the past be informed by the
future.

Even if we loaded a whole project every time in order to determine in
advance if any *potential* elements explicitly depend on a build tree
and caused that build tree to become mandatory, it would be impossible
to determine that the test elements which you decided to maintain in a
separate BuildStream project, which depends on the build trees of the
source project across a junction, exists or not (i.e. because of
junctions, it is quite literally impossible to presume to have any
knowledge of reverse dependencies at any time).

If we can answer the question of how to reconcile this plan of build
tree optionality, with the plan of explicitly depending on build trees
in test elements, then I'm sure we can solve this in time.

Cheers,
    -Tristan
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]