Re: Dual cache key calculation and build planning modes

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

On Wed, 2017-07-05 at 15:10 +0200, Jürg Billeter wrote:
> Hi Tristan,

After thinking on this further, I wonder if we're blowing this out of
proportion with regards to worries about how unstable or unpredictable
this can be.

Just because the weak cache keys themselves do not depend on specific
versions of their dependencies, does not mean that weak build planning
modes let you assemble entirely arbitrary artifacts together: the
versions of every element will always be exactly what is dictated by
the refs in a project.

As far as I can see, here are some cases where I anticipate some
breakage:

  o When building a C library which does not trigger it's reverse
    dependencies to rebuild (but is still the exact desired build plan
    according to the project), it can be that the C library I am
    rebuilding has changed some macros and expects things to be
    recompiled.

    If this does not happen, then in any case the C linking interface
    should normally remain the same, so either:

      A.) A strong build plan would also fail anyway
      B.) Its failing purely because of a macro expansion changes

  o When interpreted languages run installations using a language
    specific installation technology which has changed, if the
    underlying language specific tools change without rebuilding the
    reverse dependent elements, then the expectations of the language
    specific tooling can be fooled.

    For instance, if python setuptools changes in some incompatible
    way, then anything which was previously installed with:

       `./setup.py install`

    may not work with the new setuptools (this is unlikely, but
    presents a possible case of breakage).

  o Maybe this poses a problem especially at the toolchain
    bootstrapping level ?

    Even then; if I can install a new version of binutils on my system
    overriding the existing one, without rebuilding the rest of the
    toolchain (which I suspect I can do with a fairly good succeess
    rate), then I could do the same in a buildstream sandbox.


I may have missed some possible failures, but let's also take a closer
look at the anticipated use cases for some added perspective:

  Developer testing
  ~~~~~~~~~~~~~~~~~
  So we want to rebuild a low level element without rebuilding webkit
  and everything else; either because:

    A.) A developer wants to quickly test a change to a low level
        component, and has a checked out workspace to build from.

    B.) A developer wants to test how a change to another module
        effects their app, i.e. lets say Benjamin makes a fix to
        GtkListBox and says "I've just pushed a fix, try it out !"

        In which case I'll want to just `bst track gtk+.bst` and
        rebuild my app against that. Maybe I'll end up building
        GTK+ master, or maybe I'll get a fresh artifact which a build
        server prepared for me within the 10 minutes that the IRC
        conversation took.

  The intent here is clear enough: Lets try this with an upgraded
  dependency / library, as fast as possible, and see if it works.

  In this case I feel like its clear enough that the result is not
  necessarily going to work properly. The distinction between
  rebuilding everything or not (i.e. picking up any changed header
  files which might effect reverse dependency builds) should also
  ideally be clear to the developer (although there is room for
  misunderstanding here I admit).


  Quick turn around in CI setups ala GNOME Continuous
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  First lets keep in mind that using weak cache key mode in CI systems
  is a feature that allows early failure reporting, at the cost of some
  false positive reports occurring from time to time.

  In this scenario we assume a case where the CI system has built
  all of the artifacts of one system once, in any case the first time
  around, building in 'weak' mode will produce the same results as
  'strong' default build mode.

  After a timeout, or provoked by some watching of git repositories,
  we relaunch a build; running `bst track` either on only the changed
  repositories, or the whole project pipeline.

  Here the tracking branches of a project should already point to
  sensible branches for the CI; possibly master or latest-stable
  of a given stack (like GNOME, but could be another), and probably
  a more rigid set of base dependencies not subject to arbitrary
  changes by `bst track` (it's not like we expect building master
  of everything from the toolchain to high level apps is ever going
  to work, but feel free to try).

  So lets say that in one CI run, a few modules change, and we want
  to build only those, and try to assemble a system, boot it, and
  run some tests on the booted VM.

  In this case we can produce some false positive "failures", but
  assuming we trigger some unconditional full builds to also run
  in the background, we're going to fall back to strong cache keys
  soon enough (and can even include in a failure report, whether
  the failure was produced from a strong build or a weak build).


Cheers,
    -Tristan