On Thu, 2018-11-01 at 23:21 +0000, Sander Striker via BuildStream-list wrote:
> Hi Jürg,
[...]
>
> This requires a cache key version bump doesn't it? If we are only assuming
> backward compatibility in terms of consuming from existing caches, we
> would generate the cache key both with the old and the new version. We
> check if an artifact exists under the new key, if not we check under the old key.
> If not present we build the artifact. We then store the artifact under the new
> version key.
We currently do not make guarantees about the cache key stability
beyond the promise that cache keys will not change from one major
release to the next (e.g. they have not changed in 1.2, and they will
not change for the duration of 1.4).
Which is to say cachekeys will not change until 2.0? Or did you mean "will not change from one _minor_ release to the next"?
There is a sensible plan for supporting cache key stability, however it
requires that we revision the artifact version in such as way as to
cope with artifacts which are formatted differently.
i.e.:
* When implementing features, we necessarily change how an artifact is
stored (most notably the metadata changes, but introduction of build
trees is another tangible example).
BuildStream would have to continue to understand every previous
artifact version, from the point of implementing this stability
onwards.
* Projects would need to inform BuildStream which artifact version they
intend to use.
* BuildStream would also have to lock down some new features to only be
available since a given artifact version, and error out gracefully.
While this is very much a long term goal for us, I have been advising
against implementing this in the short term due to the immense amount
of churn and feature additions we've been seeing in the last year.
I would advise to wait until the dust settles a bit more, when
BuildStream is less of a fast moving target, we can implement this with
much less overhead.
I think that's ok still, we can revisit post-1.4. To spell it out: it currently means a client upgrade could result in 0% cache hits. And as multiple versions of clients exist, they cannot benefit from eachothers cached results.
Cheers,
-Tristan
Cheers,
Sander
